zzz[.i2p.xyz]

This page is old - see zzz.i2p.xyz

News and helpful hints for i2p. Direct comments on any of this to #i2p or zzz@mail.i2p.
Please upgrade to 0.6.1.33 if you haven't!
Quick links:
GPG key | I2P Home Page | (in i2p) | I2P Forum | viewmtn | history.txt | host add form | new hosts | stats.i2p.xyz dashboard



CURRENT TOPICS

Quick Hints and Tricks lots of updates 2008-05-21
Older Stuff



Blocklist Discussion

2008-05-27
Updated 2008-05-29

Use blocklists? Have GUI selectable subscription, and enable separately for inbound connections, outbound connnections, inbound tunnels, outbound tunnels, SSU peer tests? Or totally bogus, or harmful? Might be nice for BOGON list but other than that maybe not. Easy way to bring down the whole network if a default subscription gets compromised. Comparison of i2p router list to splist and level1 lists posted at i2p pastebin. Comparison of i2p router list to additional lists (bogon, level2, level3) posted at i2p pastebin. Sponge thinks there are malicious routers out there now, based on improvements he's seen from blocking connections at the kernel level. Worth investigating further.

Any blocking within i2p would have to be implemented at the following places in the code to be fully effective. Different places address the various possible malicious behaviors and vectors.

Certain places can be covered simply by shitlisting a blocklisted peer, these are noted.

Covered by shitlisting (?):

  1. Outbound connections (transport bids)
  2. Peer selection for inbound and outbound tunnels
  3. Floodfill send queries to

Not covered by shitlisting (?):

  1. Inbound NTCP socket accepts (immediate or wait until you get their peer ID?)
  2. Inbound SSU session requests (immediate or wait until you get their peer ID?)
  3. Drop existing connections when blocklisted
  4. Other-end inbound gateway for client connections
  5. Floodfill send stores to
  6. Floodfill accept (RouterInfo) stores of that router
  7. Inter-floodfill flooding
  8. SSU peer tests (Alice, Bob, and Charlie)
  9. SSU address determination
  10. Re-check when published IP changes
  11. Bogon overlap with private IP checks
  12. Have a peer hash list as well as a IP list?

Can't cover, anonymous:

  1. Floodfill accept stores from (can't do that, anonymous)
  2. Floodfill accept queries from (can't do that, anonymous)

Blocklists are only a part (perhaps a small part) of an array of defenses against maliciousness. In large part the profiling system does a good job of measuring router behavior so that we don't need to trust anything in netDb. However there is more that can be done. For each of the areas in the list above there are improvements we can make in detecting badness. The continuing efforts to address the reachability problem, for example, are much more effective than relying on a blocklist.

Automatic subscription to a list gives the list provider the power to shut the i2p network down. Completely. So we probably won't be doing that. Inclusion in i2pupdate files perhaps? Signed or unsigned? In-I2P subscription?

Maybe distribute an up-to-date bogon list only, the user can add if he wants.

Initial results (see i2p pastebin) show that splist is overly broad - at the least we don't want to block Tor users that are in splist. Level1 + bogon + (maybe) level2 is more reasonable?

This may have to wait for better reachability mapping so that over-blocklisting peers can be avoided for inbound tunnels.

I have basic in-i2p parsing, sorting, merging of blocklist entries, display on profiles.jsp. Shitlist for 180 days upon detecting a blocked peer. Some of the places on the list above are caught by the shitlisting. But inbound connection unshitlists, so have to implement checks at inbound to make it useful.

So what is the threat model? Traffic analysis? Crypto cracking? Or just general troublemaking (dropping messages, flooding messages, floodfill disruption)? State-level adversaries? RIAA/MPAA/Anti-P2P? For which of these are blocklists effective?

Update - code available, looking for testers, will branch and check in if somebody would like to try it. sample output showing blocked peers



Preview of The 0.6.2 Release

2008-05-15
Updated 2008-05-26

0.6.2 will contains the following significant changes. Target release: June 7-8. Version 0.6.1.33-8 in mtn contains these changes and more. Please test and report results on #i2p. Also please review my 0.6.2 TODO list for anything not yet done that you feel is important to include in 0.6.2.

See history.txt for more information.



Upcoming in 0.6.2 (Maybe)

2008-04-21
Updated 2008-05-29

NOTE - 0.6.2 is pretty much done, this list will become the 0.6.2.1 TODO list shortly.

Yes, we're ready for the next release after 0.6.1.33 to be 0.6.2, after two years on the 0.6.1.x series. Target release June 7-8. The main reason for this release is to let the majority of the network upgrade to 0.6.1.33 to fix reachability, tweak the reachability / tunnel selection algorithms based on the results, then reintroduce persistent lease selection and strict XOR peer ordering.

These are things I'm thinking about, pondering, or testing. Presence on this list does not imply commitment, agreement, consensus, ... If you'd like to work on one of these let me know on #i2p and I'll add your name to the item. Note - this is my list, there is no "official" list. See also active tickets on trac and the forum for more bugs.



The Reachability Problem

2008-04-14
Updated 2008-04-21

Confirmed - The Outbound Endpoint - to - Inbound Gateway (OBEP/IBGW) connection is the problem. All other connections are verified when the tunnels are built. Only ~ 1/3 of routers advertise NTCP, and only 10% of the remainder advertise introducers. So a little over 50% claim they are reachable using SSU only, w/o introducers. Many of them are wrong due to SSU Peer Test bugs.

This problem has been around for at least two years. However, performance improvements we have made in the last few releases have had the side-effect of worsening this problem. See below. This appears to be the problem that tino was having.

Note to testers: -20 works best after it has been running for a few hours and has identified the unreachable routers.

Some thoughts:

So here's the plan:

We will re-enable some stuff we backed out, after .33 is out and we have wide usage of .33.



Summary of The 0.6.1.33 Release

2008-03-19
Updated 2008-04-27

0.6.1.33 contains the following significant changes. Released April 26.

Note to users: .33 works best after it has been running for a few hours and has identified the unreachable routers.

See the three entries immediately below and history.txt for more information.



The Next Problem - Message Priorities? Or client bandwidth limiting?

2008-04-01
Updated 2008-04-05

Streaming lib improvements in .32-8 allow i2psnark to hog all the upstream bandwidth, causing problems for database lookups and local services. What to do?

Message priorities aren't very helpful because messages get wrapped in other messages and the priority gets lost before it hits the transport layer. So almost all messages get priority 400 of the DataMessage. Also, the NTCP priority mechanism may be too simplified to be very useful. See the I2NP protocol page.

OR Do we have to assign different clients different priorities, and bring the priority all the way from I2CP to OutNetMessage? (hard)

OR do per-client bandwidth limiting (hard)

OR just bandwidth limit in i2psnark and leave all the other stuff for later (implemented in .32-15)

In summary, I don't think that tweaking priorities will solve too many problems. But it's worth a try...



Floodfill Churn - Analysis and Fixes

2008-03-15
Updated 2008-03-23

We went down to only one floodfill for a couple of hours on March 13 (approx. 18:00 - 20:00), and it caused a lot of trouble. (Ignore that dip to 0 on March 14 in the picture above, it isn't real) I had already been researching a problem where my router couldn't contact any eepsites for half an hour after startup. It turns out that the floodfill search algorithm sent all queries to the two "closest" (in an XOR sense) floodfill peers it knew about - always.

That means for all retries, whether they were up or down, responsive or not. It didn't affect everybody - only peers who were closest to two routers that were down. And reboots don't help since the XOR calculation doesn't change. So it seems that if 3 out of 4 go down, you have a 50% chance of being dead (3/4 * 2/3). The symptom is not being able to make outbound connections. And it may be worse - a ff router can stay in the netDb for days after it goes down. Right now, we have 4 ff routers up but the netDb lists 6 or 7 ff routers total. This situation is nothing new, it's been there since the beginning of floodfill. But we never had the number of floodfills or the churn that we have now.

So I made two changes to address this:

  1. 0.6.1.32-5: Randomize the ff peers we use for search each time. This will get you past the failing ones eventually. This change also fixed a nasty bug that would sometimes drive the ff search code insane.
  2. 0.6.1.32-7: Prefer the ff peers that are up. The code now avoids peers that are shitlisted, failing, or not heard from in half an hour. (By "avoids", I mean it only tries them if there are less than 2 that are up).

One benefit is faster first contact to an eepsite (i.e. when you had to fetch the leaseset first). The lookup timeout is 10s, so if you don't start out by asking a peer that is down, you can save 10s.

There may be anonymity implications in these changes. For example, in the floodfill store code, there are comments that shitlisted peers are not avoided, since a peer could be "shitty" and then see what happens. I think searches are much less vulnerable than stores - they're much less frequent, and give less away. So I don't think we need to worry about it - but need help in the analysis. But if we want to tweak the changes, it would be easy to send to a peer listed as "down" or shitlisted anyway, just not count it as part of the 2 we are sending to (since we don't really expect a reply).

There are several places where a floodfill peer is selected - this fix addresses only one

  1. Who a regular peer searches from [2 at a time] (this fix - now random + qualification)
  2. Who a regular peer stores to [1 at a time] (random - need to add qualification, because timeouts are long)
  3. Who a regular peer searches to verify a store [1 at a time] (random - need to add qualification, because timeouts are long)
  4. Who a ff peer sends in reply to a failed search (3 closest to the search)
  5. Who a ff peer floods to (all other ff peers)

Lots more that should be done -

That's a long list but it will take that much work to have a network that's resistant to DOS from lots of peers turning the ff switch on and off. Or pretending to be ff. None of this was a problem when we had only two ff routers, and they were both up 24/7. Again, jrandom's absence has pointed us to places that need improvement.

I want to thank the people that stepped in and added more floodfills on the 13th to bail us out of trouble.

See how_networkdatabase.html for background on floodfill and the netDb, and the stats.i2p.xyz floodfill page for current stats.



HTTP/1.1 Persistent Connections and Pipelining

Older Stuff