| Process | run | @ | status |
| Freenet | 20185732 | WARN 5107 | down: 0 30261 latest 5109 |
| I2P latest | 20136419 | WARN 0.8.8 | down: 2011-08-23 |
| I2P router1 | 20136419 | W 0.8.7-0 | down: OK 4d t:1025+63+12 m:14408/224888 |
| I2P router2 | 88393609 | 0.7.5-34-rc | down: OK 45h t:411+13+18 m:52992/58828 |
| Reachability | 66 | 184/159/258 | good: 1 (1-1) |
| deadloop | 1 | 264/323 | sprongle.i2p |
| hourly | 58 | 82/215 | hiddenchan.i2p |
| probeloop | 56 | 1821/2364 | sftp.pegasys.i2p |
[ Home | Status | Graph | FAQ | Links | More.. ]
I2P router currently down!
Your IP is 38.107.179.210,
you are probably not anonymous
(i2p.tin0.de).
Old news
[2010-09-19] Connectivity was bad for 12 hours due to router failing for no apparent reason.
Here is what happened from my perspective:
For me the router was failing for no visible reason at all.
It looks like something went wrong at initialization time.
Note that I suspect this is not the first time this problem shows up at my side.
Perhaps there is some race condition in the startup code of the router?
Perhaps the CRIT is the culprit?
Saved version of
Daily graph and
last 6 hours
[2010-08-16] Updated to router version 0.8
[2010-08-15] Had a downtime of est. 10 hrs.
Router machine was unavailable for 8 hours.
There was no apparent cause for it.
It became unresponsive but still ran.
Perhaps network outage.
However I rebooted it into the recovery system
while it was not reachable.
This did not work directly, so I had to contact the ISP.
When recovery worked I did some manual checks like for filesystem.
It revealed nothing in particular.
After a kernel update to the latest one
it came back up normally.
Afterwards the router did not want to make connections.
Removing the netDb and restarting the router did the trick (aka. reseed).
[2010-07-30] Unexpected weird downtime
Weird things going on? I2P.to hijacked?
No. Just two bugs.
First, the bridge to my I2P router came down from 5:35 UTC to 12:00 UTC.
Second, the apache config here does weird things in this case.
This second (longstanding) bug now has been fixed, in future a proper 503 is generated.
The logfile output of the bridge code hit the 2 GB barrier after some years.
I was not even aware that my code has such a barrier (cannot deal with big
files). The logging does not contain any really interesting data, it just
stores some output of stdout/stderr of the tools which do the bridge.
The fix was to remove the logfile and let cron restart the bridge.
The not-so-funny other side is, that the bridge is the proxy for Apache.
If Apache detects that the proxy is down, it does not give you an error.
For unknown reason, it skips it's config for this part and reverts
to some other config. This is auth-o-mat.com here, which then does an
HTTPS to auth-o-mat.com.
No, this was not the right way to deal with such a problem, right.
[2010-05-21] 0.7.13 does a pretty good job
Looking at the stats, it is a long time ago that a router runs that smoothly.
For a -0 this is something to note as I do not remember any other release
which gave us such a stable network.
There was a nearly similar stable network period a long time ago,
but at that time my second router was there,
and this one autoupdated to the latest dev version at that time.
Read: Things improve. I dunno why, but leave the details to the devs.
[2010-05-03] Updated to 0.7.13
As the update to 0.7.13 wrecked my configuration,
this service here had a downtime of ca. 22hrs.
Now the status etc. has to recover in the next 2 months ..
It is entirely my fault. That's why there is supposed to
be a backup router which keeps up the service when something breaks.
However this here still is running with only one router,
and if something happens to it (like when I err) we have a problem.
[2010-05-03] 0.7.12 runs again. But still experimenting.
http://forum.i2p2.de/viewtopic.php?p=24268
Bridging is enabled again.
However as I am working on it, this gateway continues to be instable a little bit
for the next hours.
[2010-05-03] Router down until I get it back working.
For unknown reasons, I2P 0.7.12 does no more work, too, sigh.
I have no idea why everything fails.
Looks like I have to bring up the second router first to
find out what went wrong with the primary one.
This can take days. Sorry.
[2010-05-03] Router down, until I get back I2P 0.7.12
For unknown reasons, I2P 0.7.13 does not work at my side.
For unknown reasons, I2P 0.7.11 does not work at my side either.
For unknown reasons, I2P 0.7.12 is missing from the download archive.
And I do not dare to downgrade to I2P 0.7.10 again.
Therefore my system is down as I need to re-install 0.7.12 but cannot do so.
[2010-03-18] Updated to 0.7.12
The current version does not run very stable at my side.
I think it is a wrapper issue as it tears down the router each 30 minutes.
However it may take some time until I have sorted this out.
Another thing: Added an 504 error page, as there now is an eeproxy timeout after 1 minute.
[2010-02-25] net6.i2p.to restored .. somehow.
If you still see that net6.i2p.to is blocked, flush the DNS-caches used and try again.
Note that your ISP's DNS caches might keep wrong information for the next 48hrs.
You have the wrong DNS entries if i2p.to and net6.i2p.to refer to the same IP.
Well, after being asked, I found some way to re-enable net6.i2p, somehow.
However this "fix" is only of temporary nature as I cannot support this fix for long.
What I did is to pointer net6.i2p.to to some other host which then fulfills the request.
This other host is my primary I2P router.
So no intermediate connections, just directly handed over to Java by Apache.
This leaves us with no redundandy (however it is missing currently anyway)
and it shifts load somewhere else (St. Floriani principle).
I am not sure how long this holds. If I run into trouble again I have to stop this fix.
Also note that the host will be shut down in june this year, so it will vanish if not solved by other means.
(My replacement I2P router host cannot act as inProxy as well.)
[2010-02-24] net6.i2p.to blocked.
I really do not know what's going on,
but zillions of IPs are requesting data from net6.i2p.to.
As this destination is unreachable currently it takes ages to fulfull the requests,
therefor it floods my machine.
The only possible solution currently is to block this destination. Sorry.
If my theory holds true, that somebody used it as a tracker for internet based torrents,
then this destination will stay blocked until this type of traffic vanishes.
If you want to serve torrents, please do this within I2P and
abstain to use an i2p.to-subdomain in your .torrent. Thanks.
[2010-02-21] Machine completely overloaded.
There now is the point reached where the current setup of this machine here
is no more able to serve all the parallel requests coming from all over the
world. Therefor I have to change the setup here. This may lead to downtimes.
Until everything is running again properly we have to live with this situation. Sorry.
[2010-02-16] Update to router 0.7.11-0 complete disaster.
My router stopped working after I upgraded to 0.7.11-0.
The effect was that the eeproxy did not work anymore,
I was unable to retreive any eepsite (not even the local ones).
Therefor I reverted back to 0.7.10-0 which works fine.
[2010-02-16] privacybox.i2p.to can be faster reached as privacybox.de!
In the last time I see more and more requests going to privacybox.i2p.to
coming from IPs from all over the world. First I thought, possibly a
bot network hit me, however it looks like legit traffic.
So what the people do is use privacybox.i2p.to to access the TOR gateway there!
I really do not understand why they do not use privacybox.de directly.
Because routing Internet - inProxy - I2P - privacybox.i2p - TOR is much more
resource intensive (and slower) than going directly privacybox.de - TOR.
And it becomes more and more unreliable the more my service here is overrun.
Please note that privacybox.de still can see the IP of your request, as my proxy
adds an IP line into the HTTP headers send out to I2P (this is to allow EEPSITEs
to protect against automated Internet bots etc.). So you do not gain any
additional privacy by routing this way.
The only right way is to install your own I2P router and access privacybox.i2p
this way. (This proxy here is not to give you more anonymity, it is to promote
I2P.)
Currently the situation is not desparate, however if this traffic gets problematic,
I think about making privacybox.i2p.to a CNAME of privacybox.de. However this
needs help from the Privacybox people, as they do not allow HTTP traffic.
[2010-02-04] bookpusher.i2p disabled.
I disabled bookpusher.i2p due to a massive DDoS attack like download
pattern to this system coming from a continuous network block
apparently from Singapore (perhaps a Proxy?).
Please note that Intelligent people will install I2P for faster download experience,
instead of DDoSsing the network infrastructure with braindead settings, like trying
to use 9 IPs in parallel doing more than 20 HTTP-Request/s each(!).
BTW, the block is permanent, mainly to protect I2P/bookpusher against such attacks.
[2010-01-30] Router (was) down for unknown reason.
The machine running my I2P router became unresponsive.
The hardware reboot feature did not work,
so I informed the ISP. The machine currently is up again.
Downtime est. 8hrs.
[2010-01-16] Living with a strange Java Zombie.
Strange from the weird sense, not the worry side.
When I came home I was greeted by my monitoring which went
berserk because of my Freenet node being down.
In the kernel log there is a line telling me
"Fixing recursive fault but reboot is needed!"
(and shoot me if I knew what this means, except for the obvious).
Apparently a page fault from the Freenet Java side produced
an unkillable Java Zombie process which made it impossible
for the automated termination script to tear down the broken
Freenet service, therefor it was not restarted as well.
Downtime of Freenet a little more than 2 hours.
And now, there's a dead Java Zombie living in the box until
I decide to reboot.
[2010-01-06] Router is currently dead.
After installing a new Kernel the machine did not boot correctly.
However I cannot fix it, as the recovery mode is unsuccessful, too.
So there currently is no I2P router anymore for the inProxy, sorry.
Is up again. Downtime was nearly 34h.
[2009-11-16] Router updated to 0.7.7.18
The last few days had a lot of red spots in the statistics
due to some bug in the devel version of the router.
This was a classical Murphy: I accidentally updated router1
to 0.7.7 which (I had been warned by ZZZ!) would break my
router survaillance scripts. So I had to update to the current
development version which allows my scripts to extract all those
information again. However exactly this version had some bug.
Everything should be back to normal with 0.7.7.18 again.
[2009-08-02] Degraded service in the next days.
Due to changes in my server infrastructure in the next
days there will be only Router1 running until I manage
to bring up another Router2. For some reason I did not
find the time to set everything up properly before the
old machine went down. Sorry, but this is a free
service, so there are often quite more important things
to do.
[2009-07-17] Changed SSL tunnel receiver on Router2.
For some reason socat openssl-listen,fork starves after some
rounds (it can be more than 100,000 which is quickly reached).
So I moved to socat2, which is experimental. Let's see how
this works out. You can see this problem when my I2P-Proxy
switches to Router1 permanently. The listen loop there is
based on my own tool socklinger, however Router2 is missing
sslwrap, which is no more part of Debian today. I do not
know any usable replacement (Debian Lenny).
[2009-07-01] proxy.i2p blocked as it's administrative.
ZZZ noted me (thanks) that proxy.i2p in future is used to
access the router's core proxy to serve error pages etc.
and therefor it's better to block it from the inProxy.
Done so, master! ;)
[2009-06-13] TOR service down
My machine which runs the TOR service has a defective harddrive
and there is an error for the recoverysystem of the ISP,
such that I cannot fix the problem over the weekend.
Therefor I decided to retire the machine faster than planned.
First all important services (like web and mail) will be
migrated, then I have to find a new home for everything else.
This rest is TOR, the TOR-Proxy etc. and some debugging thingies.
Sorry for the delay.
[2009-05-21] Machine rebooted, unknown why
Perhaps an evil adversary has virtualized my machine,
or simply the kernel hung. Whatever. The machine rebooted,
and I had to activate the web service again
(it needs a password for the SSL certificate).
Downtime 90 minutes or so.
[2009-05-16] New I2P outproxy for 1 year, perhaps.
Some ISP behaved really stupid and insisted to loose money by not
terminating my contract early. So I am now left with a machine for
1 year or so which I do no more need. I will dedicate this machine
to I2P as an outproxy, as soon as it is repaired.
The stupidness of the ISP is to insist that I have to fulfill the
contract. I will do and pay my 330 EUR. However in return,
this ISP has to do following: Repair the machine (ca. 200 EUR), provide
power and traffic (ca. 150 EUR) and handle complaints due to the
machine becoming an I2P outproxy (invaluable). Plus taxes (my side
is including taxes).
Mind your wishes as they may come true! (Last word is not spoken,
though, perhaps the technical service is more clever than the sales
departement.)
Update: And so it was. The technicians retired the machine instead
of repairing it for nothing. Well done.
[2009-05-15] New blocking works.
The new blocking code which has a history (so does not block you based
on the current situation, instead it is based on your resource usage
seen previously) seems to work much better.
Now if you click on "More.. Blocked" you can see how long your IP is blocked,
this is, you cannot access names below .i2p.to or the old name .i2p.tin0.de.
If you see "cannot access DB", the database was currently updating,
so just hit reload (the web code is of lowest priority to prevent DoS).
Note that this page now is delayed 3 seconds now to tarpit braindead downloaders.
If you come from I2P you sometimes will see this count, too.
This means, my router IP is currently in the blocking database, too.
This will tell you that the inProxy is under high stress.
But it does not affect you, as only re-entering I2P via inproxy.tino.i2p
is blocked (you cannot do this with normal browsers anyway).
There still is the second line of defense, which will NULL IPs which
have a request rate higher than 1 per second. This kicks in when
9000(sic!) exzess requests are seen. Thanks to the new tarpitting
this shall be needed more seldom now.
There is a new FAQ entry to explain that download accellerators
now hinder you to download things over this I2PinProxy.
[2009-05-13] Machine under stress.
I had to take down the proxy for several minutes to fix the blocking
code, as several parallel downloaders brought this machine into
high stress. Now the blocking time (for the first blocking level)
is increased dynamically by the number of connections seen, so to
release a lock you have to stay calm for a while (this means: hours).
Before, the blocking was released as soon as no connections were
seen at some point in time, which lifted the blocks apparently too early.
However this still is just a hack, as there is no good way to count
how much stress is coming from which IP, as Apache does not give me
any good method to do so.
Also I fear that the machine is no more capable to handle all the load of the
I2PinProxy, so perhaps I have to move this service to another machine.
However I currently have no good idea for a replacement.
AFAICS I have to start experimenting with round robin DNS and
offloading to a dedicated VM running on a virtual IP. We will see what
I can come up with quickly.
Sadly I do not know any good "reverse proxy" server software,
as Apache is plain overkill and not really suited to the task
and Squid is a horrible piece of code for this task.
Let's see, perhaps Twisted Python can help to create such a beast.
Note that the web pages can stay where they are, because the stress comes
from all those requests from Internet to I2P and not for the web frontend
(and some anonymity SSL gateways which were not really meant to be for the
public, but I don't want to shut them down as quite many people seem
to use them.)
Some sad record: Load-Average spiked over 50(!) today. Usually it is at 2-4.
This load made the machine to block everybody after the first(!) request
to reduce stress (which only partly helped). Also all tunnel connections
to my I2P routers were eaten up, and 250 Apaches (maxed out) were trying
to keep up with the load. So everything flooded here.
Perhaps it now is better, we will see.
[2009-01-22] Textmode-Status now lists last probe code
brother@mail.i2p recommended to add the category of the site to the text mode page.
This was easy to implement. It is shown only if there is no sort done, though.
Note that if you select a code (i. E. 200), not all destinations must have the same code,
as the last probe can differ from the cosen code (selection checks if a certain code
happened in a certain time duration, not just the last probe).
[2009-01-26] *.b32.i2p destinations are disabled
As the new *.b32.i2p destinations cicrumvent the blocking which
was implemented here, I disabled them until I can resolve the issue.
To fix it I have must rewrite a good portion of how this proxy
works internally, and I think this will take very long.
So for now, you simply cannot access URLs which end on
.b32.i2p.to.
[2009-01-22] FAQ section opened
Because Sponge (sponge.i2p) asked me some questions,
I added a FAQ section. In this section I will answer publicly
some questions related to this inProxy.
[2008-12-22] Reduced logging a lot
I have calculated that implementing proper Vorratsspeicherung (VDS)
conformant logging would cost me around 90000 EUR one time cost
plus 1500 EUR/day (asking IBM) or 2500 EUR/month (doing it myself).
This means a minimum monthly cost increase by 50000% or so at my
side for this I2PinProxy here, let alone the one time cost.
Therefor I decided not to add logging at all to my I2P-routers
and risk being sued in case our Federal Constitutional Court will
not declare the VDS void. But I am confident it will.
Having no logging at my routers there is no additional benefit by
doing logging on my I2PinProxy. However looking at the logging
I found out that I do not need a lot of the data which was logged.
So to clean up resource consumption (and therefor manual effort
to clean it up regularily), I did remove logging where it was not
needed and such reduced it to the bare minimum where it was needed
(like for blocking IPs overloading this service here).
Marry Christmas!
[2008-10-31] 3 years still beta.
I think it's time for a change. For you this means, this proxy will change it's
IP to a more powerful VM on one of my other servers. However this must wait
until I managed to teach VMware server the most basic security features (one
property of software from the US seems to be the lacking of even the most
primitive security features, like: Never open ports on public interfaces.
Fixing this with netfilter is like letting a bleeding wound bleed,
but put the person behind glass such that dirt cannot do harm. But glass
can be broken quite too easily, so I want to close this type of wound).
[2008-09-13] Bugfix in statistics view (due to history system rework)
Well, changing the history system introduced a bug which made the hist display
in the Status page, as well as the history click (reachable via the "h" link)
show outdated data, as it only accessed the old history, not the new one.
This now has been fixed, hopefully. Note that the color of the "Time" column
in the Status page still was correct (it did not rely on the history table).
[2008-09-07] Working over the history system - Part 2
The history table now is switched to the new location. The new table only
contains two months of the history today. So if a resync is needed this will
fail for some time until it has been fixed.
Note that at the end of the history rework a complete resync will be needed.
[2008-09-06] Backup I2P router running again
Updated to current 0.6.3-3. This is a historic entry.
[2008-09-05] Unexpected Downtime (6)
I am still investigating the real source of the problems.
Therefor logging increased, webserver reconfigured a bit (nicelevel, maximum number),
also started to use file locking to prevent some concurrency.
Perhaps it helps to run some scripts serially instead of parallely,
hoever this might slowdown some requests (or speed up, will see).
[2008-08-28] I2P destinations are now pointing back to I2P if comming from I2P
One of the longstanding bugs (it was a big annoyance) is now removed:
If you access this proxy from within I2P, the links to eepsites now no more point outside I2P.
This new feature has not been debugged thoroughly yet, so let's see if it works.
If you find bugs, please leave me a message at my pager, it is at
http://hydra.geht.net/pager.php
or
http://www.tino.i2p/pager.php
Also done: Rework of the menu, as the menu became too long.
Now it is splitted into several independent parts.
I am not completely happy with it, but it is an improvement.
[2008-08-23] Operation instable as my backup I2P router is down
My backup I2P router is the stable one of my routers. For some unknown
reason my primary I2P router is very instable in the 64 Bit environment.
However the machine with the backup I2P router does no more boot for
unknown reason. The 2.6-kernel rescue system fails for unknown reason,
and the 2.4-kernel rescue system is unable to revive the boot process.
Until this problem is fixed there is no backup I2P router. And therefor
the operation of this I2P proxy is somewhat instable currently.
The main I2P router (the one with my eepsite, too) only stays up for
minutes under the load and then crashes and is rebooted. Sigh.
[2008-08-13] de-ebook-archiv.i2p blocked.
On request I blocked de-ebook-archiv.i2p.
See my post to forum.i2p for slightly more information.
[2008-08-07] "deadloop" added
Now expect long red bars in case a destination goes down!
This feature is experimental and likely to change.
The deadloop scans "dead" destinations more quickly in future.
The idea behind this is, a failing destination which was there for a long
time will be scanned longer than a destination which never showed up.
For now it must run for several weeks until it cuts down the number of
failing destinations with a not yet high enough failing probecount.
Perhaps I will start more than one such deadloop, to speed up the process.
[2008-07-14] Unexpected Downtime (5)
Again down. I did not find any trace why the machine died.
This one was a little bit non-symptomatic, as before the machine kept up pinging,
however this time it died completely. Perhaps it was EM-Interference (lightning).
If it continues, I have to improve the logging.
[2008-07-12] Working over the history system
The naive implementation of history now grew the db over 500 MB big, so it is time to rework that.
As a consequence the graph resync process will not work until it is fixed, too.
While changing all this, statistics information might be affected.
Everything else shall be unharmed in the meanwhile.
[2008-07-09] Unexpected downtime (4)
Again a cresh due to OOM. Still no cause found.
However I think it's a cascading effect.
Something hangs, causing processes to be created which then eat up memory.
Also I think it is triggered from outside (people are overloading this proxy).
This may be due to insane settings, or just because browsers open too many
requests in parallel. Will see.
Machine now does internal logging of processes etc.
Hopefully this will reveal the problem when the next crash occurs.
[2008-07-05] Updated pages
Status page (and similar) now automatically switch to text mode on big pages ("All" and "504").
Text status now can be sorted, too.
The currently actively used router is shown bold in Process info box now,
which get's rid of the red bar in case the router2 is used.
[2008-06-22] Updated to 0.6.2
Router1 (which drives my eepsite and other destinations) now updated to 0.6.2, too.
I had to update (improve) lots of my scripts to do that.
Now "t" shows: Participating tunnels+In+Out client tunnels
[2008-06-10] Unexpected downtime (3)
I am coming closer to all those downtimes. As it looks like,
the machine runs out of free RAM. Still not knowing what is causing this.
[2008-06-10] Backup router running 0.6.2 temporarily.
0.6.2 seems to have introduced some features which make it hard for me to run it.
So I am not completely convinced that I can upgrade to 0.6.2 without problems.
Therefor I will run the backup router on 0.6.2 until I have some data.
Until July I hope there will be enough status information to see if this
version is successfully running in my environment or not.
[2008-06-09] Update to 0.6.2
failed! (Updated)
The console was changed in 0.6.2, therefor some of my checkscripts seem to be unable to detect that the router is live and healthy
(they look at oldconsole.jsp do catch the various counters and tunnel states).
I think it is because the router needs too long to participates on tunnels now.
Effectively this hinders my router to work properly, as it is killed all 10 minutes or so by the monitoring system as it is considered "hung".
Until this is resolved I reverted back to 0.6.1.33.
If you have similar problems, downgrade to 0.6.1.33 is possible manually:
It's like a manual upgrade, but download instead:
- http://mirror.i2p2.de/i2pupdate_0.6.1.33.zip
- http://mirror.i2p2.de/i2pupdate_0.6.1.33.zip.sig
Note that your mileage might vary, so perhaps the problems are solely at my side.
[2008-05-15] Unexpected downtime (2)
Again for unknown reason the services on the machine became unresponsive.
As I was unable to SSH into the machine (ssh never prompted) I had to restart it.
No traces why this happened again. As httpd still was operative (but slow as hell)
I am really puzzled what happened.
[2008-05-12] Main router upgraded to 0.6.1.33
Also I renamed this proxy from "Tino's I2PinProxy" to "I2PinProxy",
as I think nobody cares about who operates it when accessing i2p.to.
And there still is the imprint.
[2008-05-09] Unexpected downtime
For unknown reason the LOAD of the machine went up to over 800 and it became unresponsive.
The trouble started around 8:00 UTC. At 10:00 UTC I rebooted the machine.
However it took until I was home again at 14:30 UTC when I managed to get the machine running.
[2008-05-06] Hopefully the 404 phenomenon is fixed now. (Anatomy of a BUG)
Note: Most 404 destinations are removed now and hopefully never show up again.
The fix involved a short downtime period of the database and a graph resync.
Doing this fix some (minor) historical data might have become deleted
(the gathered state according to the last code, this is the first table
before the long one in history. This data was not backed up, sorry).
On this proxy, in the status, there are 45 sites in state 404.
You can see them as a blue ribbon in the graph, too.
I am puzzled where they come from, as they are not in the addressbook of my routers
(that's why they respond so quickly and are listed under 404).
However the router's addressbook shall be the source of all the destinations here.
That's weird. I'm scared ;)
OK, just kidding. It looks like some unknown process concatenates two destinations,
and adds them to the database. Plain rubbish which should not be there. Or to say:
There's a BUG somewhere I did not find before.
The investigation shows, that I concatenate two host lists. The first is from the
main router, the second from the backup router. The list of the backup router
(currently) starts with tanstaafl.i2p. And if the first list does not end on a
newline (LF), well, you guess it, this will concatenate the last line with the first
of the second list. But why is a LF missing? It's because of some lazieness.
To get the list of destinations from hosts.txt I remove everything after the first
= on each line. Now if the transfer of the first list terminates prematurely,
there is a high probability, that the destination is transmitted, following the =,
but the rest of the line is broken within the line. So this line has no LF!
As I use regexp to remove the =, everything up to - but not including - the LF is
removed and the line is output. If there is no LF, it's still missing.
So this introduces an incomplete line without LF which is concatenated to the first
line from the second list. Voila, we got our rubbish!
Well, this bug now is removed, and I killed the wrong destinations from the database.
Therefor the number of destinations has shrunken and the history of the rubbish
has been wiped out, too. This here is only to document that fact.
[2008-05-05] Backup router updated to 0.6.1.33
Sigh .. for some $*^%!@ reason I did not came around to upgrade anything until today.
Mainly because I seem to be too stupid to get monotone under control to access the
current devel repository.
So I now upgraded to 0.6.1.33 (I have missed it was released until today, sorry).
However the history graph now looks very smooth since 2008-05-01, so I think
it's really a good thing to update. Let's see how this works out.
The reachability still is at "WARN" state as long as the 4week-flap-count is higher
than 3 times the "ok"-count (Reachability shows ok/2d/4w where 2d is the 2 day flap
count and 4w is the 4 week flap count), this may take 2 more weeks, though, as it
looks at the last 28 days. In the yearly graph you can see 4w declining fast (the
big purple line).
And perhaps sometimes I can re-invent a big enough hammer to fix my monotone here ..
Note that my backup router is meant to be at the devel head and the main router
to stay a the latest release (or manually updated to a stable devel head).
With CVS (which I know for nearly 15 years) this was easy (for me),
but I am still fighting it out with monotone to fit into my (very special) needs.
[2008-04-21] Please
upgrade your routers to 0.6.1.32-20!
As you can see in the graphs the network starts to become much more stable again.
ZZZ explains what was changed.
It looks like this was successful. A big sigh!
I will now upgrade my Router2 to the current Devel Head again (0.6.1.32-20)
and if reachability stays as expected my main router will follow soon.
(Note that, for unknown reason, the I2PinProxy prefers the backup router.)
[2008-03-27] Main router up again
One of the two SATA drives failed. Luckily it is only the secondary one which
keeps backup data. It did not fail completely, however I think it must be
replaced soon. More unexpected downtimes are possible because of this.
Note that the main router is responsible for feeding new destinations into this proxy.
As long as it is down no other destiation sources are polled.
Also my own eepsite and other destinations are handled by it.
[2008-03-26] Machine with main Router (Seed) and Freenet-Service crashed.
The machine running the primary router, my eepsite, the netDB seed and Freenet
crashed for unknown reason a few minutes after midnight local time.
It still pings and accepts network connections, but then it does nothing more.
Looks like the harddrive is inaccessible which might be a kernel problem or a harddrive crash.
I will try to revive the machine tonight.
[2008-03-22] Network currently very instable
I do not know what changed last Sunday (2008-03-16 05:00 UTC).
But from then on things started to become worse.
I downgraded my routers on 2008-03-13, so this cannot be the cause.
Since march my routers start to shutdown "CRIT" without any cause very often.
Indicator is a 'PM org.mortbay.jetty.Server$ShutdownHookThread run',
whatever this means. It is like some unknown remote process makes my routers
to stop and then the wrapper then restarts the router.
I have an internal router restart mechanism if the JVM stops working,
however this is not involved into this process. Also there seems to be
no relation to any log messages (there are simply none) or other local source.
However this also does not seem to be related to the problem observed directly,
except this problem harms the complete network badly.
[2008-03-14] Running stable on elder router versions again.
ZZZ wrote on zzz.i2p "we went down to only one floodfill for a couple of hours on March 13, it caused a lot of trouble.".
(Go to zzz.i2p to see this message.) Perhaps this explains the trouble seen yesterday,
but I do not think that this explains everything in detail.
Restarting my 0.6.1.32 did not help at my side, but reverting back to 0.6.1.31 had immediate success.
So I think that there shall be some fallback in the code in case no floodfills are reachable.
Therefor I will wait with updates until forum.i2p.net is back up again,
as I want first read what others say before doing so.
As 0.6.1.32 is backward compatible, no big harm is done.
One funny side note about something I do not have any idea why:
This proxy always operates on the backup router. This is the machine with less power,
less RAM, less everything, even less bus width (the main router has 64 bit).
However the Backup is far more stable.
So when the problem struck at the backup router, switching to the main
router did not help, because the main router already was broken, too.
Two out of two (with different hardware) let's me think, that 0.6.1.32 was the source
of the problems somehow.
[2008-03-13] I2P router trouble. Reverted back to 0.6.1.30 and 0.6.1.31.
I really have no idea what's going on.
i2p-projekt.de is out of service and forum.i2p.net is unreachable.
And now both of my I2P routers (version 0.6.1.32) started failing.
It looks to me that version 0.6.1.32 has a bug.
Therefor I reverted back my backup router to 0.6.1.30 and it reintegrated
into the I2P network (I was able to reach nearly all destinations), but my
first router (under 0.6.1.32) was unable to reach destinations on my backup router.
This made me downgrade my primary router to 0.6.1.31, too. Let's see how this works out.
If you are puzzled where you can find the old version:
- 0.6.1.30 is
still on my mirror
(as I did not get around to upgrade the mirroring to the new sites yet)
- 0.6.1.31 can be found at
http://code.google.com/p/i2p/downloads/list.
[UPDATE 2008-05-05: it's on the deprecated list now. Note that with 0.6.1.33 out it looks like the problems are solved.]
I think it's enough to grab the i2pupdate.zip (and check the .sig, of course) to downgrade as well.
(If you ask me how to check the .sig, I really have no idea. Or moreover, I know how to check
.sig files, but I really have no idea what a good .sig shall tell me! It just tells me, that
the file matches some public key. And now? How do I test this public key for correctness?
However I am telling you that to check such that you do not need to trust me.
It's unlikely, but if somebody else would have cracked into my server and changed the file,
I do not want to be the source of problems at your side. OK?)
If somebody is curious: I still have the sources of 0.6.1.30-20 around.
This is because Router2 automatically upgraded against the CVS "the old days".
And this process then starved when i2p.net came down ..
[2008-03-11] Updated Routers to I2P 0.6.1.32.
Historic entry, just to note. It maybe that the day is 1 day off the told date.
[2008-03-02] Updated Router2 to I2P 0.6.1.31-0, Router1 will follow soon
As the I2P CVS went down I tried to upgrade my updater process to Monotone.
However this currently fails because Ubuntu 6.06 LTS does not support Monotone version 6 yet.
Therefor my update process does not yet support the development head.
[2008-01-29] Bug in freenet server check removed
The freenet server checker did not detect the case where the freenet status starved,
this is, always stayed the same. This led to a reporting of the last state instead
of reporting that something failed. Because freenet was down for nearly 20 hours
but this was not detected automatically this bug now has been fixed.
[2008-01-24] Added i2host.i2p as source for destinations.
Actually I created a cron job which pulls new hosts from i2host.i2p and writes those into
a local hosts.txt file which then can be added to the addressbook.
If you are interested in the script, it is at http://tino.i2p.to/i2host.sh
(Note that you have to create an empty i2hosts.txt file first, else it bails out.)
I run this each 8 hours from cron now.
[2008-01-20] www.i2p is down, see
see forum.i2p.net
This affects dev.i2p and squid.i2p, too. The downtime now is over a week.
And it seems to be unknown why it takes so long to get i2p.net up and running again.
It's now down for 7 days and the net starts to crumble a bit, see Graph.
However I think it's only bad luck, as there now is missing a good seed source.
Therefor I added a seed source at
http://i2pdb.tin0.de/
[2007-12-21] http://trevorreznik.i2p/hosts.txt added as source for destinations.
As orion.i2p is down now for nearly a month, I decided to add a third source for hosts.txt.
(FYI the first one is from dev.i2p.)
However I now have a lot of destination conflicts as there are different keys in
those databases. I have no plan yet how to solve this, but perhaps I must add a
destination mapper method to map the names free from conflicts. This would need
to fundamentally change the way this proxy works, as it then no more checks names.
Instead it must operate on destinations.
This way would also solve trouble with destinations, which have a name which is
incompatible(!) to DNS (and therefor cannot be reached by this proxy yet).
This method would work with two new loops instead of two:
- The first one checks live destinations each hour as before.
- The second one checks dead destinations in a loop as before.
- A third one would try to reach destinations for the first time
and add them to the live ones as soon as they can be reached.
If the destination has not yet assigned a name it will get one,
which might be different to the wanted name (if that is already taken
or would prove incompatible).
- A fourth loop will be for fast checking destinations.
This works like the third one but has a significantly reduced timeout.
If a destination cannot be reached within a day or so it is handed over
to the third loop.
- A fifth loop will remove a dead destination from the second loop
after a year or so and add it to the fourth loop.
Well, leave this to next year. But one important note:
Probably I will drop support for names of the form numbers.i2p,
where numbers is a number counting from 0 to infinity.
All destinations then get such a fixed number when they are added the first time,
regardless of reachability. This way this number never changes in future.
Also note that the number will be a hash (initialized by the destination key),
not a count, so you will not see any real sequence.
(The hash will be something like:
Use dest as a number, have some fixed primes p[n] which are all different.
base[0]:=0. n:=0.
loop:
n:=n+1.
base[n]:=base[n-1]+p[n].
k:=base[n-1]+(dest%p[n]).
if hash[k] is not empty then goto loop.
hash[k]:=dest. end.
This way you will see increasing numbers which are not sequenced.
Also note that you can speed up this algorithm, because if you know that
all slots of a particuliar prime are populated, you can start with a higher n.
As destinations are cryptographic numbers the probabilty that all slots
are populated is very high, else the destination generator would have a bug.
Even in this case you can rise the starting n, then not all slots are populated,
which is not needed at all. So if you happen to loop more than 10 times, you can
just rise n for the next invocation. This way this algorithm is nearly constant
in time. FYI: Always try to use algorithms with O(n)=1. If you cannot
do that and limit the numbers somehow, try O(n)=ld n, O(n)=n or O(n)=n ld n.
Abstain from algorithms with a higher complexity. Always check your software
with counters bejond the billions. And tell this Microsoft. For example:
Outlook only allows 2 GB of mail storage. At my side this limit is reached
within 20 hours ..)
[2007-12-15] This proxy switches name to
http://i2p.to/ soon.
Merry Xmax I2P from Tino.
[2007-11-19] Announced downtime 2007-11-21 22:00 UTC.
According to my ISP the server will not be available from
22:00 UTC to 07:00 UTC of the following day. The system
will shut down 30 minutes prior to the downtime. However
I cannot take it online, as I am at work, and it has a lower
priority than other servers I own which go down, too.
So do not expect this system to be available before
2007-11-23 again. Sorry for the delay.
[2007-10-09b] Minor issues and bugfix, updated to 0.6.1.30
The proxy tunnel starved again, perhaps because I left not
enough time between upgrading my two routers. The code was
considered to recover itself, but due to a bug this did not
happen. This bug now has been removed, so hung connections
shall cleanup themself now as expected.
Also thanks for noting inproxy.tino.i2p in I2P's readme.html!
It was already done for 0.6.1.29, but I wasn't aware of ;)
[2007-09-29] Changed networking parameters
If this machine becomes unresponsive, try again 15 minutes later,
as the new parameters then shall have cleaned up the mess.
Before it took over 2 hours to recover from such an "attack".
In detail:
As a countermeasure against broken clients out there which
flooded this system with connections in CLOSE_WAIT state
this machine now has a very short keepalive timeout. The
/proc/sys/net/ipv4/tcp_keepalive_time now is 120.
This means, after 120 seconds inactivity keepalive messages
are sent and if those are not answered within 10 minutes the
connection is teared down. Perhaps I make this timing even
tighter in future so that the grace period goes below 5 minutes.
[2007-09-15] The torcheck feature was removed.
On 2007-07-07 I installed an updated version of the "dig" tool from
the official bind distributon. Sadly, the ISC version was not compatible
to the SuSE dig which went unnoticed by me. The result was, that no
TOR IP was detected anymore. Not a lethal bug, though.
Anyway, I detected that the TORcheck list I used (tor.dnsbl.sectoor.de.)
ceased to work and I was not able to find a proper replacement.
So I disabled this feature again until I can find something reasonable
that works in a reliable fashion. It should be called 'anoncheck'
anyway, as only checking for TOR is suboptimal.
[2007-08-24] Bug in the unblocking code has been fixed.
As unblocking takes place some hours after the block, it was not tested.
And therefor it did not work, I'm sorry. Those IPs which got blocked were
never unblocked. This bug has been fixed now (hopefully).
Note that 6 IPs were affected in the last week.
[2007-08-19] IPs with excessive requests are now blocked on IP level.
People do not want to hear. So I had to define more tight countermeasures.
The solution is easy: IPs which do more than 1 request per second in average
are now blocked on IP level, this means, these IPs are locked out of all
services (including primary nameservices and other webservices).
This way those abusing IPs cannot continue to excessively consume resources.
Note that it is nearly impossible to do that many requests if you access
this proxy with sane settings, as this blocking kicks in if you exceed your
limit with 9000(!) requests. So if you are blocked, just wait 2 to 3 hours,
and you are back in business.
If you change your IP to escape this filter: Your system will quickly reach
the given limit again and thus is then blocked again, automatically. So
do not try to circumvent the blocks, it's futile, just set your system to
sane values (ONE downloading thread maximum) and this system will never
(hopefully) harm your downloads. If you do normal browsing, this block
shall not affect you ever, too.
[2007-08-13] Freenet server crashing.
For some reason the freenet server (Java service) repeatably crashes.
It first crashed with a Buffer Overflow due to some bad network packets,
then, after updating the seednodes.ref, it crashes with OOM while booting.
I increased the memory from 700 to 900 MB. However I am not able to keep
the memory usage this high on this machine, as freenet then uses up more
than 100% of the memory this way. Because of this I reduced the filestore
from 99GB to 90GB. Will see if this helps in future.
[2007-06-08] New manually compiled kernel installed.
As there still was this resource leakage (related to file descriptors)
which made the system become unresponsive after 4 days or so,
a new kernel has been compiled and installed now.
Also /etc/sysctl.conf entry fs.file-max reduced to 25000,
and the affected system resources are now added to my central monitoring.
Let's see how this works out. (Apparently this one works correctly.)
[2007-05-14] Again downtime of about 100 minutes: Memory leak at the kernel layer.
There apparantly is a memory leak at the kernel layer in this machine somewhere.
I really don't know what happens, but after shutting down all processes
with only the minimum processes active I had 700 MB allocated on a machine
with 768 MB total virtual address space. This all started when the machine was
rebooted some months ago and therefor came up with a newer kernel.
So I tried an older kernel, but the machine did not boot with it
(thanks to missing modules deinstalled by SuSE), so I had to use the recovery
system to re-install an elder kernel.
The high stress in the last days combined with the memory leak got the
machine to "go over the edge" quickly. However when the problem struck
today the load was very normal (the blocking was successful), so I was able to
track down the problem a little bit further. Now the machine runs again with
an older kernel. Let's see what future brings. If it still does not help,
I will consider to compile an own kernel etc. (or upgrade this machine to Debian).
[2007-05-11] Found and corrected source of recent downtimes (I hope).
Currently IP 61.171.21.209 is stress-testing my server. From this
IP over a thousand(!) connections can be seen here permanently:
1890 61.171.21.209 SYN_RECV=28 ESTABLISHED=2 FIN_WAIT1=21 FIN_WAIT2=5 TIME_WAIT=1831 CLOSING=3
Luckily, most of them are in TIME_WAIT, that is they are already
closed and besides of this listing they do not consume much resources.
However please observe the high number of SYN_RECVs, too, so this
moron tries to open over 20 connects per second to my web server,
which shall only have 25 threads max!
Due to a race condition my bridging code was not able to completely
block all requests, so this person found out that by raising the
request rate to insane levels my block could be bypassed a bit.
However this bug now is fixed. Over one million(!) requests
now have been successfully blocked for the last minutes(!)
from this single source, and not a single one bypassed the block.
But no, I am not angry (only a little bit because some moron out
there abuses free resources this way, and no, this is not relieved
because this IP is from China and thus they shall not use I2P).
I am not particuliar angry, because this gives me the opportunity
to make this service more robust against any type of abuse.
Doing so in an automated way is far better than complaining anywhere
because some moron misbehaves.
Please note that, thanks to this person, I am now able to develop a
second line of defense based on route blocking. This is because this
futile attempts nearly use up 2 GB traffic per day(!) or nearly
10% of the free traffic of my server. Automatically blocking the
route will free this traffic for more important things than waste.
To stress it: These blocks are neccessary for two reasons:
First, my machine is unable to cope with such a load, as this
machine only has 256 MB RAM, but even if I put in 10 GB RAM
this would overwhelm the machine, as you can only handle this load
through commercial expensive load balancer techniques (you need to
distribute the load on N+2 machines for N such morons).
But more important, this might overload destinations within I2P,
as these destinations will see hundreds of connects comming from me.
I do not like to cause trouble at others, therefor my countermeasures.
[2007-05-04 and 2007-05-09] Downtimes because of resource shortage.
For unknown reason the machine came down on those two dates because
all RAM was used up. The cause for this resource shortage was found
in 2007-05-11 I hope.
[2007-04-27] Machine was unavailable for approx 7:20 hrs.
After 702 days the machine became unresponsive because of resource
shortage. It went out of file handles so that I had to reboot
the machine. Unfortunately there was a new not yet activated
kernel installed which booted but was not able to activate the
networking. The error was tracked down to a missing module in
initrd. Originally the file was there, however by installing
the new kernel also the initrd of the old kernel apparently was
rebuild, too, and thus showed the same error (thanks for this
faulty behavior goes to SuSE and perhaps partly to fou4s).
With the recovery system the settings were corrected, such that
the current kernel now loads the needed network drivers.
I think the missing setup was in the machine since it was installed,
it was harmless until mk_initrd was called and thus needed drivers
were no more loaded. Please also note that I currently don't know
why all filehandles on the machine were used up. Logfiles did not
help in this riddle. I have to extend logging to be able to track
down such errors in future (that's what this is all about, right?).
[2007-04-25] A too high request rate now blocks your IP.
Abusive access limiting now is in place. If you open too many requests
in parallel, this proxy will block your IP until your request rate
reaches sane values. Note that you can prohibit this from occurring
by making sure to not leave many open connections to the web port
of this proxy. So be sure your software correctly terminates the
connection before you open another one.
Broken firewalls tear down connections which have no traffic after a
while without closing the connection properly. In this case you
can end up blocked even if you only do single connections. If youj
leave open 50 "single connections", you use up 50 web service threads,
which is too much from one IP (and thus further requests are blocked
by this proxy).
You can tell the block is active if you are transferred to the "block"
page of this proxy when you try to access any eepsite through it.
The block automatically is removed if your IP stays silent for a while,
such that the pile of connects time out (which can take some hours).
Note that this is a fully automatic process. So you cannot even
escape it by changing your IP or using several IPs in parallel.
This proxy starts to block IPs as soon as it becomes overloaded.
So if you see a block you overloaded this service and trying to
circumvent this block only overloads this service further. So the
blocking kicks in even faster and becomes tighter. All you can
do is DoS this service, you cannot improve your throughput.
The only way to increase the speed is to install I2P at your side!
And this way you do not need this proxy here any more, as you are
then directly connected to the I2P network. Note that installing I2P
nearly is a one-click process. (Well, all you must do after installing
I2P is to configure your browser to use I2P as a proxy.)
[2007-04-19] I reactivated I2P bridging again.
Sorry, but some people startet to overwhelm this web server here with
more than 150 parallel downloads, such that this system here was no more
able to handle the number of connections. As the I2P proxy is not the
only service on this webserver, I had to take countermeasures and shut
down the proxy for a while until those people went off.
Now I was able to re-activate the proxy again. Sorry for the delay.
I don't want to blame those people, because I don't know if these know
about I2P. The proxy is not telling them yet, so perhaps they thought
"let's grab it before it goes away" and did as fast as they were able to.
They cannot know that it's destructive at my side.
The problem is not the traffic imposed or the load. It's the simple fact
that this proxy service was never thought to handle such demands. It shall
tell people to do the right thing, that is, install I2P and do the downloads
in anonymity. This way here is dangerous, as this system here still takes
logs (it's exerimental and it tells you you are not anonymous) and thus
I can (and will!) hand out logs to authorities (if they ask). I must do
this, because I want this service to stay alive.
(Some notes to the authorities: If some I2P service is unwanted, I can
block it! All you need to do is to explain me what and why. However
I am not able to do selective locks, so all I can do is block a destination
completely. However I am not the only proxy allowing others to access
I2P from non-I2P-enabled sites.)
[2007-04-18] I think I found and fixed the bug in the graph calculation.
Previously I thought it was a race condition which made the graph calculation
behave irregular and start over from the date 0, however it was a database
lock. As sqlite then outputs the error to stdout (instead stderr) this
gave some irregular input to the pipe to the further processing script.
(The fact that sqlite gives an error code does not help in pipe situations.)
Now I check for these irregular input such that database locks cannot
disturb the process any more (I hope).
[2007-04-17] Changed the graph subsystem.
As the RRD updating via cron had a race condition again,
I now wrote another independently running subsystem script. This
shall get rid of such conditions in future. The script is monitored
by the (heavily modified old) cron script which updates the process
table (Reachability) accordingly. It should not look differently,
though, only the resyncing process now is automatic and faster.
[2007-03-21] Proxy downtime yesterday
For unknown reasons the I2P router did not come up correctly after a
machine restart due to a power loss due to infrastructure replacement
at the ISP. After the I2P router service was restarted the problem
vanished. Also unknown is why the I2PinProxy did not switch to the
backup router (it only does so when the main router becomes unavailable,
in this case it was available but malfunctioning as it seems).
You can see this in the graph as the number of destinations dropped to
nearly none (only local destinations staied reachable, though).
Downtime was several hours, afterwards everything went back to normal.
I am currently improving some background services of the inProxy,
it shall not affect reachability.
[2007-03-16] Legends improved
Some of the explaining texts were missing or misleading.
[2007-03-01] Search engine improvements.
To reduce the confusion of search engines, I added some heuristics,
which hopefully will keep them away from pages which they do not need,
by setting the robots-directives in the status.php page headers to guide the
machines to output pages with sane values only. This shall remove the
clutter within search engines and hopefully improves the usefullness of this proxy.
[2007-02-26] TOR detection added and a bug in the new check script of I2P router 2 removed.
Well, the TOR thingie already is some days old now. I added some
highly experimental routines to detect if you come from a TOR gateway.
This is done with help of http://www.sectoor.de/tor.php
The detection is done in background to improve fault tolerance, so
it does not show your TOR status immediately. Also it does not work
perfectly, so it may be that you surf via TOR but this presence is not
detected. The background process is "torcheck" in the Process table.
Due to the fixed bug router 2 always was rebootet after 10 minutes of
operation and was reported non-working in the process information
window, too. Because of a major cold I wasn't able to detect and fix
this bug more early. However I don't think this has to do with the
"yellowness" of some probes. Still don't know why this happens and
the effects seem to decline by itself in the last days. Really weird.
[2007-02-22] For some unknown reason my router show "yellow spikes"
since some patches introduced after 0.6.1.26 but before 0.6.1.27.
I don't know if it is is of local origin, but I did not change anything
except updating the version to 0.6.1.26-something. However this effect
did not immediately show up after the upgrade (if I remember right),
it showed up a little bit later. So it either is a local effect
(which I doubt as I did not change anything) or it is a network issue
(either within I2P or my ISP). I really don't know, but for now we
have to live with the yellow spikes. You can tell this from the fact,
that more than one hourly process is running very often, as there are
a lot of (over 100) destinations probed (as the red line is far above
the graph) and the high number of 000 destinations.
Note that this yellowness seems to show up after the router is running
for quite a while (yellow hops in the daily graph). As the router is
instable under my 64 Bit OS
(which is perhaps caused by the JVM version of Ubuntu64 6.06 LTS)
I think this yellowness now is a sign for the nearby crash (starvation)
of the router, and thus can be fixed within Java. But I don't know how
to do it myself as I don't hack the router. And more important I really
don't know why this shows up now. Either the I2P network reached a
size which triggers this effect or a patch introduced after the
release version of 0.6.1.26 has such a side effect (as it then shows the
crash more clearly this is a good thing, but a thing I cannot fix).
So for now we have to live with this, and that the statistics shows the
reachability "very poor" or even worse.
[2007-02-15] For some reason there was some "yellowness" of the router.
You can tell this from the much work of the "hourly" processes piling
up in the status (right top of each page).
For some days probes came out randomly with 0 after 30 seconds.
Except for this everything looked pretty normal.
After an update to the latest devel version (0.6.1.26-19)
and a router restart this problem vanished.
I really don't know know what this was and it hopefully
doesn't show up again. The many hourly processes shall depile
tomorrow when the amount of work goes back to normal.
This behavior is by design.
Note that this problem is only loosely related to the problem
that the graph got out of sync. This came from a race condition
(yes, this is a bug) in the scripts which perhaps was triggered by
the amount of work seen due to the problem noted above. The graph
is resyncing slowly and shall be back operational in some days. I
speed up things a little bit by running the sync manually (while
the manual process runs the graph vanishes), however to get really
get rid of this problem these parts need a complete redesign which
has no high priority yet.
My monitoring has been changed to inform me about irregular states
like these above, such that I hopefully will detect this more quickly
in future.
[2007-01-07] An unplanned 3hrs outage at the ISP level made
this proxy unreachable.
Both I2P routers and the freenet node were not affected.
[2006-12-05] My freenet node runs instable because of
Java Heap Space issues.
Even 1 GB heap space is not enough for it to run.
After 10 minutes so the java processes fill all the
available RAM and after an hour or so they start to
get in trouble. Then the freenet node starts to
become instable and crashes later.
Don't found a workaround yet.
Perhaps it has to do with the increased datastore?
I would like to test JVM 1.4, however I did not find one
running on the 64 bit platform.
[2006-11-23] Had to increase the number of interconnecting
tunnels to the I2P router from 50 to 250. This is due to
some people using DownThemAll with insane settings.
This was needed to handle the increased load on the
I2PinProxy, as over 60 Apache worker processes waited to
get a chance to connect to the router. Maximum number of
clients is 150, though. If this still increases, I have
to move the I2PinProxy handling server to another machine.
It looks like some people out there start to leech I2P
via my proxy using DownThemAll with over 40(!) parallel downloads
to do partial transfers. Well, I understand your needs, I2P
sometimes is slow and you can tell DownThemAll to use many
partial transfers to speed up the transfer. But, please,
this is extremely destructive to others, as this occupies
trainloads of resources on servers.
I would suggest that you reduce the load a little bit (10
parallel downloads per IP max), as I don't want to implement
countermeasures. And yes, I can do this, but I don't want
to. However I don't blame you, I only ask for some mercy ;)
[2006-11-13] The I2PinProxy is running on the main router again.
The old broken router machine was replaced by a new server.
I managed to activated I2P on this new server again.
Hopefully it runs smoothly again.
At the current throughput of 120 KB/s the machine utilizes
up to 105% CPU (well, it's a Dual Core, so it can reach 200%),
which tells me that I2P still has too high CPU needs.
[2006-10-04] Running on the backup router as long as
it takes to take the new replacement machine online.
The new machine has more than double the capacity of the
old one, has 10 MBit/s flat and is ordered until,
at least, mid 2008. It's running Ubuntu 6.06 LTS and
hopefully will do a good job the next years.
[2006-09-30] Main I2P router is instable and
switching to the instable backup I2P router forth and back.
So things are very instable for the moment. Sorry!
The machine running the I2P router currently has a lot of
trouble. It shows a strange error such that I now ordered
a replacement machine.
In the meanwhile I switch this service (manually) forth and
back to my backup I2P router. However the latter is not
running stable yet (this time it's a software or resource
problem).
[2006-09-29] Expect instabilities. The machine running the
I2P router has strange memory bit errors.
Memory test says everything is OK, so I contact the ISP to ask
what options I have.
[2006-09-29] I2PinProxy runs on my instable backup I2P router.
Some destinations are not reachable because of this.
The server of my main I2P router is down for maintainance.
This means, the freenet node is down, too.
[2006-09-17] Unplanned downtime due to the server failing again.
This time the machine stayed up but the router crashed and did
not come up again due to a corrupt file. However the error was
spurious (the byte error disappeared by itself) which suggests
that the machine either has harddrive or RAM issues. My
suggestion is that there is slightly defective RAM.
However I do not have the time to investigate deeper now.
[2006-09-16] Unplanned downtime of a little more than 12 hours.
The machine running the router crashed for unknown reason.
I had no time to investigate immediately, but when I came around
all I was able to find was "just a crash with some kernel OOPSes",
so the machine was restarted and got back to work.
[2006-09-15] The router hung but the kill script did not work.
Apparently java does not always go away with a simple kill, it must be
killed -9 to do so. Script was changed accordingly.
[2006-09-09] Updated router to I2P 0.6.1.25. Changed monitoring scripts
a little bit.
[2006-09-08] Trouble ahead, sorry! The machine running the I2P router and
freenet server (it does not run this I2PinProxy) currently is overloaded.
I reconfigured a little bit in this machine such that it - hopefully - can
cope with this situation better now.
It is unknown how long this situation will last, but I am informed quickly
about such failures thanks to my obscure automated monitoring services.
Please note that this I2PinProxy still has experimental status.
Some other additional services which have productive status were moved
to the machine as it is the "hot spare" for such situations. The machine
barly has enough resources to run everything, so services are prioritized.
Freenet, with the lowest priority, stopped working today as there simply
were too few resources left for it to keep up even basic things. For the
moment Freenet is running again.
[2006-08-22] The statistics graph now is in the rebuilt phase. It will take
some time until the graph is synced again. The status now shows when the
graphs (which calculates the reachability too) is not up to date.
Only the reachability prognosis and the history graph was affected by this
bug. The probe loops overcame this trouble nearly unharmed.
For some unknown reason the statistics calculation failed to work.
Looks like the scripts stumbled upon some serious database locks which
lead to a race condition rendering the current state garbage.
From there on some domino effect happened which led to false error conditions
to the monitoring, which in turn alerted me to fix it (oh well ..).
I should rewrite the cron jobs into a ptybuffer autostart loop, such that
no such race conditions can show up again in future. Let's see when I find
the time to do so.
[2006-08-20] The router stopped working for unknown reason and was killed.
As the restart feature wasn't yet working as expected, downtime was 9 hours.
Fixed this, so the router restarts automatically in future in such cases.
The good news is, as orion.i2p is back, new destinations start to show up again.
[2006-08-12] Started a second experimental I2P router
based on
build from CVS.
[2006-08-05] The status page now is standalone for better search machine optimization.
[2006-07-29] Updated to 0.6.1.24 which needs a lot of RAM but works very stable.
[2006-07-28] Updated to 0.6.1.23 which is extremely instable.
Added a killscript which kills the I2P router when the router
console becomes unreachable (so the router stops working).
This keeps the I2PinProxy somehow operable, but the reachability
is a mess.
[2006-07-20 16:30 UTC] The machine with the I2P router crashed again.
The monitoring system detected it as intended. But there now
is a bad "yellow spike" in the reachability graph of I2PinProxy,
as the probe loops did not detect that the router became offline.
This bug now has been fixed. Note that there are no new probes
now which can be entered in the graphs, so they show a
"white area" until new probes come in. This behavior is by purpose.
[2006-06-02] All important parts run smoothly now. At least locally.
Optics improved (you cannot see this with IE as IE has CSS issues)
and minor bugs fixed. Now the second step of the inProxy improvements
can start:
- Daemonize the tasks which currently loop from shell.
- Start to implement the forwarder which does proxy warnings,
censoring and caching.
- Implement some further UI features like "search/quickjump".
- Leave behind minor bugs like the non working Mirror.
Don't expect anything in the next few months, though, but I hope to be able
to start to gather some proxy (thus local) statistics in the next weeks.
[2006-04-16] "flaps" added to the graph. Router version now shows uptime, too.
Also there is now a "Reachability" prognosis shown in the top right box.
[2006-04-13] This pages are now compliant to HTML 4.01 according to W3C, too.
See check-buttons at the bottom.
[2006-04-01] This pages are now compliant to HTML according to HTMLtidy.
And this is no april's fool ;)
[2006-03-29] Today my provider is somewhat instable.
You can see this that the "hourly" processes pile up on the top right.
[2006-03-08] For some days now my I2P router is somewhat instable.
The automatic relaunch fails due to a too short timeout.
I am working on the problem. Stay tuned, and sorry!
Note that you cannot see this in the probe history, as the probes
are detected as false (most time; Some yellows still show up) and
are not counted. But you can see it in the probe history if you examine
the timestamps (for this click on the "h" link of a destination).
[2006-02-22] I2P 0.6.1.11-0 is much more stable than 0.6.1.10.
InProxy downtimes shall be less frequently, it looks like reachability
is back to normal.
[2006-02-20] I2P 0.6.1.10-0 extremely instable but old versions are incompatible.
If the "hourly" (right top) pile up while eepsites become "yellow", it's likely
that I have to reboot the router manually. Afterwards the eepsites show up
"green" again while the "hourly" slowly depile ..
[2006-02-19] I2P 0.6.1.10-0 crashed because it ate all diskspace.
See
Forum.
Fixes applied.
[2006-02-17] Updated to I2P 0.6.1.10-0. As it is not compatible to 0.6.1.9
you can see in the statistics how the network became unreachable until I came
arround to update.
[2006-01-18] The bandwidth of the whole machine has reached 2 GB in December.
I am in the phase of considering a move of this service to another site.
This will take some weeks. In the meanwhile:
Feel free to use it. But use it wisely and don't abuse it. Thanks.
[2005-11-14] This proxy uses a bandwidth below 100 MB/month today.
This is far below 10 GB/month, the maximum level I can spare on this machine.
If the traffic exceeds 1 GB/month I will start to consider moving this
service to a dedicated machine.
So feel free to use it. But use it wisely and don't abuse it. Thanks.
(Bitte dem Link zum Impressum folgen.
Diesem ist eine Erklärung vorgeschaltet, damit man besser versteht,
worum es sich bei diesem Proxy handelt.)
[ms] (user+sys)/run (10+0)/625 -- According to German law, I must give a link to an Imprint in German language.
$Date: 2011-05-22 14:11:11 $ Valentin Hilbig