blackblog.

snippets of blackwings life

PITA. rtorrent-0.7.9 on Ubuntu Dapper 6.06-LTS

Today, the usual pain of release-based distrubutions such as Debilian, eh Debian or Ubuntu occured to me – I wanted to have the encryption feature of rtorrent that was introduced into libtorrent 0.11 / rtorrent 0.7. Unfortunately the latest version in Dapper is 0.5.3 – so far too outdated.

Doing a .deb package is always useful to keep the system maintained – Building just from source was out-of-question. So after some years of Debian absence I started “fighting” with the build system again.  After some pain-in-the-ass things, I finally had the packages ready: rtorrent_0.7.9-0ubuntu1_amd64.deb and libtorrent11_0.11.9-0ubuntu1_amd64.deb

full stuff is available at: http://blackwing.de/files/contrib/ubuntu/ – So if you want to build a i386 package, go ahead :)

24c3 in Berlin

Right after christmas I took a flight from Frankfurt to Berlin in order to get to the 24th Chaos Communication Congress. I participated again as a member of the NOC-Crew taking care of the IP connectivity to the outside world.

First of all: After not having been to 23c3 last year, I have to summarize this was probably the best congress I have ever been to – Relaxed, infrastructure worked well and it there were some good talks. Compared to my first congress (which was actually 10 years ago in Hamburg) there had been significant changes – Way more people, professional organization and a lot more talks, tracks and so on. In 1997 the topics were more technical and there was less stuff around “politics” – Not that it bothers me, but all in all this shift in focus occurred to the CCC in general in my opinion. It seemed that the classical underground hackers foundation shifted to a non-governmental organisation (NGO)  with all its pros and cons.

Besides the NOC stuff I had this year enough time to watch some of the talks. Some of them were quite good, others quite unsatisfying. To name some:

  • I liked the talk of fabs and FX on portbunny, a kernel-based portscanner. That is kernel-based is not the interesting point – But they use trigger packets to implement a congestion scheme like known from TCP to speed up port-scans.
  • The unusual web bugs talk was quite interesting also – some of the aspects were completely new to me.
  • Iljas talk on a collection of random things had some new things (I never thought about using OOB data in TCP to do funny things) and some well-known aspects (/dev/[k]mem issues) – But unfortunately he left out the part I was most interested in: The TCP fuzzer :)
  • Arien, also a NOC member, did a talk on real-time 10GbE monitoring using some FPGA setup on a Force10 network card was nice.
  • Drew Endy, a MIT professor in biological engineering, did an enlighting talk on DNA coding – quite kewl.
  • Florian Bischof’s talk on Sex 2.0 was interesting – Completely different from what I’ve expected. Some people disliked it, because it mainly focused on homosexual stuff, but I found it interesting anyway.
  • I liked the vivid talk cryx, denis and erdgeist on openTracker where they described all the nice stuff around “driving” an bittorrent tracker in the wild-wild-net.
  • A total disappointment was the talk on hacking embedded devices – No serious content, just basic stuff.

All in all: Good congress! Although I am interested on more deeply technical talks.

centericq – icq contact not showing online

Heyho. Time for some more technical posting.

I’m using centericq as my console-based jabber & icq backed, even though it is now unmaintained and is going to be replaced by centerim. But yesterday, the problem evolved, that icq contacts did not show their status anymore, which is pretty anoying. So I tried to find a solution on the net and actually a Bugzilla-Thread gave me the hint, to upgrade to a current centerim, in order to fix that problem.

My server is running Gentoo, so importing the brand-new version was not a problem – even the old contacts worked instantly. But unfortunantely, it was not possible to write messages via the compose dialog… Being quite unhappy with the “solution”, that now contacts show online, but I’m not able to write messages, I was quite unsatisfied.

So I just added the HotFix from the centerim-git repository to the old centericq source code – actually just one single line has been changed! And tada: Everything is working fine again :)

The whole fix is related to the fact, that ICQ changed their protocol again – Now your client has to ACKnowledge that it has received the ServerBasedContactList – After this you get the online status of all the Users on the List. And thats the point: From now on it is not possible to see Online-status of  users not on your ServerBasedContactList! So start getting authorizations in order to fix that.

Collaborative Scientific Citation Managment

I finally started my final thesis to complete my degree in chemical engineering – and again, I was stuck with the problem of literature managment. Of courseI use LaTeX to write this thesis after the successful try-out during my last years Studienarbeit. The built-in BibTeX system to generate the reference list is pretty good, but it does not solve the problem of how to do your citation managment, namely how to save the PDF files, name them and find them in case you need them. It gets even worse, when you want to share these things in your scientific workgroup… So I found situations like: “Eh, go use SciFinder search for $Author and $Year and then you can find full text”. What I need is a tool like Flickr where I can organize my Citations, Tag them, maybe share those in a work-group and finally export them as a BibTeX file…

So I started searching for a software solution. First I found several commercial solutions using Windows (like Citavi, Endnote or RefWorks) – But all of those did not really satisfy me. After searching for some time I accidently found out about BibSonomy, a research project of Kassel University. THIS is what I was searching for! Tagging of publications like pictures in Flickr, importing stuff via existing Bibtex-file and exporting stuff in various ways: There are countless export methods such as BibTeX, Endnote or even as a HTML Table! You can search by tags, organize your literature and even upload your own full-text copy… People, just thanks for this great tool – It made my day!

Repair status of the earth-quaked trans-pacific fiberlines

As mentioned earlier, there have been severe disruptions to the Asian internet connectivity due to the Taiwan earthquake in late December. As connectivity is still quite slow I checked on the Net how the status of the repair is…

So here is the current situation of the pacific cables:

So Situation remains quite unclear. AFP reports, that situation is close-to-normal, but I disagree with that. Finally the Henchun earthquake shows, that even the “fault-proof” Internet protocol is not really perfect when all routes go through a single bottleneck.

To be precise: Almost all communication from Asia to Europe (and US) runs via the Pacific Ocean even though shorter links via the middle-east are present. This was also obeserved by CyTrap. As seen in the global submarine cable map the actual bandwidth linking South-East Asia via the Middle-East or Russia is quite low…

So Singapore suffered from severage outages even that the direct fiber-links to Europe were NOT affected at all! There is really a need for some direct routes via Middle-East in order to ensure a fault-protection and reduce impact of such one-spot failures as just seen. Additionally this way physically shorter, which results in lower delays. But of course the US-based Tier1 providers are not interested in such upgrades.

So the situation remains like this: Like flying via San Francisco / New York when you want to get from Singapore to Frankfurt…

My c-plugins for qmail-spp: rblchecks and greylist

In order to fight spam, methods such as using a dns-based realtime blacklist or greylisting have been proven effective.

I use qmail with the very useful qmail-spp patch-set, that adds plugin capabilities to qmail-smtpd. Instead of just adding more and more chained commands to the run; file of tcpserver, you just specify a small plugin that is executed by qmail-smtpd when a certain smtp command is issued. In fact, this method is way nicer than forking many programs in a row… My run-line for the smtpd grew constantly, due to addition of rblsmtpd, qgreylist, … and I started to really dislike the increasingly huge memory footprint of a single smtp session. So I started searching for a rbl and greylisting plugin for qmail-spp – preferably in c, because forking several perl-instances per smtp session is just overkill. What I found was:

  • greylisting-spp by Peter Conrad. His website indicated, that he claims greylisting-spp to be alpha-quality, so I kept away from using it (After writing my own greylisting solution and putting him into cc, he said that it is actually quite stable – but the website did not indicate it *argh*).
  • ra-plugins by Roberto Alsina. Great collection of plugins – including a rblchecks one. But unfortunately his implementation lacks whitelisting support.

Because of this I did some own implementation – rather small and in c. To be honest, both greylisting and rblchecks are based on existing ideas / implementations. I suggest the following usage in the [mail] section of qmail-spp:
[mail]
skip-if-relayclient
skip-if-smtpauthuser
rblchecks
greylist

Update: People keep asking me per mail, what kind of blacklists and so on I use: I use the dul.dnsbl.sorbs.net and ix.dnsbl.manitu.net as blacklist and whitelist common mailservers such as web.de, gmx.net etc.

rblchecks.c – Check TCPREMOTEIP against a list of white- / blacklists.

Based on rblchecks.c of ra-plugins written by Roberto Alsina. Features & behavior of this plugin as follows (are README, and Makefile will be added at a later point).

  • If SKIP_RBL is set, plugin exits without doing anything (so following plugins in the chain will be executed)
  • The bstring library is used. So please have its header files present.
  • If TCPREMOTEIP is listed in one of the servers specified in the RWLSERVERS environment variable, plugin exits with A answer code. Please use tcp.smtp to specify the RWLSERVERS. Multiple servers can be supplied using : as separator.
  • If TCPREMOTEIP is listed in RBLSERVERS, plugin will exit with error 541 and qmail-smtpd will quit the smtp session.
  • Be sure to set a DNS timeout via resolv.conf In a future version, there will be an timeout handling within this plugin.
  • Plugin will give diagnostic messages via STDERR, so that qmail-spp will pass them to the logging system of qmail.
  • In case a IPv6 connection is found ($PROTO=TCP6) the plugin exits with a note – because IPv6 is not (yet?) subject to realtime blacklists.

greylist.c – Perform greylisting on TCPREMOTEIP.

This plugin was inspired by qgreylist by Jon Atkins. This implementation does a IP based greylisting – Sender and Receiver addresses are not taken into account! Right now there is just the c-file. Documentation as in README will follow.

  • If SKIP_GREY is set, plugin exits without doing anything (so following plugins in the chain will be executed)
  • Greylist creates an empty file named by TCPREMOTEHOST in the specified BASEDIR (default is /var/qmail/greylist) when a host is first seen.
  • A host, that is seen the first time gets a temporary 451 error.
  • After a minimum waiting time set via GL_MIN_REJECT (default is 300s) a SMTP session is accepted.
  • If host comes back before GL_MIN_REJECT, it gets an 451 again.
  • Once a session was successful, the entry has a lifetime of GL_ACCEPT_GOOD (default is 32 days).
  • If a host does not come back after GL_MAX_WAIT (default is one day), the entry is subject to cleanup.
  • On every subsequent SMTP session of TCPREMOTEHOST the file-access-time of the corresponding file is updated to the current time.
  • The modification time of the file corresponds to the time, when the host was first seen, while the access time refers to the time when it has been seen for the last time.
  • Be sure to have enough inodes in the specified BASEDIR! For high-volume servers you might consider using a ramdisk. But even for a server with about 15000 mails per hour, there was no slowdown. Best results have been seen on ext3 with htree, xfs and reiserfs. (I personally use a ramdisk with reiserfs on it. Do NOT use ramfs/tmpfs!
  • BASEDIR must be read-/writetable for the qmail-smtp user (usually qmaild)
  • Concept is also valid (and working!) for IPv6.
  • A clean-up cronjob for the BASEDIR is suggested. Just cleanup files, that have not been seen again within GL_ACCEPT_GOOD (now > atime + accept good) or that exceed GL_MAX_WAIT (now > atime +maxwait and atime == utime). A simple perl-oneliner does that:
    perl -e "my $time = time(); for my $file ( </var/qmail/greylist/*> ) { my ( $atime, $mtime ) = (stat $file)[8,9]; if ( ( ( $atime == $mtime ) and ( $atime < $time - 300 ) ) or ( $atime < $time - 2764800 ) ) { unlink $file or print "unlink $file failed"; } }"

skip-if-relayclient.c and skip-if-smtpauthuser.c – Simple plugins that stop further qmail-spp processing if RELAYCLIENT or SMTPAUTHUSER

The use of creating a kind of if-then-else handling in qmail-spp made me write these trivial little programs. So get them in front of your tool-chain in order to stop further plugins from being executed when you have a trusted sender!

  • If SMTPAUTHUSER or RELAYCLIENT are set, plugin exits using the A answer code (no further plugins in the chain will be executed)

Connectivity Singapore to Germany

First of all: It really sucks. These days the connection is so incredibly slow, that working via interactive protocols (e.g. ssh ) is virtually impossible – A ping roundtrip is around 350-450ms. This is actually not too bad, but jitter and loss really get me crazy – Sometimes connection is stuck for about a minute. In November lag & loss were quite acceptable – It was even possible to upload stuff with about 16kb/s – But right now were down to less than 2kb/s and frequent disconnects cause of too high packet loss. First I thaught, that the Uplink of NTU is just packed, so I took my Laptop and tried it at other locations. But even when you get on the net via Wireless@sg or other carriers, the situation is quite the same. Most of the slowdown seems to be related to the Taiwan earthquake in late December 2006, which cut down many fiber optic links in the Taiwan straits. I just found some news, that none of the cables has been restored yet. Other bloggers in the region also complain about lag & packet loss…

So lets get into some investigations, what’s really happening…

So let’s hope that they get the cables fixed soon…

Book recommendation

Besides all the tour-blogging of my Singapore related stuff, it is worth mentioning that I found a lot of time during the last weeks to just sit down and read – Something that was pretty wiped-out during the busy last semesters in university. Probably you understand: After getting slammed by reading scientific stuff, sitting in the library and learning/reading, I was not really motivated to take a book and read for relaxing. So, here is my recommendation:

Planet Germany Eric T. Hansen, Dec 2006, in German
Book cover I got this book as Christmas present from my girlfriend – and she did a really good suggestion: This book is really a pleasure to read! The author itself grew up in Hawaii, but studied in Germany and is living there for about 20 years now. He is working as professional journalist now – And you can easily see that. Facts have been accurately collected and presented in a clear manner. Full of nice side-stories the author points out many typical German behaviors and critically analyzes the German’s habits. Most times you start laughing about the introductive story in a chapter, but when he clearly presents funny facts you actually start thinking. Topics like “Why do Germans always complain about anything?” or the topics about the German national identity are really interesting to read. This book is full of indirect provocation, social critics and interesting views – Presented by someone who is truly a proud American and who lived among these Germans, that always pretend to get americanized. Definitively worth reading!

Update: My girlfriend added, that this book was actually suggested to her by shutdown.

Halftime – Planning the second half.

Time is running fast, really fast. Just today, it is half-time of my time overseas. Some people asked my, at what time I’ll be back, so here are the brief details:

  • Flight back to Germany: Tue 20.3.2007
  • Back to Erlangen: Probably Sat 24.3.2007

Actually my internship is also over quite soon – Only about 4 weeks are left! After the attachment is finished, I have the huge amount of 6 weeks left, which is available for traveling around South-East Asia. Quite many people asked me, what will be on my schedule for that time – So here is an overview:

In February, I’ll do some multi-day trips to destinations such as:

Then it is time for Chinese New Year – So I’ll be back at Singapore to celebrate this with friends. So at the 22.2.07 I’ll head for a backpacking trip around Malaysia. Although no detailed plans are done (and never will – Because this is subject to daily mood and weather), I plan to visit the following places (partially visualized at the map).

To summarize: Getting around the East Coast of west Malaysia starting in the very south to the very north and then get back via Cameron Highlands and Kuala Lumpur.

Visualisation of places I visited

Today, by reading some other tour blog, I found out, that quikmaps.com offers an easy-to-use service to draw lines, markers and whatsoever on GoogleMaps

Just yesterday GoogleMaps added local streetmaps for Singapore – better than the blurred sat-pics… Wanna know where I stay at the University? See here!

Because many people asked where I had been and what I did… So here is a first version containing some of my major trips. All the stuff I did in Singapore itself is missing – Not enough time to get it all in!

So: Enjoy the map!

P.S.: Today I will get to Thailand, to be more precise: Phuket Island in order to spend there some days for vacation… So: Merry Christmas! Stay tuned!

Powered by WordPress