Markus Breitenbach

I just read the book Cult of the Amateur and quite frankly disagree with a lot written in it. To make a long story short, the authors world is one where there is a simple transcendental reality; a truth, purveyed by trained experts, journalists and professionals. Apparently the danger of dissolution is upon us by the radically relativistic truths of Wikipedia where the community sets the agenda. I think things are far less black and white than he makes them seem to be. One particular example I very much disagree with is his criticism of recommender systems. The author claims that nobody will need to read movie reviews anymore when there are AI systems making the recommendations. I have to say that so far I found movie ratings by experts – be it ratings in IMDB or a professional review in a newspaper – totally useless. I’ve liked films with low ratings, and hated others with high-ratings. I discovered so many things that I liked with recommender systems like Pandora. Yes, there are problems with recommender systems, but relying on an expert opinion for matters of taste can in my opinion never work out. People simply have different tastes and I don’t think that one reviewer writing movie-reviews for a particular newspaper is speaking for the entire readership. For another wonderful example using Spaghetti Sauce as an illustration, see the TED talk from Malcom Gladwell. That said, there are some points in the book that deserve consideration. Anybody who has ever read through comments (“noise”) on youtube knows that there seems to be a mass-infestation of stupidity out there – something that needs to be taken into consideration in all the Web 2.0 experiments. Stupidfilter anyone? 🙂

Last night I was out with a couple of friends at our favorite club. I had the time of my life and met some interesting new people. When we left the club, a girl got hit by a car right in front of our eyes. I’ve seen quite a bit of “shit going down” in my life now (3 fires, one guy getting beat up in an alley, 2 times one or more people getting hit by a car, once getting hit by a car myself) and what I find interesting is the different reactions by people. Last night some guys freaked, some were calm, some tried to help, some girls just started crying. Most people just stared in some kind of apathy and did nothing. Some “tough guys”, who I wouldn’t have expected that from, freaked out. In all the instances I’ve been involved with I’ve stayed calm and just did what was necessary (including the one where I got hit). That said, 911 operators are kinda slow – why do they ask if the person hit by the car is OK when I already told them that the person is seriously injured? What a weird end for a good night – I’m still shaking a bit…

Last Saturday (3/31/07) I went to the Caffeine Music and Arts Festival Rave party (by Triad Dragons) in Littleton (Denver). The event started at 8PM and was supposed to go until 5am in the morning. This must have been the worst organized event I have ever been to. I bought tickets online, arrived at 9PM and stood with thousands of other people in line in the cold. Everybody had their ticket already and it took 2 hours of standing in line to be able to see the entrance. It was cold, and everybody (including me) was wearing thin stuff like t-shirts etc. This is ridiculous. At 11PM I left as it probably would have taken another hour or so (given the speed the line was moving) to get to the entrance from where I was. Given that they expected a couple of thousand people to attend, they had (from what I could see from where I was) only two (2!) people to pat down visitors for drugs and weapons. Maybe the rest was on break or whatever, but one would expect that they can afford enough staff for the event for $35/ticket. I will most certainly not attend the Caffeine Festival 2008, and I will let anybody (who wants to hear about it) know about how things went in 2007. I am upset …

When I started my blog I was already aware about the Comment Spam problem and thus enabled a WordPress plugin to prevent comment spam (“did you pass math”). The other day a friend complained that when he wanted to comment on something and forgot to fill out the captcha-field his comment got lost (and pushing the back-button had his browser loose all that he had typed up). And when I was reading through raw apache logs and saw somebody trying to post a comment and apparently not succeeding. So I turned the plugin off and within a day I had 8 spam comments on my blog (which does not have a high pagerank and uses nofollow-links; What’s the gain?)… So I’ll keep it turned on. There!

Spam is an interesting problem, because you have an “adversary” with a lot of resources who will do whatever it takes to get your attention, an email in your inbox or a comment with links on your blog. The more filters we build, even with machine learning, the more sophisticated they become. It will probably be a driving force for classification for some time to come. However, machine learning and filters are very expensive in CPU time and do not scale very well. Sander told me about the email server at their institute having a backlog in emails of 40 Gigabytes, i.e. 40 Gigabytes of emails staying in the spool waiting to be scanned for spam and virii. Given that this server was only serving about 50 users and given that 99% of the email in the spool is probably spam illustrates the problem. Currently (in my opinion) mechanisms like Grey-Listing and such are a better solution simply because they scale better as they exploit “implementation issues” of the spam-software and don’t require the CPU-intensive scan of every email. That is, until the next generation of Spam-bots will adapt to those measures. Build a better spam-filter and somebody will build a better spam.