Douwe Osinga's Blog: October 2003

Friday, October 31, 2003

Visual Poetry Revisited

A couple of weeks ago, I published Visual Poetry, a Google hack to use image search to translate a sentence or a poem into a series of images. Visual Poetry was a windows program that showed the images in a slide show. I now added an online version which works with the thumb nails of Google and shows a sentence directly as a series of pictures on one screen. Check it out.

Thursday, October 30, 2003

That can't be good

According to the New Scientist, a US government funded scientist developed a extra deadly mouse virus, simlar to small pox, that kill mouse also if they were vaccinated and taking drugs. There might be a medical upside to research like this, but the potential danger in such discoveries is much larger. The next white powder envelopes might take out New York.

In general, I'm all for research. People want to know stuff, you can't stop them. But this is rather specific. It is not like trying to find out what atoms are made of and discover a new and very efficient way of killing a hundred thousand people a shot. Designing a killer virus is something doctor Evil in would do in a bad comic book.

There are enough killer diseases out there already. We don't need new & improved ones.

Wednesday, October 29, 2003

Nigerian e-mail scams

Ten years ago I received a fax from a Nigerian businessman with a proposal: he had 5 (five) million dollars and if I arranged for a bank account, I could keep 15% or so. It had scam written all over it, so I didn't do anything about it. I did wonder what the deal was, though.

Nowadays, I receive such proposals three times a day. Mostly, they claim to be from family of now dead West-African dictators. One day I got an email from the lawyer of Charles Taylor and then one from Charles him self. For fun I asked the lawyer where this miscommunication came from. He said that Charles had been confused and I only should mail with the lawyer. This place is full of vultures, vultures everywhere.

What is the plan? I know now how it works. It is called the "advance fee fraud" or "419 fraud". Once you contact them, they'll make up lots of little reasons why you should pay up some money upfront before they'll transfer the loot. Government fees, attorney fees, travel expenses. Then they'll try to get you to come to Nigeria. They'll arrange for the hotel, which is expensive, maybe an airticket. They'll get you in without a visum (not necesary they'll claim) and get you busted for it too. A big bribe will do, thank you very much. Etc, etc. Some people are eventually killed in Africa.

But what is the plan? If I receive one letter in ten years, I might consider it to be genuine. But I get three a day. Most people probably get less spam than I do, but still. It is not very convincing. And why doesn't the Nigerian government do something about this? The scammers might make some money that stays in Nigeria, but on the long run this hurts Africa/Nigeria a lot. The place is corrupt and scammy, is the general impression we get. It might be true, but it is bad advertising.

Monday, October 27, 2003

Iraq as a terrorist trap

When Bush said he was going to attack Iraq as part of his War on Terrorism, this didn't seem very convincing. Sadam did not have a lot to do with Al Qaeda, except for some splinter group in the Philipines. No direct connections have been discovered since the overthrow of the regime.

Iraq has turned into a playground of terrorists on the other hand. 20 minor attacks a day and every week or so a really big one. It seems that this terrorism isn't home grown either: from all over the world Islamic hardliners move to Iraq to fight the Great Satan. So Bush gets to fight terrorism in Iraq after all.

This is probably not the original plan. But it is a way to catch terrorists. Invade a country and make it clear to Terrorism International Inc that this is the place to do business. Then, if you have them all within that country, catch them. It is like a big mouse trap. The bait is set. Now let's hope the mouse doesn't get way.

Sunday, October 26, 2003

Two new projects

I put two projects online I had laying around for a bit. ZAmazon is a zope product, allowing you to do Amazon searches from within Zope. SixMovies is a game where you have to find the shortest route between movies by finding actors that played in movies that had actors that played in movies etc. until you get to the target. The games doesn't really work because of the shallowness of Amazon database, but it gives you an idea.

Saturday, October 25, 2003

What are the chances?

This just in. The actor that plays Jezus in a controversial movie was struck by lightning during the shoot. Unlikely, but stastically explainable? It was the second time this happened during the shoot. If this guy is killed the third time he is hit, he can't say God didn't warn him.

Friday, October 24, 2003

Cool Stuff

I have no ambition to be the first blogger to write about this, but some stuff is too cool not to blog.

First a real laser printer. Versa Laser is a printer that connects to your computer through USB and is a normal printer as far as your computer can tell, except for that it doesn't really print, but cuts through all kinds of materials with its laser. Woodworking, paper cutting and plastic modelling are all possible. Engrave your plastic lighter now with 'I am stupid'

The second one is Amazon. They added full text search inside books. They scanned 120.000. books and you can full text search any of 33 million pages. Actually, I wrote about that they should do this a couple of months ago. You type in a word, Amazon finds the books containing it and lets you click through to the page that contains it. Great stuff.

Thirdly for the gadget insane, the USB Air purifyer. You plug it into your computer and you have fresh air all the time. Maybe not that useful, but it shows how far stuff that plugs into the USB port has come. Cell phone loaders, lamps and memory sticks, the USB port has become the cigarette lighter of the computer.

Thursday, October 23, 2003

Computers: so fast and yet so slow

My first computer was a BBC Micro (ok, it was my parent's). 2 Mhz, 32KByte. If I recall correctly, wordprocessing wasn't much slower than nowadays on MS Word. I'm pretty sure that WordWise started a lot faster. The 6502 that powered the BBC was per megaherz probably a factor 10 or so slower. That means a middle of the road pentium is about 10.000. times faster. But my computer isn't.

Of course not, you might reply. Word 2000 does a lot more stuff than WordWise ever dreamed about. Sure, but 10.000. times? That's the difference between a snail (11 meters/hour) and a car on a highway.

Starting applications is the worst. Programs like Photoshop or Delphi take forever to get ready. And what do they accomplish? The initial state, which when you think about it, comes down to maybe 20 megabyte of memory initialized in the same way anytime they start. That's got me thinking, why can't the app tell the OS after initializing, ok, I'm ready to be used, this is the my initial state. Please write the image of me to disk and next time you want to start me, just load the image. Ultra DMA100 should do 100Mbyte/sec, so word should start in 0.2 seconds.

Tuesday, October 21, 2003

Mind World Map progress

Daniel Moore comments on the lack of progress in the Mind World Map project here. If you look at the map, you'll see that a reasonable map is created pretty quickly, but then the progress stops and it seems just random noise is added. I think there is a logical explanation. If 95% of the people click the right pixels and 5% click random pixels, you'll progress really fast towards a map of 90% accuracy, but the progress slows down and you'll never make 95%. 95% sounds good, but it means that 100 pixels or so are wrong. And 95% is probably rather high.

I have been thinking of optimizing the map, i.e. more majority voting and maybe than have the red pixel be chosen from the pixels with the greatest difference of opinion or one that borders on land and water. On the other hand, most of the 16000 clicks I got in two days after metafilter posted something about the project and ever since, progress has been slow. For now, I'm mostly curious whether the maps for different toplevel domains will start to grow apart.

Refactoring the Law

Computer programs tend to start out small and as time goes by, features and new functionality is added. This makes old code more complex as it was never intended for these new tasks. It is normal to say at one point: enough is enough, let's refactor the program. It doesn't mean: throw away the old code and start anew. It just means to take a step back and consider what you have and what is actually needed. Create a new design and rewrite the parts that don't fit. Maybe there is some legacy code that works, but is in another language or uses arcane constructions. Maybe there are some very complex routines that are only used to do execute some relatively simple tasks. These could be cut down and simplified. It might take a while, but the overall program will improve a lot and it will allow new programmers to work on the system with a lot less effort.

Most Western law systems were created somewhere in the 19th century and it shows. The language is arcane (legalese). A lot of laws used for modern phenomena actually talk about things of 80 years ago and can only be matched on the current situation by interpretation. Great fun for the lawyers, but it makes the whole thing rather incomprehensible for the lay man.

The law needs to be refactored. Get the best lawyer, let them make a list of what the law essentially says and have them work out a structure that is most suited for this. Then we can step by step start to transform the existing body of law into that structure. It will take some time and effort, but it will make things clearer and more efficient on the long run. And while we're at it, let's translate the law from legalese into real lanugage. Anything worth saying can be said clearly.

Friday, October 17, 2003

Best Month To Visit by Google

After I wrote about a failed project to use Google to find the best month to visit somewhere, I went back, had a look at the code and tortured it until it gave me reasonable answers. It is here: best time to visit

The code works by searching for the names of months in the returned google descriptions. The two top months are assumed to be the month between which you should visit. One problem is determining which one comes first. For Amsterdam and Australia, the code returns April and October. But you shouldn't go to Amsterdam in februari, if june is an option.

Wednesday, October 15, 2003

Using Google as Common Sense engine

Hjalmar Gislason has an interesting piece about Google Miner. Basically, Google Minder uses Google to extract common sense from the web, using the ordering feature of Google to get the most relevant information. Is it doable?

When I started with Google History, my aim was actually something broader, a Google application that could answer any type of question. The first results of that were so bad, that I decided to limit myself to questions that have years in the range 1800-2050 as an answer. That did work. Better.

Is it possible to mine the Web for common sense? Another failed Google project of mine tried to work out the best time to visit a location, interesting information for the travel site I help building, world66. I fed Google the name of the location and the sentence "best time to visit" or a synonym. I would then search the returned descriptions of Google for the names of months and choose the mostly named one. It worked pretty well, returning june a lot of the time for European locations. But it also returned june for Australia and februari for Denmark.

Now, I only scanned Googles top-10, so the results might improve if the program would take a top 100. I'll look into that later. But the point is that if a rather trivial piece of information like this is hard to extract from Google results, then the more arcane things will be very hard indeed.

Things might look different for Google self, though. They have 3 billion documents indexed in all kinds of ways. Running some clever algorithm against 3 billion documents in stead of 10 might improve results drastically. Google became self-aware at 2:14 a.m. eastern time, August 29.

One other thing. A couple of weeks ago, I wrote a piece titled 'how much is a billion', complaining about the number-unawareness that is riding high. Currently the entry has position two at Google for the term "how much is a billion" and there are quite a few people arriving on my site with that very question. Ironic and insulting for the visitors. One guy wrote me to complain that I didn't actually explain how much a billion is, in an easy to understand way. Couldn't I post something about that on Google?

What about: if a thousand people lose a thousand dollar, every day, for three years, they have lost a billion dollar. Still no way to understand the size of the American deficit. Anybody with a better explanation?

Tuesday, October 14, 2003

Internet history

The Internet doesn't forget that easy. Especially on Usenet, the original discussion forum of the Internet predating the web, there are some interesting first posts, things that became much bigger later.

Searching a bit around, I found the following monumental posts. Geeks announcing something that was at the time a tiny experiment but would become a revolution:

  • One Richard M. Stalman has a bright idea: free unix. From these humble beginnings GNU grew on which a lot of Linux is based.

  • Talking about which, one Linus Benedict Torvalds posts about a pet project he had, trying an OS for some old hardware he had lying around.

  • Then there is of course that other pet project of Tim Berners-Lee, something he called the WorldWideWeb project, as if the hacking of one guy had a chance of making it world wide.

  • Followed by a post of Marc Andreessen about a browser called Mosaic, which was to be followed by Netscape, which started the Internet boom in earnest.

  • Lawrence Page asks how to set the User-Agent-Field in a java app he wrote. That app later became Google.

Of course there are many more posts, mostly of people who didn't make it (yet). Usenet used to be this big platform bubbling with ideas. People would post, other people would respond and something good would happen. Today blogging has taken over that process. May be in ten years or so, there'll be a page[?] with links to very old blogs who where the beginning of something great.

Open Movements that closed

Nowadays, if you insert a CD in a computer and the CD isn't copyright protected, chances are that the computer will retrieve the titles of the songs on the CD. This happens through the magic of the CDDB. Every CD has a unique id and the CDDB has a list of songs associated with that id. Great application, nice company, you might think. But you'd be wrong. The CDDB started as a collaborative project with lots of volunteers typing in the titles of songs. But then the Internet Boom came and the guys at CDDB realized they could make money on it, so they did. Nowadays, if you want to incorporate the CBDB in a program, you will have to pay. None of the proceeds go to the original volunteers. Luckily enough, there is the FreeDB, which is free and will always be free.

Open Movements that close are not the seldom. The IMDB started as a volunteer project (but we have the YMBD (Your Movie Database)) and is now commercial. TravLang is another example. An I'm sorry to say, that, a project I helped set up, is guilty of the same crime. The good news is, we're going to open up. World66 will be a truely open source travel guide, with an open content license and probably open source code (if I get it cleaned up enough). In the coming weeks, I'll be reporting from time to time about where we're going.

Here's a first one. The license. Of course we want an open license, some thing similar to the GPL. The thing is, I would like everybody be able to copy everything. But if somebody put it on a website, it would be best if the license required them that anybody could still edit the content.At that website or with a link back to world 66. Is this feasible within Open Content?

Monday, October 13, 2003

Steven Berlin about email

Steven Berlin writes about saving email by better organizing it. All I want is two things: faster search in Outlook (why does it take minutes to scan my 500Mb mail file? Weren't computers supposed to be fast) and rules to categorize my mail a day after I receive mail, so that my inbox contains the most recent entries from which I can work. Then after I have replied, Outlook can move the mails to the correct folder.

Google Hacks

A look at my referer logs showed that quite a few people come here for one of the Google Hacks, so I added a new category with six projects doing something interesting with Google.

Friday, October 10, 2003

Do we need Digital IDs

Kevin Werbag predicts the coming battle whether we'll Digital IDs will become mandatory. A lot industries will be asking for them, with seemingly convincing arguments. But the ability to be anonymous has achieved great things on the Internet. It will be a high price to pay if we loose that.

On the Internet nobody knows you're a dog, the old saying goes. But not for long. Their are a lot of powerful parties that want to end anonimity on the Internet and they'll be leaning on our governemnts to require Digital IDs to surf. They'll claim we'll need it to fight illegal copying of music, spam, terrorisme and that old stalward, kiddie porn. It will sound convincing: you need a drivers license to drive on the highway, so why no driver license for the Information Super Highway?

The openness of the Internet makes it vulnerable to misuse. But if you close it, you also close a source of new ideas. Anybody can implement a new protocol on the Internet and start using it. This is why so many good ideas have developed so fast: you don't need a lot of resources to write a blogging system and maybe more importantly, you don't need a license.

Ubiquous Digital IDs will stop this, because it will require all software to be compatible with it. Not a problem for Microsoft or Sun, but it will be for the new Dave Winer or Shawn Fanning. Stopping innovations at the grassroot level will be high price to pay.

The weird European stability pact

Why again do we have the European Stability? The Germans feared the Euro might be a weak currency, weaker than their D-Mark, if other countries would join with less of a tradition when it comes to paying attention to the state finances. Of course Germany is now one of the countries flouting the rules of the pact. The fact that the Euro is rather strong now, when the deficits in Europe are rising above the limit set in the pact, when the Euro was really weak when the pact was functioning, adds to the irony.

The stability pact was a bad idea of course to begin with; The Euro takes away one tool for individual countries to control their economies, setting their interest rates. So why also limit the options for these countries to control economies using the government budget? It seems not to be important anyway, since France is not going to do anything about their budget deficit and who is going to fight France?

The trouble is of course that the whole pact will be incorporated in the new European Constitution. Dead on arrival, but cut in stone.

Thursday, October 9, 2003

Bombing the Vatican

Whether or not Sadam Hussein had the weapons of mass destruction, most opponents of the war agreed that in principle removing a head of state by force should be possible, if he was a great enough thread to world peace. Currently the head of state that is the greatest thread to human welfare on this planet is the Pope. Removing him from power shouldn't take the American army longer then a few minutes. One daisy cutter should do the job.

So far, about twenty million people have died of AIDS. 42 million people are infected. Correct information about how AIDS is transmitted and access to cheap condoms could have saved most of these people. If no action is taken, 70 million people will die in the next 15 years. This is not only a human cathastrophy of an unheard scale, it also threathens the security of the world. Even the Bush administration agrees.

But the Vatican seemingly doesn't. Telling people not to used condoms is really terrible in a situation where 70 million lives are at stake. Telling them that condoms don't protect and might even cause aids, is downright murderous. The Vatican obviously is a greater thread to world peace then Iraq ever was, so the justification of attacking it should be proportional greater. And it doesn't take a large army either. One daisy cutter really should do the trick. Rebuilding the tiny country as a democracy should not be that hard either, though adding it to Italy would be easier.

Seriously though. Where is the difference between Osama bin Laden calling the faithfull to kill people from the West with arms or the Pope calling the faithfull to kill their own with a virus?

Wednesday, October 8, 2003

Google Talk

Another Google project online. Google Talk. You type a sentence of three or four words and Google find the fifth based on the first four, and the sixth based on word 2..5 etc. Weird dada like poetry follows.

Tuesday, October 7, 2003

Why jobs moving overseas isn't so bad

The same people complaining that economics isn't a science tend to not understand the writings of Ricardo about competive advantages. An article on O'reilly's recently had all the old arguments about computers destroying jobs and jobs going overseas without paying any attention to economic theory.

Simply said, Ricardo argues that if everybody does what he is best at relatively, than everybody will profit the most. Not everybody will profit the same, though. So, if Indians are better in programming than in marketing relative to Americans, then it is best if the Indians program and the Americans do the marketing. But isn't the American economy hurt when the good jobs like programmers move to India? No. Let's consider the American economy as a system. Let's assume it spends 100 on software development. Now, the Indians come and they'll do the same job for 30. That is good for the system, because it is cheaper.

Some programmers become redundant and they'll have to find new jobs. Granted, not nice for them. If they find a job in which they make more than 30, the American economy wins. The programmers in India make more than the otherwise do, so the Indian economy wins too. But what if the programmers can't find a job for more than 30? Well, it that case they can stay programmers and work for the new wage of 30.

So wages are driven down by international competition until everybody earns what they earn in India and we're all poor? Wrong again. The American economy as a system produces a certain amount of wealth. The more people work, the more wealth. The more efficient the system, the more wealth. This is where economic growth comes from. Moving programming jobs to India increases the overall performance of the system and thus the overall wealth production of the system. So, in total people become richer.

This is where it gets tricky. Economic theory doesn't tell us where the extra wealth ends up. Everbody might get richer, or only the people that are already rich. Employment might rise or it might fall. But that is a point where a lot of confusing seems to be about. People tend to reason that employment falls with rising productivity because you need less people to do the same amount of work. This is what the economists call the falacy of the lump of labour.

People are 10x as productive as say a 150 years ago. If the lump of work falacy would hold true, 90% of us would be out of a job. Instead these people moved out of the factories and farms and started to do the things that make our lives so much nicer. So all is well with the moving of jobs overseas? Not really. It would be better to let the people come and work here that are motivated and capable. But the general population doesn't want that for fear of their jobs. Don't get me started on that.

Monday, October 6, 2003

Archean on

Yesterday my archean project was added to, a website about software art. Archean is certainly a beautifull project. The patterns are pleasing and always changing. But is it art? Can software be art?

In times now long gone, there was no real distinction between artists and craftsmen. People did there job and if they were really good, they were artists. Painters or sculpturers were no different in that respect than carpenters or bakers. Indeed Socrates was a sculpturer before he became a philosopher. Not as an artist, but just as a way to make a living.

So can you create art in any job? I suppose so. But some jobs have more possibilties to excell, to be creative and to be original, all aspects we have learned to associate with art. Against these criteria, art is to be expected from the best computer programmers. Like Fredericks Brooks wrote in the Mythical Man Month:

The programmer, like the poet, works only slightly removed from pure thought-stuff.  He builds his castles in the air, from air, creating by exertion of the imagination.  Few media of creation are so flexible, so easy to polish and rework, so readily capable of realizing grand conceptual structures.

It is not only the abstraction of the work, although this freedom allows the programmer to use his creativity and to come up with truely new solutions, to build new stuff in new ways. It is also something different.

A good programmer knows how to judge code not by how it works, but by its beauty. A program that is beautifully written, it will probably work a lot better than a program that is not. Indeed, if a novice approaches the master with a programming solution that is uggly, the master will tell the novice first that it is uggly and why it is uggly. Only when the novice starts to protest that it doesn't matter, because it works, will the master will explain why it doesn't work either.

So is Archean art? In a way it is only the implementation of an algorithm I discovered/designed. The code is not very beautifull, more a quick hack actually. But on some level, Rembrandts paintings were also the implementation of a new algorithm he discovered. Great implementations and a nice algorithm, that is what an artist makes an artist. 

Sunday, October 5, 2003

In favour of Digital Rights Management software

The beauty about a society where a lot of the production is actually information production, is that the economies of scale in an information society work even better than in a industrial society. If I write a program, that will take a certain amount of effort. But the total amount of effort hardly increases if the program is distributed millions of times. The same goes for songs, movies and anything digital.

The problem is of course that if anybody can copy my program, how am I going to make money on it. A tricky one, but banning the copying of the program is hardly the answer. Take music for example. The actual musicians get a few cents per sold CD, while the CD costs 20 US dollars (at least here). Where does the rest of the money go? Distribution and promotion, something that could be done for almost free over the Internet, as is demonstrated by the blog phenomenon.

The wide implementation of Digital Rights Management software that will prevent all kinds of copying will therefore be an economic disaster. Digital things will become as expensive as material things and the money will end up in the wrong pockets. But all freedom loving people should strive to have it implemented as soon as possible. Because people are not that stupid. Offer them a system that takes away there freedom and charges them for it and they'll finally start looking for an alternative.

Take Ms Office. If the technology was out there to prohibit all illegal copying of Office, most companies would be forced to get a legal copy of the software for every single one of their workstation, while currently they probably have too few licenses. But people at home wouldn't buy Office. They would start to look around and find OpenOffice, just as good, but free. Or maybe StarOffice, even better and not quite free. And they would love it. And than the companies would say, hey, if this works and our employees use it, let's use it too.

In the realm of music, the same might happen. Let the RIAA prohibit copying and exchange of all music under their control. Let them try to kill the spirit of music lovers. They won't succeed. People willl want to exchange music and just discover bands that distribute their music over the Internet in a more free way. Not as long as it is possible to exchange any kind of music with Kazaa, of course. But give them the choice between mainstream music and freedom, they'll choose freedom in the end.

Wednesday, October 1, 2003

Visual Poetry Ready

Poetry is supposed to project images in your mind. VisualPoetry translates any text into a series of images by looking up the words on Google image search and projecting the most relevant results as a slide show.