OK, I did the upgrade, but it didn't go as planned. First off all, it didn't import my old FF0.8 settings, instead it imported my IE settings. Bye bye bookmarks, cookies and history. I tried re-importing it (File -> Import), but FF0.8 wasn't one of the options.
Then I thought to be clever and I manually imported them, by replacing the new files in Documents and Settings/<user>/App.. Data/Mozilla with the contents of the /Phoenix folder. Lucky me they changed the dir name, otherwise all my files have been overwritten. They also changed the directory structure, so I had to do some moving around.
It turned out to be a bad idea, because now it loaded the theme I was using in 0.8, but that theme is not compatible with 0.9RC, so it was all messed up. I thought, let's give the new theme manager a go. But it was useless, because the theme didn't support the theme manager window (I think). I removed all .jar files leading to the theme(s) I had installed previously, but that didn't have any effect. In the end I had to manually edit the chrome.rdf file, removing all references to the old themes. It turned out to be the solution, because now everything was back to normal, including my history, cookies and bookmarks.
Now that I am using 0.9RC, I see it's much faster in startup and I also like the window that asks if I want to close FF when I have multiple tabs open.