Since the floppy and CD-ROM drives are dynamic drives, you can put icons in two ways: With the floppydrive it works just like the harddrive. But, since the floppy isn't immediately read when the floppy is inserted, I think you have to open the A: drive before Windows sees the autorun.inf file. But I am not sure if that works.
The CD-ROM drive works just like the floppydrive, but it does read the CD when inserted, so you can see the icon immediately.
When you want to change the drive-icon permantely, you can use a lot of freeware programs that can do that for you, but be warned: the autorun.inf file overides those settings, so if you just made a general icon for the CD-ROM drive, it won't be visible when a CD-ROM with an autorun.inf file is inserted. Again, I am not sure about the floppydrive.
edit: with permanent drive icons I mean the icons that are visible when nothing is inserted.