1. Exit Mozilla Firefox.
2. Browse to the "chrome" directory of your Mozilla Firefox application directory.
3. Delete "chrome.rdf" and the "overlayinfo" directory (they will regenerated on next Firefox startup).
4. Delete [extensionname].jar (or the [extensionname] directory, if it has one).
5. Remove every line with a reference to [extensionname] from "installed-chrome.txt". Usually there are two: one to the content and one to the locale.
6. A few extensions might use additional files. TBE, for examplem has a "tabextensions.js" in "[/path/to/firefox]/defaults/pref/" to store its defaults in. Search through your application dir for such files and delete them.
7. Restart Mozilla Firefox.
It's probably a good idea to make backups of the files/directories before deleting/changing anything to be able to restore the status quo in case anything goes wrong.
Note: The procedure above applies only to extensions that are installed in the application directory. To remove extensions from the profile, your safest course of action is to create a new profile and transfer over files from the previous one.