Like this>
How To Clear the Past Items or Icons List in the TaskBar
1.Open Regedit Navigate to:
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\TrayNotify
2.Delete the IconStreams and PastIconsStream values
3.Open Task Manager, click the Processes tab, click Explorer.exe , and then click End Process .
4.In Task Manager, click Applications, click New Task , type explorer , and then click OK .
This will remove all entries then programs as used are added again.