If you have replaced a package with another version, then you can just delete the package(s) you don't need by deleting the PackageName-versioninfo.egg file or directory (found in the installation directory).
If you want to delete the currently installed version of a package (or all versions of a package), you should first run:
easy_install -m PackageName
This will ensure that Python doesn't continue to search for a package you're planning to remove. After you've done this, you can safely delete the .egg files or directories, along with any scripts you wish to remove.