I think you would be fine using the trailer hitch (trailer kit works too) to attach a bike rack. I have seen two different Prius vehicles with hitch-mount bike racks running around town.

2004 Toyota Prius
Package #9, Millenium Silver
"Boo"

Mods: Always-hot Accessory Port, Rockford Fosgate DMP1 60GB MP3 Player
On Order: CoastalETech's Nav Upgrade, CoastalETech's ViewTech