I unlock my Prius ('06) on the passenger side most of the time, and have never had a problem putting it in gear. I don't usually put it in "Park" when I stop, just turn it off (I use the e-brake, just an old habit), so the other night I tried putting it in "Park", turning it off, then doing what you describe - opening the passenger door (which opens all doors), getting in, starting it up (all with the key in my pocket), and putting it in gear. Again, no problem. I think yours is "differently-abled"

.