There is more of a difference in comfort than just size.
For his wife, two kids and the dog, my brother-in-law has a Prius that's very economical and utilitarian. It projects his enviro-friendly self-image instantly.
My TCH is a much more comfortable, near-luxury car even without moonroof, GPS or seat/mirror/tilt memory. It's not big, but has plenty of head, knee and legroom. It's hybrid drive adds to the comfort by being economical and the quietest car I've owned.