I don't see much of a difference, if anything, I see a net improvement in the summer, despite the AC usage, because of faster warmup time. The reason for that is because I already throw away a large portion of my gas mileage climbing the large hill to my home. On the way back down, the hill is too steep to be able to recover all the energy with regenerative braking. However, in the summer, I actually run the AC on max in recirc when coasting down, so the AC is powered with "free" energy that would otherwise be wasted on braking. By the time I reach the bottom, the car is so cold that only minimal AC is necessary to maintain 82 degrees for the rest of my trip.

Balancing the AC load to make it work harder when going downhill should be part of the "Econ" mode on the car by default -- it makes a big difference.