As others have said, there are a lot of variables and I don't think we'll ever get a clear answer. I personally use either a headset of "speaker phone" option when I'm on the phone while driving. I also don't try to hold business conversations or anything where I need to take notes (mental or actual). I have a fairly long drive so I often call my husband on the way home and we talk about our days etc.... BUT I'm always aware that my primary job is driving! My husband hears me commenting on other drivers, traffic hold ups etc during those conversations. I also remember one memorable occasion when I was driving home behind my daughter just after we had picked up a new car for her (literally on our way home from the dealership) - she was ahead of me and all of a sudden, all my husband heard was "OH MY GOD - OH MY GOD!" ... coming down a steep hill on a multi lane road (not freeway), a herd of deer had just entered traffic... cars were slamming their brakes etc. Daughter (ahead of me) successfully avoided hitting any deer or other vehicles, a couple of cars weren't so lucky. AFTER it was evident that everything was ok, I explained to him what happened.

I personally don't think cell phone conversations are any more distracting than if I had that person in the car with me. In fact, it's possible that I'd be glancing at them occasionally so having them IN the car may actually be more distracting. I think that we Americans tend to go on "auto pilot" too much and therefore do not pay enough attention in general. The cell phone just makes a handy target.