I don't really think it's as simple as, to avoid confusion.
I think it used to be said as "wo" in older days, but the language just grew out of it. Anything with a "w" just usually isn't said in japanse anymore. The only reason they still have "wo" is because it's a particle, and even so it's almost always pronounced "o". Even the "wa" is sometimes not spoken, for example when Rika says Ishikawa it sounds like "Ishikaa".
It probably also has something to do with region, or cultural backgrounds. But for the most part, it's almost never pronounced as "wo" from what I can tell.