Because in game design, you throw out a huge portion of the work you do before a finished product ever reaches the market. Typically for budget reasons, but also on occasion because of scope, and even for later balance or game feel some things just have to go. this feature still exists in some small part as the lobby players are filtered through pre-match, and was replaced in functionality with the merc quarters probably for the sake of server load. It's probably easier to put everyone in their own little house than to keep specific groups all constantly connected to each other at once in addition to the actual battle fields running.

you'd be surprised by the amount of content which is created, but never sees the light of day. and that's just how the industry works.

If I am not mistaken, that was created on an entirely separate game engine. After a couple years of development, they found out that they really couldn't do what they needed with that engine on the PS3, because I don't think they realized it couldn't really handle it.

So they basically had to start from scratch with a new game engine.