Ballmer has been working on the changes with a small group of trusted staff and board members, but left out the wider management team, sources told The Wall Street Journal's All Things Digital blog.
The changes are expected to further Ballmer's push to making Microsoft a "devices and services" company, and could lead to more executive-level departures.
It would be funny if Ballmer did nothing in the end, but no one thinks that’s possible now
"It feels like it is going to be titanic — that Steve is doing this change for his legacy," a source told All Things Digital. "And it’s the first time in a long time that it feels like that there will be some major shifts, including some departures."
"It would be funny if Ballmer did nothing in the end," the source added. "But no one thinks that’s possible now."
After Windows 8 was released, head of Windows Steven Sinofsky departed, leading to suggestions from insiders that Ballmer was looking to end competition between divisions and foster greater collaboration inside the company.
"What I'm hearing over and over is collaboration and horizontal integration is the new mantra," said one Microsoft insider at the time. "They [top management] understand that, if they don't move to a model where devices and software are more integrated across the entire Microsoft system, they are in a weak position."
Ballmer's last major reorganisation was in 2008, when Kevin Johnson - who headed up the failed attempt to buy Yahoo - left to join Juniper Networks, and his Platform and Devices division was split into two departments, Windows and Online.