The real-estate market is splitting in two, with wealthy buyers driving stronger growth at the top while the rest of the market continues to struggle.

From $16 million log mansions in Aspen to $90 million glass-walled penthouses in Manhattan, high-end real estate is defying the broader real-estate slump and weak financial markets. In local real-estate markets with a median home price of $1 million or more, prices are up more than 10 percent year over year, and inventory is down 10 percent, according to Altos Research, the real-estate analytics firms.

Hyper-wealthy markets are doing even better. In markets with a median home price of $10 million or more, prices are up 13 percent or more. The absorption rate – the rate at which inventory is drawn down – has sped up by 11 percent this year, according to Altos.

The growth is being fueled by several factors. Wealthy buyers with good credit can take advantage of today’s historically low mortgage rates. Volatile financial markets and the Euro crisis have scared the wealthy away from stocks and sent them searching for hard assets, like real-estate and collectibles.
At the same time, the newly rich in Latin America, Russia and Asia have begun snapping up properties in America’s most exclusive buildings and neighborhoods.

“It’s become a tale of two markets,” said Michael Simonsen, CEO of Altos. “At the high end, well-financed people have taken advantage of cheap money. And demand is up, inventory is down and prices are responding.”

By contrast, the lower end of the market – especially in more rural areas – is languishing due to high unemployment and lack of financing for buyers, Simonsen said.

In some markets, demand for multi-million-dollar trophy properties is quickly catching up to supply. In Aspen, the inventory of high-priced homes for sale has fallen by more than 50 percent in 2012. In Hillsborough, prices are up by 73 percent and inventory is down by 76 percent.

Joshua Saslove, head of Joshua & Co., the Aspen real-estate brokerage, said high-quality homes in the top neighborhoods are becoming scarce.
He recently sold a $16 million home before it even appeared on the MLS listing. While he has yet to seeing bidding wars, Saslove said he is seeing back-up contacts for some homes, meaning a second buyer is waiting in the wings in case first deals fall apart.

Saslove says today’s buyers of mega-homes fall into two camps: whales and sharks. The sharks are trolling for deals and value, while the whales are looking for one-of-a-kind homes that they love. Both types of buyer are ready to do quick cash deals when they want a property, though some also finance.

“Before, we had a lot more sharks,” Saslove said. “Now we’re seeing both kinds of buyers out there. There is a very low supply of true quality properties right now.”

Paul Boomsma, president of Luxury Portfolio International, the luxury real-estate group, said may wealthy buyers got tired of waiting out the recession and looking for the perfect time to buy.

“People are feeling more comfortable spending again,” he said. “And for those people who were waiting, they realize they can’t keep waiting for the world to go back to where it was. This is the new reality.”

He added that a new crop of wealthy buyers who made their money during the past five years have also added to demand. “The people who have suddenly made their money are feeling good about their situation and ready to go.”

-By CNBC's Robert Frank