The museum draws much influence from ancient Islamic architecture, yet has a unique design. It was the first of its kind in the Persian Gulf and has a very large collection of Islamic art, plus a study and a library. A restaurant is planned in the future. Sabiha Al Khemir served as the founding director of the museum from 2006-2008. The museum has a total area of 45,000 m² and lies on the edge of Doha harbour at the south end of Doha Bay. Construction by Baytur Constr.Co. (Turkey) reached completion in 2006, but the museum's interior was subjected to a variety of changes thereafter. The museum celebrated its VIP opening on November 22, 2008, and opened to the general public on December 8, 2008.

The museum houses a collection of works gathered since the late 1980s, including manuscripts, textiles and ceramics. It is one of the world’s most complete collections of Islamic artifacts, with items originating in Spain, Egypt, Iran, Iraq, Turkey, India, and Central Asia. At the age of 91, Pei had to be coaxed out of retirement to undertake this enterprise. He traveled throughout the Muslim world on a six month quest to learn about Muslim architecture and history and read Muslim texts to draw inspiration for his design.

Declining all proposed sites for the museum, Pei suggested a stand-alone island for the structure in order to avoid the encroachment on other buildings. Thus it was built on the water, 64 acres (260,000 m^²) approximately 195 feet (59 m) off the Doha Corniche and surrounded by a park.