Mosaic Canyon is considered a geologic "outdoor museum". Walk leads into the canyon narrows, where the surrounding rock walls are composed of smooth, water-polished marble. The canyon formed because a system of cracks, known as faults, developed in the area millions of years ago. Running water has been channeled into these faults for centuries and has gradually carved out the canyon to its present size.

The most common rock formation in the canyon is the Noonday Dolomite. This limestone is rich in magnesium and formed 750 to 900 million years ago when the area was part of the Pacific Ocean. This sedimentary material was later buried to great depths by younger materials and was subjected to pressures and temperatures exceeding 1000 degrees Fahrenheit. As a result, much of the limestone was altered, or metamorphosed, into marble.