From Molecules to Masterpieces:
AI-Powered Insights into Cultural Heritage
A new interdisciplinary collaboration brings together fields that seldom converge at this depth: molecular imaging and spectroscopy, mathematical modeling, machine learning, art conservation, and art history. The aim of this project is to shed light on how and by whom artworks were originally created, how they originally looked, and how they have changed over time.
In museums and archives across the globe, paintings and cultural artifacts contain histories that cannot be discerned from their visible surfaces alone. Artworks are often decorated with several layers to create a desired visual appearance, sometimes even hiding earlier compositions or original intentions. Unfortunately, many pigment layers also change with time and exposure to light and the atmosphere. Virtually unraveling these strata can reveal artistic decisions, available materials, and the gradual processes of aging. Yet much of this record remains inaccessible with conventional analytical methods.
This research will develop state-of-the-art molecular imaging and advanced artificial intelligence techniques to illuminate these hidden histories. Using fused multimodal data, the research will model degradation and help us understand how aging alters pigment appearance. Using this information, we will develop a predictive framework capable of virtually "rewinding time." This will enable the reconstruction of an artwork’s original colors, offering powerful new tools for restoration, visualization, and historical interpretation. These approaches may also assist in attributing authorship by identifying subtle stylistic and material signatures.
The research will engage a broad range of scientific fields, providing art historians and museum conservators with new means to study and preserve artworks, guide restoration with scientific clarity, and deepen historical understanding. Furthermore, the machine learning techniques developed for analyzing multimodal data of artworks are expected to contribute more broadly to artificial intelligence, for example in fields such as biological imaging where comparable multimodal challenges arise.

Funding

We are grateful to Schmidt Sciences for thier generous support provided through the Humanities and Artificial Intelligence Virtual Institute (HAVI).
Schmidt Sciences is a nonprofit organization founded in 2024 by Eric and Wendy Schmidt that works to accelerate scientific knowledge and breakthroughs with the most promising, advanced
tools to support a thriving planet. The organization prioritizes research in areas poised for impact including AI and advanced computing, astrophysics, biosciences, climate, and space - as well as supporting researchers in a variety of disciplines through its science systems
program.