Speakers
Description
Most AI models used in art-historical analysis or image generation are trained on large photographic datasets whose statistical structure differs fundamentally from painted images. This raises a key methodological problem: how can such pre-trained models be adapted to small, historically specific corpora while retaining interpretive reliability?
This paper presents results from the ongoing project ARTofAI, which investigates the integration of generative diffusion models into art-historical research. Focusing on the 17th-century painter Almenak, active in Carniola, we fine-tuned Stable Diffusion using Low-Rank Adaptation (LoRA) on a dataset of 99 image fragments derived from four preserved works. Each fragment was paired with one of four captioning strategies – simple keywords, expert art-historical descriptions, CLIP-generated tags, and Vision-Language Model (VLM) sentences – to examine how different textual inputs affect what the model learns.
The experiments show that while LoRA successfully transfers stylistic surface features such as palette, texture, and brushwork, caption variation has minimal impact on the visual outcome. Models trained on all caption types produced comparable stylistic results, but none reproduced the compositional or iconographic coherence characteristic of historical painting. These findings highlight three broader challenges central to AI-based art history:
1. The domain gap between pre-trained photographic models and historical pictorial data;
2. The need for controlled vocabularies that translate art-historical categories into machine-readable form;
3. The importance of lightweight, transparent methods enabling art historians to experiment with fine-tuning on small, curated datasets.
Rather than aiming at reconstruction or authenticity, this study treats fine-tuning as a diagnostic tool for assessing what diffusion models can and cannot learn from historical material – an approach that informs the development of AI methods genuinely adapted to art-historical inquiry.