Restoring Ancient Greek Vases With Computer Vision
November 2024
TLDR
- Scraped and curated (cropped and masked) datasets of ancient Greek vase images from the Beazley Pottery Archive using Python.
- Fine-tuned Stable Diffusion 2 Inpainting and LoRA adapters to restore missing or damaged sections of pottery using PyTorch on Google Colab.
- Achieved visually appealing restorations on patterned vases and partial success on complex scene vases, demonstrating the need for a larger pottery dataset.
Project Overview
In this project, we experimented with restoring damaged ancient Greek vases using generative vision models. We fine-tuned Stable Diffusion 2 Inpainting on two custom-curated datasets:
- Patterned Vases: 106 high-quality “BLACK PATTERN” images from the Beazley Pottery Archive (BPAD), cropped and masked to boost data efficiency.
- Scene Vases: 6,700 “BLACK FIGURE” AMPORA images containing a scene with humans and divine figures. These were noisier but far more varied.
We tested both full fine-tuning and lightweight LoRA adapters. Our goal was to assess how well these approaches could learn stylistically faithful restorations with limited domain-specific data.
Results
Conclusion: Fine-tuning showed some improvements over the base model — especially on the patterned set — but Stable Diffusion requires far more domain-specific data to learn the intricacies of scene restoration.
Full Report and Code
What I Learned
- Data curation and organization is the most important part of a project like this.
- There is no easy way to finetune stable diffusion. It is a very hacky process.
- Parameter-efficient methods like LoRA are surprisingly effective — especially when compute is limited.
Contributions and Acknowledgements
Project by Nicky Khorasani and Desi DeVaul. Completed for Deep Learning for Computer Vision (Fall 2024), taught by Prof. Peter Belhumeur at Columbia University.