Project Requirements
Requirement
Create a system to search through images in the Special Collections and Archives(SCA) using images.
Description from our client
Access images in the Special Collections and Archives(SCA)
---
Interpret Images
Common interpretation tools include Vision Transformers(ViT) and Convolutional Neural Networks(CNN). These convert an image into an embedding, representing the meaning of the image.
ViTs and CNNs must be trained before they provide meaningful interpretations. They are trained by pairing images with text descriptions. If we choose to use pre-trained ViTs/CNNs, we will fine-tune them to the sources in the SCA.
Store the Interpretations of SCA Sources
The machine's interpretation of a source is a list of n numbers, representing an embedding of size n. In a n-dimmensional vector database, this embedding will be linked to the corresponding source on the SCA's website.
Search for Similar Images
The system will first interpret the user's image as an embedding. A comparison tool such as Cosine Similarity and Euclidean Distance will provide a similarity score for each source in the Vector Database. The images with the highest scores will be returned to the searcher.
Connect to Cline Library's Website
Our system will be accessed through a page on the Cline Library's Website.