Project Description

Our sponsor USGS Astrogeology currently faces many challenges alongside NASA scientists when it comes to generating Image Support Data (ISD) for planetary imagery processing. Users of their current system must download large portions of NASA's SPICE dataset to create ISD files, which is time-consuming and a roadblock. Additionally, their current system using Amazon ECS costs the company near $10,000 a month. We will be implementing a RESTful service that integrates USGS's current software (see related links page) so users can access the ISD data online without downloading massive datasets. This new process also ideally will be free for USGS Astrogeology as well as other organizations using the system. Our solution will implement a caching system to reduce ISD generation time by retrieving pre-calculated ISD files then compress large ISD files to reduce the data footprint. This will run on an AWS server taking advantage of the scalability features it offers.

Link to Project Briefing

Project Briefing

Link to USGS Astrogeology

USGS Astrogeology

In Your Orbit Project Schedule

Task	Done By
Gather requirements	11/01/2024
Create Prototypes	11/20/2024
Implement MVP	02/01/2024
Iterate MVP improvements	03/01/2024
Ship completed solution	05/01/2024

High-Level Requirements

Web Service Development:

Develop a Python-based RESTful web service to handle requests for generating ISD.
The service should interact with existing open-source software (like ALE) to process requests.
It should accept image labels and SPICE kernel quality parameters as input via web requests.

Data Optimization:

Design a format to reduce the size of the JSON-formatted ISD data, possibly through binary conversion, compression, or a combination of methods.
Ensure the new format can be converted back into a standard ALE-supported format.

Scaling and Performance:

Address the challenge of handling large-scale data processing, including thousands of simultaneous ISD requests.
Implement memoization to cache frequently queried data, reducing the time needed for data retrieval.
Develop a cache validation mechanism to update or invalidate cache entries when SPICE data is updated, ensuring data consistency.

Deployment on AWS:

Design the service to be compatible with AWS free-tier for easy deployment and testing.
Aim for a setup that can be quickly deployed using tools like Docker.

Stretch Goals:

Develop a queryable data store for ISD, allowing scientists to access and download cached ISD data more efficiently.