Our Project
Spacecraft missions from around the world have acquired enormous numbers of images and data about the planets and their satellites. Scientists need an easy way to manipulate spacecraft data in order to support planetary rover missions and the eventual human exploration of other worlds.
Planetary scientists are unable to work with raw data from space probes. Satellite and rover data need to be manipulated in order to produce science ready images. Currently there exists no easy way to “set and forget” a job that automates the conversion of raw planetary data.
With the IPP, we have developed a web application that will serve as an easy-to-use drag and drop interface for planetary image processing. Processing jobs will take the form of a pipeline, or set of steps, that researchers and scientists can use to run, save, and share workflows.
Our web application utilizes Directed, Acyclic Graph (DAG) concepts to make the user experience as easy as possible. Instead of having to run a single processing command at a time, users can drag and drop multiple commands into the DAG and connect them as they see fit. The commands are easily configurable using forms in our UI, and DAGs can easily be saved for reuse and shared with other researchers.
In an environment where the speed in which we gather our data is just as important as the data itself, USGS’s Image Processing Pipeline will not only aid in our understanding of other worlds, but increase the efficiency of planetary research.
Our Team
Evin Dunn
Team Lead
-
Python development and scripting
-
Full stack Javascript
-
Database administration
-
Containers
Darius Dumel
Front End Dev
Client Communicator
-
Mobile development
-
Database administration
-
UI design
-
Django development
Sara Huber
Back End Dev
Recorder
-
Python development
-
Web development
-
Agile development
Milton Ibera
Back End Dev
Release Manager
-
Agile development
-
Full stack web
-
Full stack mobile
Rebecca Legget
ISIS3 Integration Lead
-
Web development
-
Pyhton development
-
Java development
Fabio Santos
Project Mentor
AI Researcher (Representation of Knowledge and Reasoning) and Software Engineering Researcher (Helping newcomers to contribute to Open Source Software), CIO, developer, Oracle and PostgreSQL DBA.
Our Sponsors
USGS Astrogeology Science Center
main mission is to provide a national resource for the integration of
planetary geoscience, cartography, and remote sensing to enable our ongoing investigation of our solar
system.
In short they collect and store a vast amount of planetary science data.
Trent Hare
Development of Geographic Information Systems (GISs) including; GIS web sites, tools, and tutorials for researchers. Compiles Earth and planetary datasets for release and publication via ArcMap Server and on-line OGC interfaces for NASA's Planetary Geology and Geophysics, Lunar Mapping and Modeling Project, Applied Information Systems Research Program, Mars Data Analysis, USAID, and other NASA research programs.
Scott Akins
My current role involves project management and software development to provide cloud based searching, processing and analysis tools for planetary imagery data taken from remote sensing missions such as Viking and Mars Reconnaissance Orbiter.
Requirements
Directed, Acyclic Graphs
Images will be processed as a pipeline, or a directed, acyclic graph of image processing steps.
Custom User DAGs
Users will build custom graphs in order to represent the full process of manipulating their image data.
Backend Server
Graphs will be send to a processing server, along with input images.
Image Input
Users will upload one or more images as input.
Output
Users will choose a customizable output area.
Solution
We have created a web-based graphical interface for users to interact with the pipeline. Like the original solution, users can choose images through Pilot, which will then be processed through the pipeline. To make the user interface, we use n8n.io and create nodes for ISIS3 commands. Users are able to drag and drop these blocks into the diagram in any order. The user are able to see the current progress of the pipeline, and . We will have intermediate storage between each step, which will allow us to use the output of one function as the input to another.
Technologies
- n8n: To provide a web interface, we decided to use N8N. N8N provides an existing node and graphing interface, which can be modified for our uses. We will add nodes for functions in ISIS, which the users will be able to drag and drop into place in the graph.
- Minio: In order for the output of one node to be used as the input to another, an intermediate storage step was needed. We decided to use an Minio S3, which is compatible with Amazon S3. This storage server will allow us to store data between each step of the pipeline.
- Python: For our main programming language, we decided to use Python. Our team memebers are the most familiar with Python, and there is a library for ISIS written in Python (Pysis). We will use Python to call ISIS3 commands in our pipeline
- Pysis:Pysis is a library for calling ISIS3 commands in Python. We will the Pysis library to call the commands in our pipeline.
- NodeJS:Since N8N uses NodeJS, in order to modify the N8N interface we will need to use NodeJS.