Drone Image Processing using Open Drone Map (ODM)

[nrjadkry]

Overview

This pull request introduces a comprehensive image processing pipeline that:

• Downloads images from MinIO (S3-compatible storage).
• Processes the images using NodeODM (via PyODM).
• Uploads the processed results as a ZIP file back to MinIO.
• Integrates the processing into a FastAPI application using background tasks.
• Provides endpoints to initiate processing and check task status.

Key Features

DroneImageProcessor Class: Manages the entire processing workflow, including downloading images, processing with NodeODM, and uploading results.

S3 Integration: Uses existing functions in s3.py to download the images and upload the processed imagery back to MinIO

FastAPI Background Tasks: Implements background processing to keep the API responsive.

Task Status Tracking: Introduces a TaskManager to track the status of processing tasks.

API Endpoints:
POST process_imagery/{project_id}/{task_id}/ Initiates processing

[spwoodcock]

Nice work 🙌

This should function is it is, but I left some comments for optimisation if needed.

There are many areas we can optimise for sure, but it's good to get a first version working before over-optimising 😁

[spwoodcock]

I would recommend asyncio over threading for this type of downloading onto a filesystem.

asyncio.gather would work nicely and be less blocking (using a single thread via coroutines).

Example from FMTM:

https://github.com/hotosm/fmtm/blob/35e7240cc78d62c288f0ea60aef0eb7bde8beaed/src/backend/app/central/central_crud.py#L560-L570

But not this would mean you need to update functions to async and use await

[spwoodcock]

As you rightly say below, once everything is working and we are happy, this definitely needs to be changed getting elevation data by querying the cloud optimised DEM. A DEM can be quite large, so downloading every time we process imagery is very inefficient