fix(backend): replace hardcoded /tmp paths with Python tempfile module

[m4cd4r4]

Fixes #597 (backend portion).

Summary

Every backend write to a hardcoded /tmp/... path is replaced with temporary files and directories managed by the tempfile module. This stops the server's disk from filling up over time because temporary files are now cleaned up automatically (either via context managers, finally blocks, or Starlette BackgroundTasks), and makes the code portable across Linux/macOS/Windows.

Rebased onto current dev (2026-05-12) and extended per @spwoodcock review to cover the vendored drone-flightplan package in this same PR.

Changes - backend app

app/gcp/gcp_routes.py - save_gcp_file now writes the upload via tempfile.NamedTemporaryFile(delete=False) and unlinks the file in a finally block once the S3 put returns.

app/waypoints/waypoint_routes.py - both generate_waypoint and generate_wmpl_kmz route flightplan generation through one tempfile.mkdtemp directory per request. For the download path the temp dir's lifetime is handed to FileResponse as a BackgroundTask (shutil.rmtree) so the file can still stream before being removed. For the non-download path the temp dir is removed synchronously. Errors during generation wipe the temp dir immediately. The generate_kmz_with_placemarks endpoint follows the same mkdtemp + BackgroundTask pattern.

app/waypoints/flightplan_output.py - build_flightplan_download_response gets an optional cleanup: BackgroundTask parameter that it passes through to FileResponse(background=...). This keeps the download endpoints from having to choose between "leak the file" and "delete it before FileResponse can read it".

app/projects/project_logic.py - both the terrain-follow branch in update_projects_flight_metrics and the process_waypoints_and_waylines helper now use tempfile.TemporaryDirectory context managers. Cleanup happens automatically, including on exceptions, replacing the hand-rolled os.makedirs + try/finally shutil.rmtree. Also removes the un-created /tmp/{uuid}/dem.tif path that update_projects_flight_metrics was passing to get_file_from_bucket.

app/jaxa/upload_dem.py - the DEM download worker replaces Path(f"/tmp/tif_processing/{project_id}") with tempfile.mkdtemp(prefix=f"dtm-dem-{project_id}-") and a finally: shutil.rmtree(...). The previous hand-rolled glob-and-unlink cleanup is gone.

app/projects/classification_routes.py - the reflight download endpoint now writes the KMZ inside a tempfile.mkdtemp directory and schedules shutil.rmtree as a BackgroundTask instead of os.remove on the single file.

Changes - vendored drone-flightplan package

Per @spwoodcock comment - the in-repo copy at src/backend/packages/drone-flightplan/ got the same treatment so the package is consistent with the calling code.

drone_flightplan/create_flightplan.py - the terrain-follow branch wraps the elevation-geojson write in a tempfile.TemporaryDirectory(prefix="dtm-flightplan-"), replacing the hardcoded "/tmp/output_file_with_elevation.geojson" that collided between concurrent calls.

drone_flightplan/output/dji.py, drone_flightplan/output/litchi.py, drone_flightplan/output/mavlink.py, drone_flightplan/output/qgroundcontrol.py - each output writer's output_file_path default changes from a hardcoded /tmp/... literal to None, resolved at call time via tempfile.gettempdir(). Caller-supplied paths still work exactly as before.

A matching PR against hotosm/drone-flightplan upstream will follow so the two stay in sync.

Verification

• python -m py_compile passes on all 11 touched files
• python -m ruff check passes with no findings on all 11 touched files
• grep -rn "/tmp/" src/backend/app/ src/backend/packages/drone-flightplan/ returns no matches other than a single comment line referencing the system temp root in upload_dem.py

Test plan

• Unit/integration suite green in CI
• Manual: download a flightplan via POST /waypoint/task/{task_id}/ with download=true - file should be served and the temp dir should be gone from the server after the stream
• Manual: download a reflight KMZ via the classification reflight endpoint - same expectation
• Manual: run the JAXA DEM ARQ job against a test project - temp dir should be gone whether the job succeeds or fails

[spwoodcock]

Cool! I've wanted to see this for ages - thank you 😁

Only comment so far, without a good review, is that the drone-flightplan package is actually in this repo too, under src/backend/packages/drone-flightplan, so could be updated too =)

[spwoodcock]

This is really useful & appreciated, but I just realised it was made against main, instead of against the default dev branch 😅

It's significantly out of sync now - sorry!

[github-actions]

Contributor Signature Required

Thank you for your contribution!

Before we can accept your pull request, you need to sign our Contribution Policy.

Why do I need to do this?

• Ensures you have read and will abide by our contribution guidelines.
• Adds a small barrier for bot / AI account contribution, which we do not allow.
• We care deeply about our community at HOT, and want everyone to be on the same page.

How to sign

To sign the agreement, please comment on this PR with:

I have read the CONTRIBUTING.md document and I hereby sign and agree with the guidelines

---

⚠️ Note: You only need to sign once. Future contributions to this repository will not require re-signing.

---

I have read the CONTRIBUTING.md document and I hereby sign and agree with the guidelines

---

_{You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.}

[m4cd4r4]

Thanks for the nudge on both, @spwoodcock - this is now rebased onto current dev and the in-repo drone-flightplan package is covered in the same PR (defaults switched from hardcoded /tmp/... to tempfile.gettempdir() resolved at call time, plus the elevation-geojson terrain-follow path moved into a TemporaryDirectory). Calling code is unaffected because explicit paths still flow through.

I'll send a matching PR against hotosm/drone-flightplan so upstream and the vendored copy stay in sync. CI is running now.