Planner
Detects new flight triggers, validates incoming data, creates tasks, and manages priorities and dependencies.
Engineering case study
Flight Test Data
Management Platform
An event-driven platform that automates, orchestrates and monitors the processing of flight-test data from aircraft landing to cloud availability.
~700flights processed
15+automated tasks
5–10×faster workflow
Dailyengineering-team use
01 — The challenge
Engineers were supervising the workflow instead of analysing the data.
Before the platform, flight-test engineers waited for data, manually retrieved and organised files, launched independent scripts, monitored each step, and reacted to failures themselves.
The real constraint was not just processing speed. Repetitive operational supervision consumed attention that belonged on engineering analysis and decision-making.
02 — The solution
One observable system from landing to cloud availability.
The platform coordinates the end-to-end flow while preserving the validated logic of the existing engineering tools it integrates.
PostgreSQL
The single source of truth for flights, tasks, status, file locations, and execution history.
Workers
Execute ready tasks based on priority, dependencies, and available capacity—critical work first, overnight work later.
File Manager
Performs integrity checks, storage optimisation, compression, and cloud synchronization.
Dash-based monitoring provides live task status, worker activity, notifications, retry controls, and priority management.
03 — Workflow
Six steps, no manual handoff.
- 01
Flight data transferred after landing
- 02
JSON event trigger created
- 03
Automatic validation of incoming data
- 04
Task planning with priorities and dependencies
- 05
Parallel execution by workers
- 06
Automatic synchronization to the company cloud
04 — Key engineering decisions
Event-driven architecture
Processing starts automatically as soon as flight data becomes available. No manual launch is required.
Dependency-aware scheduling
Tasks run only after their required predecessors succeed, protecting reliability while maximizing parallelism.
Priority-based execution
Critical deliverables are produced immediately; resource-intensive background processing is automatically delayed until overnight.
PostgreSQL as the source of truth
Workflow state lives in PostgreSQL rather than application memory, providing transparency, traceability, and straightforward maintenance.
05 — Reliability
Validation is part of the workflow.
Automated checks cover flight identifier consistency, raw and converted files, mandatory auxiliary data, dependencies, and failure propagation. Each execution leaves an auditable history in PostgreSQL.
The platform was continuously validated with flight-test engineers throughout deployment, ensuring operational behaviour stayed aligned with how campaigns actually run.
06 — Distributed processing
Fast feedback, even away from the main site.
During remote campaigns, critical processing runs locally while intermediate files are compressed for limited-bandwidth transfer. Remaining work automatically continues on the main server.
This keeps critical reports available quickly without treating bandwidth as an unlimited resource.
07 — File and cloud management
Data operations built into the platform.
Additional services perform nightly storage integrity checks, compress intermediate datasets, convert recorded display videos, verify obsolete indexes, and synchronize approved data to the company cloud.
Intermediate TDMS datasets are compressed by approximately 10× before archival, materially reducing storage requirements without turning optimisation into a manual cleanup task.
08 — Results
A production workflow that makes engineering time count.
09 — My role
I built the platform around the realities of flight-test work.
I designed and developed the overall platform architecture, workflow orchestration engine, PostgreSQL data model, Dash user interface, worker system, monitoring tools, file-management services, and cloud-synchronization package.
Existing engineering tools were integrated into the platform while preserving their validated processing logic. Throughout the project, I worked closely with flight-test engineers to validate the workflow, improve usability, and refine the platform from operational feedback.
10 — Lessons learned
Build reliable automation
Reliable orchestration is often more challenging than implementing individual processing algorithms.
Design for observability
Clear monitoring and traceability dramatically reduce operational effort.
Keep humans focused on engineering
Automation should remove repetitive operational work so engineers can concentrate on analysis and decisions.
