Dependencies before priority
A high-priority Limit Report cannot start until post-processing succeeds. As soon as that dependency clears, it becomes eligible immediately—even when several flights are queued.
Production engineering system · technical ownership
Flight-Test Data
Operations
A platform that turns aircraft data arriving after landing into prioritised, traceable engineering outputs—without relying on an engineer to orchestrate every processing step.
Since 2022evolved in production
~15planned tasks per flight
700+flights across its history
~50 GBraw data per flight hour
01 — The operating problem
The human was the workflow engine.
Flight Test Instrumentation (FTI) converts the aircraft’s raw recordings — the Chapter 10 recorder format — into TDMS, the measurement file format the analysis tools read. From there, an engineer in Flight Test Data Analysis (FTDA) still had to select files, launch processing in DIAdem — the measurement-analysis environment used by the team — and its VBS scripts, wait, start reports and fatigue calculations, and monitor each handoff.
The clicks were not the main cost. Human latency between stages, forgotten tasks, sequential execution and limited status visibility became increasingly risky as more processing was automated and several flights or prototypes could be active.
The target operating model became clear: automation executes deterministic work; the duty engineer supervises exceptions and engineering decisions.
02 — Evolution
Built incrementally around operational reality.
This was not launched as a formal platform programme. Each generation addressed the next constraint exposed in production.
- 2022
Manual scripts → central GUI
A single interface replaced repeated file selection, script loading and manual sequencing for the first four or five processing tasks.
- 2024–25
GUI → automatic trigger
New-flight triggers replaced active supervision, while the main post-processing implementation moved from VBS to Python.
- Q1 2026
Persistent orchestration
PostgreSQL, a Planner and a Worker introduced durable state, priorities, dependencies, parallel execution and visible failures.
- June 2026
Distributed field operation
The same operating model supported one remote flight-test campaign after preparatory site and network exercises.
03 — Current architecture
Persistent state makes the workflow visible and recoverable.
Task definitions and priorities live in JSON configuration. PostgreSQL holds planned work, status, dependencies, Worker assignment and failure details. A continuously running Worker selects only work that is eligible under priority, dependency and execution constraints.
The database is the operational source of truth—not an in-memory queue—so the team can inspect, recover and relaunch work without reconstructing what happened.
- 01FTI delivery
Chapter 10 is manually converted to raw TDMS, uploaded to the shared storage and followed by a folder trigger.
- 02Python Planner · ~3-minute scan
Checks naming, source consistency and supporting flight information, then plans approximately 15 tasks.
- 03PostgreSQL
Persists priorities, dependencies, execution state, assignment, traceability and Python errors.
- 04Python Worker
Locks eligible work and executes critical tasks first, independent work in parallel and lower-priority work overnight.
- 05Outputs and operations
Reports, decoded data and engineering datasets are surfaced with dashboard status, desktop notifications and retry controls.
04 — Engineering decisions
Scheduling that reflects engineering urgency.
Guarded parallelism
Independent tasks can run while the roughly 45-minute Limit Report is executing, while explicit concurrency rules protect fragile report and PDF operations.
Immediate vs overnight work
Critical engineering outputs run first. Storage and database maintenance can wait until night or be manually reprioritised when operations require it.
Operability over complexity
The design supports multiple Workers, but normal operation deliberately uses one Planner and one Worker on the same machine for a three-person operating team.
05 — Verification and operations
Test the workflow; preserve validated engineering logic.
Major orchestration changes are exercised with historical flights, controlled folders, simulated triggers and known outputs before closely observed production rollout.
When processing logic itself changes—as in the VBS-to-Python migration—outputs are compared directly with the established implementation. This separates workflow validation from engineering-algorithm validation.
On failure, the dashboard counter increments, the duty engineer receives a desktop notification and the Python traceback is stored in PostgreSQL for investigation and relaunch.
06 — Remote campaign
Critical capability stays close to the aircraft.
A campaign server can run a local Planner and Worker while sharing the PostgreSQL operating model with the main site. Worker assignment and priorities determine where each task runs, with local reassignment available when connectivity prevents central processing.
- Process locallyPost-processing, the Limit Report and ARINC 429 avionics-bus decoding can run at the campaign site.
- Compress and verifyData is compressed and a BLAKE3 checksum is calculated before transfer.
- Continue centrallyAfter decompression and checksum verification at the main site, remaining tasks continue on the central Worker.
Operational evidence: preparatory site and network tests followed by one remote flight-test campaign in June 2026, without a major platform failure.
07 — Measured and defensible impact
Less latency, earlier engineering feedback.
The main gain is dependable flow: automatic sequencing removes human waiting time, parallel work prevents long tasks from blocking unrelated outputs, and failures become immediately visible.
The roughly one-hour versus two-hour turnaround comparison reflects several architecture and implementation changes, not a single measured multiplier. Faster Limit Report availability can expose an inspection need before the end of the day and protect the next flight window.
The three operators and the 50–75 engineering users are different groups: three engineers run the platform, while a wider engineering population consumes what it produces. The two figures are not additive, and they are not the same population as the user counts on the infrastructure integration case study.
08 — My role
End-to-end technical and operational ownership.
I initiated the first GUI, designed the automatic workflow and Planner/Worker/PostgreSQL architecture, and developed approximately 99% of the platform code. I also built most integrated processing capabilities, the dashboard, File Manager, Cloud Uploader and remote-campaign architecture.
Ownership continues after deployment: I gather operator feedback, define improvements, test and roll out major generations, prepare documentation, train colleagues and support production troubleshooting across FTDA, FTI, flight-test engineering, specialist engineering and infrastructure interfaces.
The boundary matters: I designed and developed the orchestration platform and most processing capabilities, while integrating specialist tools and engineering logic produced by colleagues. This is technical and platform ownership, not authorship of every algorithm or formal programme management.
09 — Lessons from running it
Architecture follows operations
The system grew from a practical GUI into persistent and distributed orchestration only when operating conditions justified each step.
Observability changes the job
Durable state, visible failures and notifications let a small team supervise exceptions instead of continuously shepherding work.
Simple operation is a design outcome
Retaining multi-Worker capability while colocating the normal Planner and Worker made the platform easier for three direct operators to run.