Pre-compute instead of rescan
Repeatedly opening hundreds of flight files was the real bottleneck. Metrics are calculated once during ingestion and reused for later analysis.
Engineering case study
Flight Test
Analytics Database
An analytical PostgreSQL database that converts segmented helicopter flight-test data into instantly searchable engineering metrics.
~85,000historical flight segments
2,000+recorded signals
30 GBactive analytical data
2 h → <10 ssingle-signal retrieval
01 — The challenge
The data existed. The analytical answers did not.
Before this database, a lightweight SQLite index linked flight identifiers to their corresponding files. It made flights locatable, but the analytical information still lived inside the original recordings.
To answer a question across the complete history, a script had to locate every relevant flight, open each file, load the requested signals, calculate the statistics, close the file, and repeat the operation across hundreds of flights.
A request spanning two prototypes and more than 300 flights per prototype could take over two hours. The process could run unattended, but the answer remained unavailable during that time—and some broad analyses were avoided altogether.
02 — The solution
Compute once. Query repeatedly.
Antoine moved the expensive work into the standard processing workflow. Each processed flight is analysed once, then its pre-computed metrics are stored in an indexed relational structure for later retrieval.
The Python data-interaction package identifies Event Marker segments, calculates representative statistical and signal-behaviour metrics for approximately 2,000 recorded signals, and inserts the results into PostgreSQL. A complete flight normally takes about ten minutes to process, occasionally up to fifteen.
Analytical recordFlight → Event Marker → Signal metrics
Metrics are stored at segment level, so an engineer can compare manoeuvres and transitions across flights without reopening the source files.
03 — Data workflow
From recorded flight to searchable metric.
- 01
Flight post-processing completed
- 02
Database task launched by the Flight Test Data Management Platform
- 03
Event Marker signal divides the flight into segments
- 04
Metrics calculated for each available signal
- 05
Results inserted into the prototype PostgreSQL database
- 06
Data retrieved through Python, CSV, or interactive plots
The ingestion task is orchestrated by the existing Flight Test Data Management Platform; this database adds the analytical layer around the processed flight data.
04 — Key engineering decisions
Segment-level analytics
Statistics are stored by Event Marker segment, preserving the operational structure of manoeuvres and transitions.
Relational model aligned with flight-test data
Each prototype uses the same schema: flights, events, and ATA-organised data tables linked through Event Marker IDs.
ATA-aligned table organisation
Dividing data tables by ATA chapter follows the engineering analysis environment and avoids one excessively large table.
05 — Data integrity and reliability
Partial data should remain usable without compromising consistency.
The reliability model is practical: database constraints, controlled access, backups, operational review, and failure handling support continued use of flight-test data as it changes.
Consistency
Unique constraints prevent duplicate flight identifiers and duplicate signal records for the same Event Marker.
Access
Read-only credentials are used by normal users; a dedicated operational account handles automated database interactions.
Recovery
Daily backups are maintained. If connectivity is unavailable, the processing task fails and can be restarted.
Incomplete recordings
Missing signals remain absent from the result set, while null or unavailable recorded values are stored as null.
When a flight is modified, its analytical records are recalculated and replaced. Task execution, database constraints and routine operational use provide the practical validation model adopted by the team.
06 — Engineering access layer
A database designed for the team that uses it.
Antoine developed the Python Database Reader around the database: reusable query templates, CSV export routines, Plotly-based interactive HTML visualisations, and static image export for reporting workflows.
Users select a prototype database, one or more signals, the required metrics, and optional flight filters. The package hides most SQL complexity and returns data ready for Python analysis, CSV export, or plotting.
A dedicated table stores available signal names. Requested signals are checked before extraction, and close-name suggestions help identify spelling mistakes or naming differences. The flight-data team uses these tools regularly and commonly provides CSV datasets or plots to Design Office engineers.
07 — Results
The main gain was not operator time. It was answer availability.
The previous workflow could already run unattended; it simply kept engineers waiting while files were reopened and rescanned. The database reduced the time required to answer an engineering question from hours to seconds, making broad historical analyses practical during a campaign rather than something deferred.
Engineers can quickly check whether a temperature, speed, load, vibration, or other behaviour has appeared previously. Historical temperature datasets have also been extracted to study trends against outside air temperature for the composite helicopter structure. CSV exports let Design Office engineers build their own plots, labels, comparisons, and report figures, including analyses used while preparing certification-related reports.
08 — My role
I turned historical flight files into an analytical engineering resource.
I identified the limitation of repeatedly scanning historical flight files and proposed a database-centred approach after studying Big Data and relational database concepts. I independently designed and implemented the PostgreSQL schema, prototype database structure, flights, events, signal and ATA data tables, indexes, and uniqueness constraints.
I designed the Event Marker segmentation logic, metric-computation and insertion package, and database-processing task. I integrated the ingestion task into the Flight Test Data Management Platform so analytical records could be created as part of the established flight-data workflow.
I developed the Python Database Reader, reusable templates, CSV exports, Plotly visualisations, and signal-name validation tools. I also handled server migration, PostgreSQL version upgrades, backup and access-management practices, and have maintained the solution through approximately five years of operational growth across three helicopter prototypes.
09 — Lessons learned
Optimise access patterns, not only algorithms
The major bottleneck was repeated file access. Changing where computation occurred produced a much larger improvement than optimising the scanning script.
Design the data model before volume arrives
Years of flight history require deliberate schema structure, keys, ATA organisation, and constraints before the database reaches scale.
Data becomes valuable when engineers can access it easily
Python templates, CSV exports, interactive plots, and signal validation turned stored metrics into a usable engineering resource.