CogniFlight Edge

┌─ cockpit ──────────┐
│  ◉ cam     ♥ hr    │
│    └───┬────┘      │
│     [ rpi ]        │
└───────┬────────────┘
        └─▶ mqtt ▶ cloud

The problem

Fatigue degrades a pilot long before they notice it themselves — blink duration stretches, yawns become frequent, heart-rate variability drops. By the time it is subjectively obvious, performance is already impaired. CogniFlight's answer is an edge device that watches for those signals in the cockpit and streams them to a ground station that can alert operators in real time.

What runs on the device

A Raspberry Pi with a camera runs the whole sensing pipeline locally:

• Vision — a real-time face pipeline computes the eye-aspect ratio (EAR) for blink and microsleep detection and tracks yawning frequency. Aviation-informed thresholds (EAR below 0.15 sustained across frames) flag microsleep events.
• Biometrics — heart rate and heart-rate variability (RMSSD) feed a stress index; sustained anomalies escalate the fatigue assessment.
• Environment — cabin conditions such as altitude join the fusion, since hypoxia above ~3,000 m compounds fatigue risk.
• Identity — pilots enroll with face embeddings, so a device knows whose baseline it is comparing against.

The fused indicators publish continuously to cogniflight/telemetry/{node} over TLS MQTT — a protocol chosen because cockpit connectivity is intermittent and every byte counts. If the link drops, the broker's session semantics detect it immediately; the device just keeps sensing.

Why the edge, not the cloud?

Streaming raw video from a cockpit is a non-starter: bandwidth, latency and privacy all forbid it. Computing fatigue indicators on-device means only compact telemetry leaves the aircraft — and detection keeps working even with no link at all. The constraint that shaped our work was doing real-time computer vision within a Raspberry Pi's thermal and compute budget — tuning the pipeline rather than reaching for a bigger model.

My part: leading the edge team

Eight of us built CogniFlight, split into an edge team and a cloud team. Brian Felgate led the cloud side; I led the edge. On this device that meant building the circuitry — wiring the camera, heart-rate sensor and environmental sensors into the Raspberry Pi — guiding the architecture around what that hardware could sustain, and sourcing the hardware itself. The Python/OpenCV pipeline that runs on it is my teammates' code, not mine.

I also built the other end of the link: the ground station's operator frontend and pilot-enrolment API were mine too. Sitting on both sides — deciding what the device could afford to compute, then rendering what it sent — is the view behind Detecting pilot fatigue in real time on a Raspberry Pi.

Outcome

The full system — edge devices plus the cloud ground station — was our eight-person final-year capstone at Belgium Campus, presented on real hardware with live telemetry. Because Brian and I kept the two teams in step, edge and cloud were built in parallel and integrated continuously — and the result worked well enough to surprise even us. The hardware was made possible by the Atterbury Trust, whose interest-free study loan bursary also carried my studies. It earned 97%, and it convinced me that the interesting engineering lives where hardware limits meet software design.