Detecting pilot fatigue in real time on a Raspberry Pi

Our final-year capstone, CogniFlight, had a one-sentence brief: detect pilot fatigue before the pilot does. The interesting part wasn't the machine learning. It was that the machine had to be a Raspberry Pi in a cockpit, and the answer had to arrive in real time.

Eight of us built it. My seat was the ground station — the operator dashboards and pilot enrolment — so this is the view from downstream: what the edge team had to solve, as experienced by the person whose UI had to render their answers.

Why not stream video to the cloud?

The first architecture everyone sketches is a camera streaming to a GPU server. It dies on contact with reality:

• Bandwidth. Cockpit connectivity is intermittent and narrow. Continuous video is a fantasy.
• Latency. A microsleep lasts a few seconds. A detection that arrives late is a log entry, not a safety system.
• Privacy. Nobody certifies a system that exfiltrates cockpit video.

So the constraint became the design: all sensing happens on the device, and only compact fatigue indicators leave the aircraft.

What the device actually computes

The pipeline fuses three signal families, each cheap enough to run continuously:

Vision. From facial landmarks we compute the eye-aspect ratio — the ratio of vertical to horizontal eye openness. It collapses when the eyes close. A sustained EAR below ~0.15 across consecutive frames is a microsleep candidate, not just a blink; yawning frequency joins it as a slower-moving signal.

Biometrics. Heart rate and heart-rate variability (RMSSD). Falling HRV correlates with fatigue and stress; sustained anomalies escalate the assessment.

Environment. Cabin altitude matters more than people expect — above roughly 3,000 metres, mild hypoxia compounds every other fatigue signal.

Each indicator is a small number computed from a rolling window. That's the trick that makes a Pi sufficient: the expensive operation (face landmarks) runs on a modest frame budget, and everything downstream is arithmetic.

The telemetry contract

The device publishes fused indicators over TLS MQTT to a topic per node:

cogniflight/telemetry/{edge_username}
  { "ear": 0.21, "hr": 64, "hrv": 38.2, "yawns": 3, ... }

MQTT earned its place: tiny packet overhead, QoS levels for lossy links, and last-will semantics so the ground station knows immediately when a node drops — the broker detects the disconnect instead of someone inferring it from silence. When the link disappears entirely, the device keeps sensing; the pilot doesn't lose protection because the network blinked.

What I took from it

The lesson that stuck: put the computation where the constraint is. We didn't choose edge computing because it was fashionable — every alternative was disqualified by physics or policy. Once you accept that, the engineering becomes honest: profile the pipeline frame by frame, budget every millisecond, and let the cloud do what it's actually good at — storage, dashboards and analysis after the fact.

The examiners gave the system 97%.

The case studies: CogniFlight Edge — the Pi — and CogniFlight Cloud, my seat on the ground.