When can we
trust a BCI?
A brain–computer interface can make a confident prediction from a poor recording. This dissertation asks whether a lightweight monitor can recognise when that prediction is unreliable.
Explore the experiment
Working titleTransfer of Lightweight BCI Reliability Monitors to Unseen Channel Faults
Starting resourcesComputing + public EEG data
Bounded scopeOne decoder · four monitors · two fault families
Predict an intention.
Then question the prediction.
Use recordings of people imagining left- or right-hand movement. A fixed decoder predicts the task class. A separate monitor estimates whether its output should be accepted or flagged for rejection.
Recorded EEG
Scalp voltage measurements, with the dataset's task cue as the offline reference.
Add noise or lose channels
Modify a copy of a trial at declared severities. Keep the original for comparison.
Predict left or right
Start with CSP features + shrinkage LDA. Freeze it before comparing monitors.
Accept or flag
Combine available signal-quality and model information, then apply a frozen threshold.
The target is prediction error. A corrupted recording can still produce the correct answer; an unmodified recording can produce the wrong one.
Same task.
A less reliable measurement.
Switch between recording conditions to see the concept. These are synthetic traces and scripted outcomes, not predictions from a trained BCI or experimental results.
A correct prediction is accepted. This is the intended outcome.
The slider changes the synthetic fault only. Scenario decisions are chosen examples; no classifier runs in this page.
What could cause this in real life?
A slipping headset, loose electrode, jaw movement or cable disturbance can degrade the recording. A detached electrode may produce noise rather than zeros.
In the experiment, declare a fixed replacement policy for an unavailable channel. Losing a streaming packet is a separate fault: it can remove time samples across several channels.
Learn under noise.
Test on unfamiliar faults.
BCI Competition IV 2a / BNCI 001-2014: 9 participants, 2 sessions, 22 EEG channels at 250 Hz. Use the left/right imagery subset. Keep EOG out of the intention decoder. Dataset description ↗
Unmodified trials + injected noise
Separate decoder training from monitor fitting and threshold selection. Specify three nonzero noise levels using power-based SNR.
Previously unseen channel-loss patterns
Freeze decoder, monitors and thresholds. Evaluate unmodified trials, noise and held-out channel faults to distinguish session change from fault effects.
Calibrated probability or decision margin.
An established score or declared quality indicators.
Distance from training support, e.g. a Mahalanobis baseline.
A small regularised logistic error predictor.
Prevent leakage: keep each original trial and all its modified copies in one split. Fit the monitor on held-out or out-of-fold decoder predictions.
At runtime: no task label, original reference, injected severity or future samples. Existing task labels establish errors during offline evaluation.
Contribution to investigate: transfer and limitations under a defined protocol. Quality–confidence fusion and uncertainty rejection already have precedents; combining them is not itself a new method.
Catch errors without
rejecting everything.
First report the decoder's accuracy under each condition. Then compare monitors at development-selected thresholds and across error–coverage curves. Report results per participant and fault severity.
Missed errors
Accepted incorrect predictions ÷ all incorrect predictions.
False rejection
Rejected correct predictions ÷ all correct predictions.
Coverage
Accepted predictions ÷ all predictions.
Accepted error
Incorrect accepted predictions ÷ all accepted predictions.
Added CPU time and memory, with EEG observation-window duration reported separately. A careful negative result about transfer can still be useful.
Build a small, reproducible
reliability benchmark.
A practical 4–6 month scope using public recordings and a CPU-first pipeline. Completion means one fixed decoder, four monitors, two fault families, clear evaluation and a replay demonstration.
Establish the baseline
Check prior work, ingest the data, lock the task and train the decoder.
Build the experiment
Define faults, implement existing monitors and prevent split leakage.
Run the locked evaluation
Test transfer, compare monitors and analyse participant variability.
Profile, demonstrate, write
Replay recorded signals, measure overhead and finish the dissertation.
A foundation for the PhD.
Expand later to streaming drift, quantisation and timing faults. If equipment becomes available, add controlled signal-generator/acquisition-board tests. The core dissertation can finish using public data alone.
Starting points & scientific context
- BCI Competition IV 2a — official dataset description
- BNCI Horizon 2020 — dataset access and terms
- Monte Carlo Dropout for Uncertainty Estimation and Motor Imagery Classification (2021)
- Robust learning from corrupted EEG with dynamic spatial filtering (2022)
- The Riemannian Potato Field — online EEG signal quality (2019)
- CLSP-REQA — a prior example of EEG quality–confidence fusion (2026)