The clinical need
- What is to be resolved, and why current care cannot do it.
- For which patients, and where in care the need arises.
- Which decision the answer informs, who makes it and when.
Our approach
Clinically grounded evaluation means a model is tested against what clinicians actually need, under the conditions where it will be used, and judged by the decision it is meant to inform.
To make that possible, clinicians and developers need a shared way to say what is asked and what is offered. This page describes the idea. It is a foundation, not a finished standard.
Four parts of every statement.
A clinician states a need. A developer states a claim. Both cover the same four parts, so the two can be read side by side.
The conditions of use
- Which signal, recorded with which device, and how it is processed and stored.
- The state of the patient during recording.
- What counts as a usable recording, and where the claim stops holding.
How success is judged
- Where the right answer comes from, and who decides it.
- Scoring at the level of the decision, with mistakes weighed by what they cost.
- A comparison with current practice, checked at other sites and in other groups of patients.
A shared format
- One template and one set of terms for every statement.
- A claim matched, point by point, to the need it addresses.
- A plain-language summary, and a version that is updated as the evidence grows.
Illustrative example
A need and a claim, side by side.
This example is made up. A team in an emergency department needs an early flag for patients who will need an urgent procedure. A developer reports a model that scores well on a public benchmark.
What the clinicians state
- Need: urgent cases are recognised too late.
- Patients: adults arriving with a suspected serious problem.
- Decision: whether, and when, the on-call team is called.
- Conditions: recorded on arrival by busy staff, sometimes on a moving trolley or in an ambulance.
- Success: catches urgent cases at a false-alarm rate the team can accept, tested on consecutive patients at several hospitals.
What the benchmark shows
- Claim: a high score on a public dataset.
- Labels: copied from an older device’s automatic reading.
- Data: one archive, recorded at rest in a clinic.
- Measure: a single average over all recordings.
- Conditions: not stated.
Read side by side, the mismatch is plain: different patients, a different setting, different labels and a different idea of success. The score may be real, but it does not show the model meets this need. Writing both in one format makes that visible before anyone relies on it.
No one group writes a statement alone.
Technical researchers help clinicians turn a problem into a task that can be tested. Clinicians help researchers say what a model offers in real care. Device makers, regulators and patients add what neither can supply.
Building on existing work.
The approach complements the datasets, benchmark suites, reporting guidelines and data platforms the field already has. It gives them a clinical specification to consolidate around, and does not compete with them.
Agreed definitions already work within single areas of biosignal AI. We aim to make that practice the norm, one clinical need at a time. Some of these efforts are listed on the resources page.
Help write the first statements.
Everyone whose work depends on biosignal AI is welcome. Current members review every application.