Work · Longitudinal reporting
Clinical tuple evaluator
- Demo ready
- Synthetic demonstration
- Companion to current research
A versioned evaluator for structured report content: canonicalisation, one-to-one exact matching, tuple precision and recall, and report-level false-content and omission states with explicit denominators.
Problem
Report generators are often scored with language metrics. A clinically useful score has to say which structured facts were right, which were invented and which were left out, and it has to keep the report count separate from the tuple count.
Contribution
Relationship to research: Companion to current research. Implements the evaluation categories described in a longitudinal reporting manuscript shared with the account owner. The manuscript's own evaluator and exact field selection are not public; this is an independent implementation and reproduces none of its results.
Account owner’s role: Commissioned, reviewed and published the implementation. The code was produced with AI-assisted generation on commission and reviewed before release; it is not the original code of any cited publication.
What was built
- Tuple schema v1 with finding, region, side, polarity, uncertainty, optional severity and change
- Synonym canonicalisation and a 29-entry region vocabulary
- Configurable evaluated-field set; one-to-one matching; duplicate and contradiction flags
- Null conventions for zero denominators instead of silent zeros
- Report states: false content only, omission only, both, neither, with the union arithmetic shown
Method
Exact equality over the evaluated fields after canonicalisation; pooled and per-report aggregation both reported and named.
Data and access
Twenty-four authored synthetic reports across two fictional cohorts, including empty, duplicate and contradictory cases.
Evaluation protocol
The evaluator is the artefact; its own tests cover the edge cases listed above.
Results
No measured result is published for this project. Demonstration outputs from the repository’s own synthetic fixtures are labelled as such in the repository and are not presented here as results.
Limitations
- The exact field selection of any published evaluator is unknown, so numeric agreement with a paper cannot be claimed.
- Tuple extraction from free text is out of scope; inputs are already structured.
Reproducibility
Offline; metrics validate against a schema in the smoke path.
Repository metadata: Python, MIT, last push 2026-09-13, 0 stars.
Source publication and links
No publication is attached to this project.