VADRR uses machine learning to prioritize your screening queue: instead of screening citations in
a random or import order, it learns from the decisions you've already made and shows you the
most likely-relevant citations first. This is a form of active learning — the more you screen,
the smarter the ordering gets.
You don't have to do anything special to benefit from this — just screen. The ordering of
Screen Next Task is driven by the model automatically.
- Every Accept (label
1) and Reject (label 0) you record during abstract screening becomes
a training example, using the citation's title and abstract text.
- Privileged decisions — conflict resolutions and quick Accept/Reject qualifications — take
precedence over individual labels when there's disagreement.
- Training needs both classes present: at least one accepted and one rejected citation.
- A scheduled job runs once a day and (re)trains the model when at least 100 new labels have
been recorded since the last training run. (As a fallback, if there's been any new label and
it's been 7+ days since the last training, it retrains then too.)
- Need it sooner? A Leader can flip Force Training on the Abstract Screenings page to train on
fewer than 100 labels. It reverts to off once training completes.
- Each trained model is stamped with a time — shown as Latest Model Time on the results page — so
you always know how fresh the predictions are.
- Relevance scores. Every unscreened citation receives a score from 0 to 1 (higher = more
likely relevant). These scores re-order your queue so high-scoring citations come up first.
- Performance estimates. VADRR also trains on a portion of your labeled data and tests on the
held-out remainder. Comparing those predictions to the real labels yields the
confusion matrix and the precision/recall/F1/accuracy figures on the results page.
- Find the relevant studies sooner. Most reviews have far more irrelevant than relevant
citations; prioritization surfaces the relevant ones early.
- Know when you're near done. As you keep screening and stop finding new relevant studies, signals
like the rejection streak and the estimated coverage curve suggest you're approaching the
end — helping you decide when it's reasonable to stop.
Everything the model knows is summarized on the
Machine Learning Results page — open it from
Abstract Screenings ▸ Machine Learning Results (Leaders).
Note: some visualizations need enough data to be meaningful — the Estimated Coverage graph
only draws once you have 1,000+ labels, and the confusion matrix/score charts need a trained
model.