QUADAS-AI
QUADAS-AI is a tool used to assess the quality and reliability of studies that evaluate how well artificial intelligence systems can diagnose diseases. It checks for potential biases and whether the study results can be applied broadly, building on the established QUADAS-2 tool but tailored for AI-based diagnostic tests.
At a glance
Use when
Conducting systematic reviews or health technology assessments of AI-based diagnostic tests where risk of bias and applicability need to be evaluated rigorously.
Avoid when
Assessing non-diagnostic AI interventions (e.g., therapeutic AI, predictive risk models) or when insufficient study details are available to answer signaling questions.
Inputs
Diagnostic test accuracy study involving an AI-based diagnostic tool
Outputs
Risk-of-bias and applicability judgments across key domains (e.g., patient selection, index test, reference standard, flow and timing, AI development and validation)
How it works
QUADAS-AI is a structured quality assessment tool designed specifically for diagnostic test accuracy studies involving artificial intelligence. It extends QUADAS-2 by incorporating AI-specific domains and signaling questions to evaluate risk of bias and applicability concerns related to the development, validation, and deployment of AI algorithms in diagnostic settings.
- HTA domains
- Clinical Effectiveness, Safety, Organisational aspects
- Assumptions
- The tool assumes that transparency in AI model development and validation is critical for trustworthy diagnostic evaluation and that standard quality assessment tools like QUADAS-2 require augmentation to address AI-specific issues.
- Strengths
- Tailored to address AI-specific methodological challenges such as overfitting, data leakage, and model generalizability; builds on the widely accepted QUADAS-2 framework; enhances rigor in systematic reviews of AI diagnostic studies.
- Limitations
- Requires reviewers with both clinical and technical AI expertise; may be complex to apply in studies with poorly reported methods; limited empirical validation in diverse AI applications.
- Also known as
- Quality Assessment of Diagnostic Accuracy Studies - AI
Questions this answers
- › Does the study use an appropriate reference standard for diagnosing the condition?
- › Is the AI model's development process clearly described and justified?
- › Was the AI model validated on independent, external datasets?
- › Are there concerns about patient selection introducing bias?
- › Could the way the AI was trained affect its real-world applicability?
- › Are there risks of overfitting or data leakage in the model development?
References & sources
Similar by meaning
Beta record. Based on the original catalogue summary; primary-source enrichment pending.