Reporting quality of participant eligibility criteria in retrospective studies: A cross-sectional investigation of medical journals with high impact factors

Abstract

Background: The reporting quality of participant eligibility criteria in retrospective studies significantly affects research reproducibility and result interpretation. However, standardized guidelines for writing eligibility criteria in retrospective studies are lacking. We aim to systematically evaluate the quality of eligibility criteria reporting in retrospective studies published in high-impact factor medical journals, develop evidence-based recommendations for standardization, and provide supplementary guidance for relevant reporting guidelines. Methods: We conducted a cross-sectional analysis of retrospective studies published in the top 40 nonreview medical journals listed in the Journal Citation Reports (JCR) from January 2023 to September 2024. We extracted article characteristics (journal, author, objective, and study type) and eligibility criteria components. Two independent reviewers did the quality assessment of literature, which focused on clarity of retrospective nature (temporal framework), purposefulness (alignment with research objectives), and logical consistency between inclusion and exclusion criteria. Results: Among the top 40 nonreview medical journals in the 2023 JCR rankings, 11 journals contained 78 retrospective studies that were analyzed, of which 2.6% (2/78) demonstrated unclear retrospectivity and purposefulness in eligibility criteria. Logical contradiction between exclusion and inclusion criteria was found in 11.5% (9/78) of articles. Inter-rater reliability for quality assessment was substantial (κ = 0.857). Conclusion: The reporting quality of participant eligibility criteria in retrospective studies published in high-impact factor medical journals was flawed. On the basis of our systematic evaluation, we propose a structured framework for formulating eligibility criteria that emphasizes temporal precision, diagnostic clarity, and logical consistency between inclusion and exclusion criteria to supplement existing research reporting guidelines.