Methodological Precedence in Health Tech: Why ML/Big Data Analysis Must Follow Basic Epidemiological Consistency. A Case Study

The integration of advanced analytical tools, including Machine Learning (ML) and massive data processing, has revolutionized health research, promising unprecedented accuracy in diagnosis and risk prediction. However, the rigor of these complex methods is fundamentally dependent on the quality and integrity of the underlying datasets and the validity of their statistical design. We propose an emblematic case where advanced analysis (ML/Big Data) must necessarily be subsequent to the verification of basic methodological coherence and adherence to established medical protocols, such as the STROBE Statement. This study highlights a crucial cautionary principle: sophisticated analyses amplify, rather than correct, severe methodological flaws rooted in basic design choices, leading to misleading or contradictory findings. By applying simple, standard descriptive statistical methods and established national epidemiological benchmarks to a recently published cohort study on COVID-19 vaccine outcomes and severe adverse events, like cancer, we expose multiple, statistically irreconcilable paradoxes. These paradoxes, specifically the contradictory finding of an increased cancer incidence within an exposure subgroup, concurrent with a suppressed overall Crude Incidence Rate compared to national standards, definitively invalidate the reported risk of increased cancer in the total population. We demonstrate that the observed effects are mathematical artifacts stemming from an uncorrected selection bias in the cohort construction. This analysis serves as a robust reminder that even the most complex health studies must first pass the test of basic epidemiological consistency before any conclusion drawn from subsequent advanced statistical modeling can be considered valid or publishable.

💡 Research Summary

The article “Methodological Precedence in Health Tech: Why ML/Big Data Analysis Must Follow Basic Epidemiological Consistency. A Case Study” argues that sophisticated analytical tools such as machine learning (ML) and big‑data processing cannot compensate for fundamental flaws in study design, data quality, or epidemiological validity. The authors illustrate this principle by re‑examining a recently published South Korean cohort study that claimed an increased one‑year cancer risk among COVID‑19 vaccine recipients.

First, the authors extract raw numbers from the supplementary Table S4 of the target study: a final matched cohort of 2,975,035 individuals with 12,133 incident cancer cases. Using the standard crude incidence rate (CR) formula (cases ÷ population × 10,000), they calculate an overall CR of 40.78 per 10,000. National cancer statistics from the Korean Central Cancer Registry for 2020‑2022 report an average CR of 52.46 ± 2.97 per 10,000. The study cohort’s CR is therefore markedly lower than the national benchmark, indicating that the cohort does not reflect the true disease burden of the population.

Second, the authors assess age structure. The national proportion of people aged ≥65 years is 18 %, yet the matched cohort contains only 12.15 % elderly participants. A chi‑squared goodness‑of‑fit test (df = 1) yields a highly significant p‑value, confirming a systematic under‑representation of the high‑risk age group. This distortion is attributed to the 1:4 propensity‑score matching (PSM) procedure used in the original study, which balanced covariates between vaccinated and unvaccinated groups but ignored the overall representativeness of the combined sample.

Third, the authors compute group‑specific CRs: 42.63 per 10,000 for the vaccinated subgroup (10,144 cases) and 33.43 per 10,000 for the unvaccinated subgroup (1,989 cases). Although the vaccinated group shows a higher incidence, the overall cohort CR remains below the national average—a paradox that arises from the selection bias introduced during matching. The authors argue that this paradox invalidates the original claim of a vaccine‑associated cancer increase because the underlying denominator is not comparable to the general population.

The paper further critiques the original study’s adherence to the STROBE reporting guidelines. Item 21 (generalizability) is violated because the authors fail to discuss the discrepancy between cohort CR and national rates, and Item 12 (bias) is ignored by not reporting efforts to correct the structural bias created by PSM. Consequently, the study’s findings lack external validity and cannot be reliably generalized to the broader South Korean population.

In the discussion, the authors stress that advanced methods such as PSM, ML, or other high‑dimensional models are valuable only when applied to data that have passed basic epidemiological checks. They propose a three‑step pre‑analysis checklist: (1) compare raw incidence rates with national benchmarks, (2) verify that demographic distributions (age, sex) match population standards, and (3) ensure compliance with established reporting standards (e.g., STROBE). Only after these steps should researchers proceed to sophisticated modeling, as otherwise the models may amplify, rather than mitigate, underlying biases.

The conclusion reiterates that methodological rigor at the descriptive‑statistics level is a prerequisite for credible causal inference in health‑tech research. By exposing the paradox in the Korean vaccine‑cancer study, the authors demonstrate that neglecting basic epidemiological consistency can lead to misleading public‑health messages, regardless of how advanced the subsequent analytical techniques are.