Background: Currently in the United Kingdom, cardiovascular disease (CVD) risk assessment is based on the QRISK3 score, in which 10% 10-year CVD risk indicates clinical intervention. However, this benchmark has limited efficacy in clinical practice and the need for a more simple, non-invasive risk stratification tool is necessary. Retinal photography is becoming increasingly acceptable as a non-invasive imaging tool for CVD. Previously, we developed a novel CVD risk stratification system based on retinal photographs predicting future CVD risk. This study aims to further validate our biomarker, Reti-CVD, (
Keywords: Artificial intelligence; Cardiovascular disease; Deep learning; Retinal imaging; Retinal photograph; Risk stratification; Risk stratification system; UK Biobank
Rachel Marjorie Wei Wen Tseng and Tyler Hyungtaek Rim contributed equally to this work.
In the United Kingdom (UK), around 1 in 9 people live with cardiovascular disease (CVD) [[
Despite these updated guidelines, their effect on clinical practice has been limited, with little change in statin prescription behavior [[
Retinal photography is a simple, effective, and non-invasive imaging tool that provides fast and accurate information on the human vasculature that may not be clearly visible to the human eye especially in recent years with the advent of deep learning (DL) [[
In this study, we aimed to optimize the operating threshold of RetiCAC, herein referred to as Reti-CVD, via simple retinal imaging. Using Reti-CVD, we aimed (
This retrospective study was deemed exempt from institutional review board (IRB) review by the SingHealth Centralised Institutional Review Board (CIRB). This study adhered to the tenets of the Declaration of Helsinki. Written informed consent was obtained from the participants of the original studies [[
We used clinical data and retinal photographs from the UK Biobank, a prospective population-based cohort in the UK [[
We excluded (
A total of 48,260 participants, representing the general population without a history of CVD, were included for analysis. In addition, we also defined three at-risk subgroups: (
Retinal photographs included in the study were taken using the Topcon 3D OCT-1000 Mark II (Topcon Corporation) between 7 December 2009 and 21 July 2010. Retinal cameras used in the training set include AFP-210 non-mydriatic auto retinal camera (NIDEK Corporation, Aichi, Japan), TRC-NW8 non-mydriatic retinal camera (Topcon Corporation, Tokyo, Japan), and Nonmyd A-D (Kowa Co. Ltd., Shizuoka, Japan). We did not include Topcon 3D OCT-1000 Mark II (Topcon Corporation) in our training set.
The other variables used in this study were defined as follows. Pre-diabetes and diabetes were defined based on (
In the UK Biobank, we used hospitalization and mortality data provided by the National Health Service (NHS) registers. The main outcome of interest in the current study reflected the outcome used in the QRISK3 risk score: fatal CVD events (ICD-10 I00-99, F01, Q20-Q28, C38.0, P29, G45) [[
For each individual, the QRISK3 score was calculated using R package, version 3.6 [[
Details of RetiCAC model development and previous validations have been described elsewhere [[
Analyses were done using p < 0.05 as the significance level, Stata/MP version 14.0 for survival analysis, and R version 3.4.4 for estimation of net reclassification index (NRI) using the R package survIDINRI [[
In the UK Biobank, hospitalization and mortality data were available up to May 05, 2021, at the time of analysis and each participant was followed up to 11.4 years from the date of baseline visit. In survival analysis, each patient was followed up to 11.4 years (median follow-up, 11.0 years) from the date of baseline visit to the last follow-up date or the date of the CVD events.
In all populations, the cumulative incidence of cardiovascular events rate was evaluated across the three groups (low, moderate, and high risk) defined by the Reti-CVD using Kaplan-Meier method. Cox proportional hazards model was used to estimate the hazard ratios (HRs) and trends in HRs and respective p-values were examined by fitting a linear model for the three categories. Unadjusted HR was provided according to three risk groups of Reti-CVD, and QRISK3-ajudted HR trend was provided.
To help future patients make an informed decision on statin and antihypertensive medication initiation, we only included the borderline-QRISK3 group who had QRISK3 score between 7.5 and 10% of 10-year CVD risk. In the borderline QRISK3 group, cumulative incidence of cardiovascular events rate was evaluated across the three groups (low, moderate, and high risk) according to the Reti-CVD and compared with participants with QRISK3 5–7.5% and 10–12.5%. The same analysis was repeated for middle-aged group (40 to 64 years).
The incremental prognostic value of the Reti-CVD over the QRISK3 in the prediction of CVD events was assessed using C-statistics and continuous net reclassification index (NRI) [[
Table 1 details the clinical characteristics of the participants by the three groups of Reti-CVD score (categorized as low, moderate, and high-risk groups).
Table 1 Characteristics of the study population
Reti-CVD score Characteristics Low risk Moderate risk High risk No. of participants 19,304 26,543 2413 Nonfatal and fatal CVD events (QRISK) 545 (2.8%) 1900 (7.2%) 321 (13.3%) QRISK3 score, mean (SD) 3.0 (2.5) 7.0 (4.1) 10.6 (4.6) QRISK3 score ≥ 0 to < 5 16,282 (84.3%) 9646 (36.3%) 208 (8.6%) ≥ 5 to < 10 2572 (13.3%) 11,393 (42.9%) 967 (40.1%) ≥ 10 to < 15 384 (2.0%) 4408 (16.6%) 876 (36.3%) ≥ 15 to < 20 55 (0.3%) 880 (3.3%) 272 (11.3%) > 20 11 (0.1%) 216 (0.8%) 90 (3.7%) Age, mean (SD) 50.8 (7.0) 59.8 (6.7) 64.4 (4.5) Gender Female, 12,702 (65.8%) 13,622 (51.3%) 741 (30.7%) Male, 6602 (34.2%) 12,921 (48.7%) 1672 (69.3%) Antihypertensive medication, 1637 (8.5%) 5642 (21.3%) 846 (35.1%) Stage 1 hypertension, 4199 (21.8%) 7178 (27.0%) 589 (24.4%) Pre-diabetes and diabetes, 269 (1.4%) 677 (2.6%) 86 (3.6%) Statin, 694 (3.6%) 1923 (7.2%) 170 (7.0%) Current smoker, 6451 (33.4%) 11,221 (42.3%) 1219 (50.5%)
Data are presented as n, n (% of participants), mean (standard deviation [SD]), or median (interquartile range [IQR]). CVD cardiovascular disease. Reti-CVD deep-learning-based retinal CVD biomarker
Among 48,260 included participants from the UK Biobank, the median QRISK3 10-year CVD risk was 4.5% (IQR 2.2–8.0%, SD, 4.2%); CVD events occurred in 2766 (5.7%) participants during the follow-up. The Spearman's rank correlation coefficient between Reti-CVD and QRISK3 score was 0.50 (p < 0.001).
CVD events rate were 2.8% (545/19,304) in the low-risk group, 7.2% (1900/26,543) in the moderate-risk group, and 13.3% (321/2413) in the high-risk group of Reti-CVD. Among the low-risk group of Reti-CVD, 84.3% had QRISK3 score of 0 to < 5%, and among the high-risk group of Reti-CVD, 8.6% had QRISK3 score of 0 to 5%, 40.1% had QRISK3 score of ≥ 5 to < 10%, and 36.3% had QRISK3 score of ≥ 10 to < 15%.
Kaplan-Meier curves were described in all participants according to five groups of QRISK3 and three groups of Reti-CVD (Fig. 1). During the follow-up (median 11.0 years; interquartile range [IQR], 10.9–11.1 years), 513,714.3 person-years were analyzed. As expected, QRISK3 stratified CVD risk well in general population of the UK Biobank (Fig. 1A). Reti-CVD also shows distinct CVD risk stratification based on the three groups within the general population of the UK Biobank (Fig. 1B). Based on Reti-CVD, the incidence of CVD per 1000 person-years was 2.6 (95% CI, 2.4–2.8) in the low-risk group, 6.8 (6.5–7.1) in the moderate-risk group, and 13.1 (11.7–14.6) in the high-risk group (Table 2), indicating a 13.1% 10-year CVD risk in Reti-CVD-high-risk group. Analysis for non-statin cohort and stage 1 hypertension cohort are provided in Additional file 4: eFigure 3 and Additional file 5: eTable 1.
Graph: Fig. 1 Kaplan-Meier curves according to Reti-CVD and QRISK3. Cardiovascular disease (CVD) event rate according to QRISK3-five groups (A), and Reti-CVD-three groups (B) in all participants (n = 48,260) were presented. In the general population, the high-risk group of Reti-CVD (B) shows a similar incident CVD rate to that of the QRISK 10–15% group (A), indicating that if a participant is identified as Reti-CVD-high risk by retinal photography, participants may be advised to undergo a full CVD risk assessment via the NHS primary care service
Table 2 Risk of cardiovascular events by the deep-learning-based retinal CVD biomarker (Reti-CVD) in all participants
Risk predictor Cases Person-years Incidence (95% CI) Unadjusted hazard ratio (95% CI) Reti-CVD Low 19,304 545 209,186 2.6 (2.4–2.8) 1 (reference) Moderate 26,543 1900 280,021 6.8 (6.5–7.1) 2.62 (2.38–2.88) High 2413 321 24,508 13.1 (11.7–14.6) 5.11 (4.45–5.86) Adjusted HR trenda 1.41 (1.30–1.52) QRISK3 score ≥ 0 to < 5 26,136 702 283,604 2.5 (2.3–2.7) 1 (reference) ≥ 5 to < 10 14,932 1099 157,442 7.0 (6.6–7.4) 2.84 (2.58–3.12) ≥ 10 to < 15 5668 695 57,723 12.0 (11.2–13.0) 4.93 (4.44–5.48) ≥ 15 to < 20 1207 196 11,988 16.3 (14.2–18.8) 6.75 (5.76–7.91) ≥ 20 317 74 2957 25.0 (19.9–31.4) 10.50 (8.27–13.35) Total 48,260 2766 513,714 5.4 (5.2–5.6)
Additional file 6: eTable 2 describes the performance of Reti-CVD in predicting the incidence of CVD events when applied to at-risk subgroup populations. Incidence of CVD and QRISK3-adjusted HR trends were provided in 3 subgroups: individuals with body-mass index (BMI) above 25 kg/m
The Kaplan-Meier analysis of CVD events in the UK Biobank according to the three strata of Reti-CVD among borderline group individuals (QRISK3 score between 7.5 and 10%) is provided in Fig. 2. Although QRISK3 was less than 10% in the borderline group, the cumulative CVD events rate in Reti-CVD-high-risk group was consistently higher than 10% of 10-year CVD risk.
Graph: Fig. 2 Kaplan-Meier curves according to Reti-CVD in borderline-QRISK3 group. Cardiovascular disease (CVD) event rate according to Reti-CVD-three groups in borderline-QRISK3 group who had QRISK3 score between 7.5 and 10% were presented in non-statin cohort (A), stage 1 hypertension cohort (B), and middle-aged (40–64 years) cohorts (C). Considering that statin and antihypertensive pharmacotherapy initiation is recommended at QRISK of ≥ 10%, Reti-CVD can be used as a risk enhancer in borderline-QRISK3 groups of 7.5–10% to reach consensus on statin initiation. In addition, although most of risk assessment systems were derived from cohorts of primarily middle-aged people and typically well-functioning individuals, Reti-CVD can still be a risk enhancer in in borderline-QRISK3 groups of 7.5–10% middle-aged people
Specifically, in this borderline-QRISK3 group, the 10-year CVD risk was higher than 10% in Reti-CVD-high-risk group with 11.5% (8.9–15.0%) in non-statin cohort (n = 45,473), 11.5% (7.4–17.8%) in stage 1 hypertension cohort (n = 11,966), and 14.2% (10.3–19.5%) in middle-age cohort (n = 38,941) (Table 3).
Table 3 Risk of cardiovascular events by the deep-learning-based retinal CVD biomarker (Reti-CVD) in participants who had 7.5–10% QRISK3 score
QRISK3 Reti-CVDa Cases Person-years Incidence rate at year 10 (95% CI) ≥ 5 to < 7.5 7842 478 83,295 5.6 (5.1–6.1) ≥ 7.5 to < 10 Low 652 47 6868 7.2 (5.4–9.6) ≥ 7.5 to < 10 Moderate 4526 401 47,286 7.9 (7.1–8.8) ≥ 7.5 to < 10 High 518 62 5318 11.5 (8.9–15.0) ≥ 10 to < 12.5 3353 394 34,314 11.0 (9.9–12.2) Total 16,891 1382 177,081 7.8 (7.4–8.2) ≥ 5 to < 7.5 2622 154 27,891 5.5 (4.6–6.5) ≥ 7.5 to < 10 Low 230 16 2427 6.8 (4.1–11.3) ≥ 7.5 to < 10 Moderate 1525 143 15,821 8.3 (6.9–9.9) ≥ 7.5 to < 10 High 186 22 1903 11.5 (7.4–17.8) ≥ 10 to <1 2.5 974 115 9990 10.9 (9.0–13.3) Total 5537 450 58,032 7.8 (7.1–8.5) ≥ 5 to < 7.5 7097 442 75,354 5.7 (5.2–6.3) ≥ 7.5 to < 10 Low 497 30 5291 5.8 (4.0–8.4) ≥ 7.5 to < 10 Moderate 2856 272 29,813 8.5 (7.5–9.7) ≥ 7.5 to < 10 High 287 43 2914 14.2 (10.3–19.5) ≥ 10 to < 12.5 1327 153 13,658 11.2 (9.5–13.2) Total 12,064 940 127,030 7.4 (6.9–7.9)
We then tested the incremental value of Reti-CVD when applied alongside QRISK3 in predicting incident CVD events among non-statin, stage 1 hypertension, and middle-aged cohorts (Table 4). By adding Reti-CVD score to QRISK3, C statistics increased by 0.014 (95% CI, 0.010–0.017) in non-statin cohort, 0.013 (0.007–0.019) in stage 1 hypertension cohort, and 0.023 (0.018–0.029) in middle-aged cohort for prediction of CVD events. Also, by adding Reti-CVD score to QRISK3, the continuous NRI was 0.133 (95% CI, 0.088–0.173) in non-statin cohort, 0.094 (0.008–0.174) in stage 1 hypertension cohort, and 0.248 (0.190–0.301) in middle-aged cohort.
Table 4 Prognostic performance with the addition of Reti-CVD to the QRISK3 in the UK biobank dataset
Non-statin cohort ( Stage 1 hypertension cohort ( Middle-aged Cohort ( Models Reti-CVD 0.624 (0.615–0.633) NA 0.591 (0.575–0.608) NA 0.611 (0.600 -0.622) NA Reti-CVD plus age, gender 0.699 (0.690–0.709) NA 0.663 (0.644–0.681) NA 0.681 (0.669–0.692) NA QRISK3 0.682 (0.672–0.692) NA 0.639 (0.620–0.658) NA 0.650 (0.638–0.662) NA Reti-CVD plus QRISK3 0.696 (0.686–0.706) NA 0.652 (0.633–0.671) NA 0.674 (0.661–0.686) NA Δ Reti-CVD plus QRISK3 versus QRISK3 0.014 (0.010–0.017) < 0.001 0.013 (0.007–0.019) < 0.001 0.023 (0.018–0.029) < 0.001 NRI Continuous NRI (95% CI) 0.133 (0.088–0.173) < 0.001 0.094 (0.008–0.174) 0.033 0.248 (0.190–0.301) < 0.001
Data are C statistic (95% CI), unless stated otherwise. The Reti-CVD plus QRISK3 model is a logistic model fit on the UK Biobank. NRI for Reti-CVD plus QRISK3 versus QRISK3 models was provided NRI net reclassification index, CI confidence interval, CVD cardiovascular disease, Reti-CVD deep-learning-based retinal CVD biomarker
Additional file 7: eFigure 4 summarizes net-benefit comparisons using a decision-analysis curve. In entire UK Biobank participants, Additional file 7: eFigure 4A presents the decision-analysis curves for QRISK3 score and Reti-CVD plus age, and gender model, indicating that the two models are almost similar in terms of net-benefit. In the hypertensive patients and the pre-diabetes and diabetes subgroup (Additional file 7: eFigure 4B and 4C), the Reti-CVD plus age, and gender model demonstrated a higher net-benefit than QRISK3 score in general.
In this study, we optimized the operating threshold of RetiCAC, which was originally based on the Korean cohort. This optimized biomarker, Reti-CVD, stratified the general UK population into three risk groups and presented with significant hazard ratio (HR) trends. Specifically, the high-risk group in Reti-CVD adequately identified individuals who had a 10-year CVD risk greater than 10%, with a CVD incidence of 13.1 per 1000 person-years. Moreover, we quantified the cumulative CVD events and prognostic value for risk stratification when adding Reti-CVD alongside QRISK3 in the borderline-QRISK3 group (10-year CVD risk of 7.5–10%). Due to the new cut-off threshold of 10% for statin and antihypertensive treatment initiation, we focused on individuals who had a QRISK3 score between 7.5 and 10% and showed that Reti-CVD could further stratify the CVD risk into low, moderate, and high-risk groups. Specifically, in the borderline-risk group, individuals who were stratified as high-risk using Reti-CVD had a 10-year CVD risk greater than 10% and was comparable to the CVD incidence of QRISK3 (10 to 12.5%) group, indicating that Reti-CVD could be used as a risk enhancer.
Like other areas of medicine, there is now a paradigm shift in cardiovascular medicine, towards using digital innovations especially those related to artificial intelligence to enhance diagnosis and risk stratification [[
Previous studies using retinal imaging as a tool for predicting CVD risk have focused largely on other risk factors such as age, hypertension, and dyslipidemia [[
In the general population, we have shown that Reti-CVD can identify individuals with a CVD risk greater than 10% in our high-risk Reti-CVD group. This finding is useful for two reasons because a CVD risk of 10% is the latest threshold for statin and anti-hypertensive medication initiation. First, at the primary care level, Reti-CVD provides GPs with a simple and effective way of providing an initial assessment on asymptomatic patients. Ophthalmologists, optometrists, and opticians can also use existing retinal photographs for opportunistic eye screening. Second, Reti-CVD can aid the CVD prevention program that NHS England has developed. The NHS CVD primary prevention focusses on three main components: atrial fibrillation, blood pressure, and cholesterol [[
An additional finding that we want to highlight is the fact that Reti-CVD identified high-risk individuals who were otherwise missed by QRISK3. 8.6% of patients identified as high-risk by our new stratification tool had a QRISK3 score between 0 to 5 and 40.1% of them were categorized in the QRISK3 ≥ 5 to < 10% group. This demonstrates the potential of Reti-CVD as an adjunct tool to identify future high-risk individuals who may be left out if CVD risk stratification is dependent solely on QRISK3 scoring. Consequently, when we incorporate only two basic demographic factors of age and gender to Reti-CVD, this model was superior to QRISK3 model across the range of risk thresholds. Moreover, in our explorative analysis in eTable 3, we confirmed that upon conducting a more detailed QRISK3 analysis in the borderline QRISK3 group where individuals were divided based on 0.5 intervals between 7.5 and 10%, more than 1.6% of individuals were classified in the Reti-CVD-high-risk groups in all subgroups. This indicates that Reti-CVD can be used as an independent risk enhancer in this range of QRISK3.
By comparing Reti-CVD to QRISK3, this study demonstrates the potential value and scope of Reti-CVD to improve discernment accuracy in the UK population since QRISK3 is part of the up-to-date guidelines that reflects the demographic profile of the UK Biobank. Study limitations include a possibility of systematic errors in the form of misclassification bias, since hospitalization and mortality data provided by the NHS registers were used for CVD definition. However, a study by Kivimaki et al. has analyzed the validity of and reported good agreement between CVD event predictions and the use of UK National Health Electronic Records, therefore minimizing the likelihood of CVD event misdiagnoses [[
In conclusion, a DL and retinal photograph-derived new CVD biomarker, Reti-CVD, could effectively stratify CVD risk in the general population. We also confirmed that Reti-CVD has the potential to identify individuals with ≥ 10% 10-year CVD risk who are likely to benefit from preventative CVD interventions. In addition, for people with a QRISK3 score between 7.5 and 10%, Reti-CVD could be used as a risk stratification enhancer for identifying high risk patients for aggressive CVD prevention.
Not applicable.
T.H.R. conceptualized the study. G.L. developed the algorithm. R.M.W.W.T. and T.H.R. wrote the manuscript and researched its contents and edited all versions. All authors substantially revised the manuscript, read, and approved the final manuscript.
This project is supported by the Agency for Science, Technology and Research (A*STAR) under its RIE2020 Health and Biomedical Sciences (HBMS) Industry Alignment Fund Pre-Positioning (IAF-PP) Grant No. H20c6a0031. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of the A*STAR.
The UK Biobank test dataset was obtained from UK Biobank (application number 68428). Data cannot be shared publicly due to the violation of patient privacy and the absence of informed consent for data sharing.
The UK Biobank dataset was obtained ethically from UK Biobank (application number 68428). The UK Biobank has also received Research Tissue Bank approval from North West Multi-centre Research Ethics Committee and the Human Tissue Authority license [[
Not applicable.
T.H.R. was a former scientific adviser and owns stock of Mediwhale Inc. H.K. and G.L. are employees of Mediwhale Inc. T.H.R. and G.L. hold the following patents that might be affected by this study: 10–2018–0166720(KR), 10–2018–0166721(KR), 10–2018–0166722(KR), 62/694,901(US), 62/715,729(US), and 62/776,345 (US). All other authors declare no competing interests.
Graph: Additional file 1: eFigure 1. Study flow chart.
Graph: Additional file 2: eFigure 2. Distribution of the QRISK3 score in the UK Biobank.
Graph: Additional file 3: eDocument 1. RetiCAC model updates.
Graph: Additional file 4: eFigure 3. Kaplan-Meier curves according to Reti-CVD and QRISK3.
Graph: Additional file 5: eTable 1. Risk of cardiovascular events by the deep-learning-based retinal CVD biomarker (Reti-CVD).
Graph: Additional file 6: eTable 2. Subgroup analysis in the UK Biobank.
Graph: Additional file 7: eFigure 4. Decision curve analysis.
Graph: Additional file 8: eTable 3. Detailed distribution of Reti-CVD risk groups among those between 7.5% and 10% QRISK3 score.
• CVD
- Cardiovascular disease
• UK
- United Kingdom
• NICE
- National Institute for Health and Care Excellence
• DL
- Deep-learning
• CAC
- Coronary artery calcium
• IRB
- Institutional Review Board
• CIRB
- SingHealth Centralized Institutional Review Board
• NHS
- National Health Service
• NRI
- Net Reclassification Index
• BMI
- Body-mass index
• HR
- Hazard ratio
• FRS
- Framingham risk score
• GP
- General practitioner
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
By Rachel Marjorie Wei Wen Tseng; Tyler Hyungtaek Rim; Eduard Shantsila; Joseph K. Yi; Sungha Park; Sung Soo Kim; Chan Joo Lee; Sahil Thakur; Simon Nusinovici; Qingsheng Peng; Hyeonmin Kim; Geunyoung Lee; Marco Yu; Yih-Chung Tham; Ameet Bakhai; Paul Leeson; Gregory Y.H. Lip; Tien Yin Wong and Ching-Yu Cheng
Reported by Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author