Agreement Between REGICOR and SCORE Scales in Identifying High Cardiovascular Risk in the Spanish Population

INTRODUCTION

Stratifying cardiovascular risk using risk charts is central to decision-making on treatment to prevent cardiovascular disease. The Framingham charts, which are used to calculate coronary morbility and mortality,¹ and the SCORE risk charts, which are used to calculate cardiovascular mortality,² both at 10 years, are the most widely used in clinical practice. Several analyses have nonetheless concluded that the Framingham-Wilson equation overestimates coronary risk in southern European countries, where there is a lower incidence of acute myocardial infarction.^3-8 Various scientific bodies have therefore proposed carrying out local population cohort studies to obtain risk charts which are applicable to given areas or which allow existing charts to be adapted for use in those areas.^4,9,10

In Spain, the Framingham-Wilson score¹ was adapted for use in the Girona region using a validated methodology⁹ and led to the production of the REGICOR scale.^11,12

Cross-sectional studies were also performed to obtain the DORICA (Dislipidemia, Obesity, and Cardiovascular Risk) charts.¹³ On the other hand, following publication of the results of the SCORE project, in which 3 Spanish cohorts were included,² several societies (Third Report of the European Atherosclerosis Societies,¹⁴ Spanish Interdisciplinary Committee for Cardiovascular Prevention [CEIPC],¹⁵ and the Health Promotion and Preventive Activities Program of the Spanish Society for Family and Community Medicine [PAPPS]¹⁶) recommend using the low-risk country SCORE chart to calculate cardiovascular risk.

Discrepancies between the REGICOR and SCORE charts have been the subject of debate.¹⁷ A 10-year retrospective cohort study in a Spanish urban population concluded that the SCORE chart had greater validity than REGICOR for assessing the risk of coronary episodes and cardiovascular mortality, although neither scale accurately reflected the reality of the situation as it was suggested that REGICOR underestimated, and SCORE overestimated the degree of risk.¹⁸ Based on the results of that study, it was proposed that the high risk cut-point on REGICOR should be reduced from 20% to 10%. Using the 20% threshold, REGICOR classifies very few patients as high risk. If the 10% cut-point is used, the number of high risk cases is similar to that produced using the SCORE chart. Agreement with the Framingham-Wilson equation also improves.^19,20

Agreement between SCORE (using the CEIPC recommendations) and REGICOR has not been analyzed in populations which are free of cardiovascular disease and there has been no analysis of the sub-groups of patients in which discrepancies are observed between the 2 charts. Likewise, the implications for treatment of discrepancies between the 2 charts have not been studied. The objectives of the present study were to assess the agreement between the 2 charts and to determine the clinical profiles of subjects in whom the 2 charts differed with regard to classifying them as high risk cases.

METHODS

Sample Selection

Subjects were selected from a preventive activities program run by the Ministry of Health of the Valencian Community in 2003 in collaboration with the Valencian Society for Community and Family Medicine (SVMFiC), the Health Care Research Network for Preventive Activities in the Valencia Region (REDIAPP-cardiovascular) and the Community and Family Medicine Unit in the Department of Clinical Medicine at the Miguel Hernández de Elche University (Alicante). As recommended by PAPPS, the program was aimed at the adult population aged 40 years or over.¹⁵ Individuals were invited to participate by letter and those wishing to be involved were given an appointment at their local health center for an evaluation by medical and nursing staff. The results were entered in a data-base and the final assessment was carried out by the Research, Teaching, and Clinical Practice Unit in Department 18 of the Valencian Community and the Department of Family Medicine at the Miguel Hernández University.

For the present study, subjects who had been clinically examined within the first 6 months of the project were included consecutively if: they had the data necessary to calculate a SCORE (using total cholesterol) and REGICOR classification; if they had no history of cardiovascular disease, and; if they were within the age range in which the 2 charts can be applied. Therefore, for this analysis, individuals were included who: a) were aged 40-65 years; b) had no established history of cardiovascular disease; and c) had a complete lipid profile performed according to CEIPC recommendations. Subjects with cardiovascular disease, those outside the required age range, or without a complete lipid profile or any of the other variables needed to calculate risk were excluded from the analysis.^2,11

Variables Analyzed

To calculate coronary risk with REGICOR, the following variables were collected: age, sex, total cholesterol, high density lipoprotein cholesterol (HDL-C), systolic blood pressure (SBP), diastolic blood pressure (DBP), smoking, and history of diabetes mellitus (DM).¹¹ Variables used to calculate risk of cardiovascular death using SCORE (based on total cholesterol figures and using the version for low-risk countries) were age, sex, total cholesterol, SBP, and smoking history.

Blood pressure was measured using mercury sphygmomanometers as recommended in several guidelines.^21,22 Blood tests were performed in venous blood, after at least 8 h fasting, in reference laboratories in the Valencian community. Lipid profiles were evaluated in accordance with the recommendations of the Adult Treatment Panel III (ATP-III).²³

The low-risk SCORE chart and REGICOR were compared using a 10% high risk cut-point for REGICOR based on the good agreement with the Framingham-Wilson equation using this threshold.^19,20 A threshold of 5% was used to define high risk on the SCORE chart.² Based on the CEIPC criteria,¹⁵ subjects with diabetes or with blood pressure ≥180/110 mm Hg, total cholesterol ≥320 mg/dL, or low density lipoprotein cholesterol (LDL-C) ≥240 mg/dL were also considered high risk using the SCORE chart.

Statistical Analysis

Agreement between the 2 charts was assessed using the kappa statistic. Discrepancies were assessed using McNemar's χ² test for paired data, with 1 degree of freedom. The area under the ROC curve was used to assess the diagnostic precision for SCORE using different high risk cut-points on REGICOR. Results were interpreted using Swet's criteria.²⁴

Discrepancies between the 2 scales were analyzed using the Pearson χ² test for categorical variables and the Student t test for quantitative variables, based on estimates of normality and equality of variances. A multivariate analysis using stepwise binary logistic regression was applied in the population in which discrepancies occurred. The dependent variable consisted of the discrepancies between SCORE and REGICOR and independent variables were those used in the bivariate analysis. Prior history of DM was eliminated from the model as a variable because, according to CEIPC,¹⁵ no individual patients with diabetes were classified as high risk by REGICOR but not be SCORE, so the model would not be able to identify agreement due to the lack of a reference category. Baseline glucose values were included instead. A P value less than .05 was considered statistically significant and 95% confidence intervals (CI) were calculated.

RESULTS

Of the 33 440 subjects included, 8942 (26.7%) met inclusion criteria (mean age [SE] was 51.3 [7.3] years; 59.9% were men; 27.7% were smokers; 14.4% had a history of high blood pressure [HBP]; 3.6% were diabetic, and 11.5% had a diagnosis of dyslipidemia). Mean blood pressure was 127.3 (17.1)/78.2 (10.9) mm Hg. Mean SCORE value was 1.54% (95% CI, 1.50-1.57), and the mean REGICOR value was 4.35% (4.28-4.43). The atherogenic index was 4.02 (3.99-4.04). Table 1 shows sample characteristics by gender.

Using a threshold of 20%, 12.5% of the sample was classified as high risk using the SCORE chart compared to 0.5% using REGICOR. When a cut-point of 10% was used, the proportion of high risk subjects on the SCORE chart fell to 6.7%. The proportion of high risk subjects on the REGICOR chart is shown in Figure 1 for different cut-points. Kappa indices between the REGICOR and the low-risk country SCORE chart are also shown. The kappa index increased as the cut-point for high risk decreased on the REGICOR chart. The highest agreement was observed for a threshold of 8%. At this cut-point, the number of high risk cases identified by the 2 charts was similar. Lowering the cut-point to 7% did not increase agreement between the 2 charts.

Figure 1. Percentage of subjects classified as high risk by SCORE and REGICOR using different cut-points; agreement with SCORE. * indicates kappa index.

Table 2 shows the distribution of the population into high risk and non-high risk groups using thresholds of 10% on REGICOR and 5% on SCORE. There was disagreement between the 2 charts for 10.2% (9.6%-10.8%) of cases (high risk on one chart but not on the other; κ=0.42 [0.39-0.45]). Of those, 8% were classified as high risk by SCORE but not by REGICOR, and 2.2% were classified as high risk by REGICOR but not by SCORE (McNemar, P<.001).

Table 3 shows the characteristics of the 2 groups in which there were discrepancies between the 2 instruments. Individuals classified as high risk on SCORE but not on REGICOR included a high percentage of cases with diabetes and high blood pressure, and had higher blood pressure than the group with classified as high risk on REGICOR but not on SCORE. In the latter group, there were higher percentages of men and smokers, subjects had lower HDL-C and higher LDL-C, and blood pressure was normal, high-normal, or stage 1 hypertension.

To investigate which variables were significantly associated with the discrepancy between the 2 scales, a multivariate model was constructed for discrepant subjects which included age, gender, smoking, baseline glucose, total cholesterol, HDL-C, and LDL-C, SBP, and DBP as independent variables. Variables independently associated with discrepancies are shown in Table 4. Discrepancies in which individuals were classified as high risk on SCORE but not on REGICOR were associated with lower age, being female, and having higher levels of DBP and baseline glucose. In the reverse situation (high risk on the REGICOR but not on SCORE), statistically significant variables were being male, smoking, and having low HDL-C. The multivariate model was statistically significant (P<.001) and explained 93.2% of the variability in discrepancies. Smoking and gender were found to have the greatest weight in the model in terms of explaining discrepancies (Table 4).

Figure 2 shows discriminatory power calculated using the ROC curve with a SCORE threshold of 5% and different high risk cut-points for REGICOR. The area under the curve was 0.8 (0.785-0.814). The optimal cut-point for REGICOR was 4% (sensitivity, 86.7%; specificity, 63.9%). At this threshold REGICOR, the kappa index was 0.250 (0.234-0.266); high risk subjects, 42.4% (41.4%-43.4%), and discrepancies with SCORE (using a 5% threshold) occurred in 33.2% of cases (31.2%-34.2%).

Figure 2. ROC curve for REGICOR chart when SCORE threshold is 5%.

DISCUSSION

Overall Interpretation of Results

In this population study in individuals aged 40-65 years with no history of cardiovascular disease, only moderate agreement in identifying cardiovascular risk was observed between the SCORE used according to CEIPC criteria (for low-risk countries and using a 5% cut-point) and REGICOR (using a cut-point of 10%). Factors associated with individuals being classified as high risk using SCORE but not using REGICOR were younger age, being female, and having a higher baseline DBP and glucose values; being classified as high risk by REGICOR but not by SCORE was associated with being male, smoking, and having low HDL-C. Compared to SCORE, REGICOR may underestimate risk in patients with diabetes or high blood pressure, particularly in younger women (young adults). On the other hand, in comparison to REGICOR, SCORE is likely to underestimate risk in non-diabetic males, smokers, or those with low HDL-C. This latter profile, which would be high risk on REGICOR but not on SCORE, could be considered a modifier profile indicating increased risk.

Limitations of the SCORE and REGICOR Charts

Using charts to estimate cardiovascular risk has its limitations. Comparative studies of the SCORE and Framingham charts in the Spanish population revealed discrepancies in the detection of high risk cases^25,26 and in the recommendations for treatment (SCORE favored intervention in hypertensive women and Framingham in men with raised cholesterol²⁷). Additionally, SCORE only estimates the risk of cardiovascular mortality and not morbidity,² and the original algorithm only included the population aged between 40 and 65 years. Risk assessment in patients with diabetes is another weakness of the SCORE chart, as they were not included in the tables.² Whereas the CEIPC classifies all patients with diabetes as high risk,¹⁵ the authors of the SCORE chart recommend multiplying risk by 4 in women and by 2 in men.²

With regard to REGICOR, using the recommended cut-point of 20% meant that virtually no individuals were identified as being high risk (in the present study it did not reach 1%). Other studies^18-20 have indicated that reducing the cut-point to 10% would identify a similar proportion of high risk patients to those identified by SCORE and would lead to greater agreement with the Framingham chart, although the VERIFICA (Validation of the Adapted Framingham Individual Coronary Risk Equation) study showed that REGICOR had greater validity than the original Framingham equation in the population studied.²⁵

Agreement Between SCORE and REGICOR in the Identification of High Risk Cases

The comparison between the 2 risk charts in the Spanish population was performed in selected groups of patients, but not in the adult general population. In a study carried out in individuals attending a health center, SCORE and Framingham would indicate the use of cholesterol-lowering treatment in a greater number of patients than REGICOR using a 10% or 20% cut-point.^18,28,29 REGICOR indicated a lower risk than SCORE in studies in individuals with HBP or other cardiovascular risk factors,^30,31 and there were considerable discrepancies between the 2 methods.

In the present study, a population aged 40-65 years with no history of cardiovascular disease was selected to compare the low-risk country SCORE chart with REGICOR using a high risk cut-point of 10%. We chose this threshold as it has been used previously in Spain and because it provided better agreement with the Framingham-Wilson equation.^19,20 As in other studies,^18,19,28 using a cut-point of 20% with REGICOR meant that almost no individuals were identified as high risk and agreement with SCORE was very low (κ=0.066). When the threshold was set at 10%, the percentage of subjects classified as high risk rose to 6.7% and the level of agreement improved (κ=0.420). Even then, however, SCORE still identified twice as many individuals as high risk compared to REGICOR.

In this regard, the discriminatory power of the ROC curve was moderate. The optimal threshold for considering a case to be high risk using REGICOR (4%) does not appear acceptable for use in clinical practice given the substantial discrepancies with SCORE. We identified a cut-point of 8% as the threshold at which the percentage of high risk individuals was similar between the 2 methods and which showed the highest kappa value. This cut-point produced the greatest degree of agreement between the 2 charts and warrant further study in the future when using REGICOR to identify high risk individuals.

Profiles of Subjects Showing Discrepancies Between SCORE and REGICOR

Of the overall sample, 8% of subjects (78.2% of all discrepant cases) were classified as high risk by SCORE but not by REGICOR. Only 2.2% (21.8% of discrepant cases) were classified as high risk by REGICOR but not by SCORE. In other words, for every 5 individuals for whom there were discrepancies between the charts, approximately 4 were classified as high risk by SCORE but not by REGICOR and only 1 was classified as high risk by REGICOR but not by SCORE.

The profiles of the discrepant cases differed. Subjects classified as high risk by SCORE but not by REGICOR were adult women with higher blood pressure, DM, or altered baseline glucose values, and a lipid profile near the upper limit except for HDL-C, which was normal. Cases classified as high risk by REGICOR but not by SCORE were less frequent, and tended to be adult men, smokers, with no history of DM, normal, high-normal, or stage 1 hypertension, and an altered lipid profile, particularly with low HDL-C.

The multivariate analysis explained 93.2% of the variability in the discrepant cases. Diabetes mellitus was excluded from the multivariate model because none of the patients with diabetes were classified as high risk by REGICOR but not by SCORE; baseline glucose levels were included as a continuous variable. The model showed that a predominantly female population with high DBP and baseline glucose values would be classified as high risk by SCORE but not by REGICOR. On the other hand, male smokers with low HDL-C would be classified as high risk by REGICOR but not by SCORE.

The identification of patients who are classified differently by the 2 methods (men, smokers with low HDL-C classified as high risk by REGICOR but not by SCORE) suggests that this profile may indicate increased risk. In these cases, the true risk may be higher than that estimated using the SCORE chart and treatment could be based on the algorithm shown in Figure 3. As shown in the Figure, if an individual was identified as high risk using the SCORE chart that would indicate a need for action based on the protocols described. Individuals who are not classified as high risk by SCORE but who have the type of adverse profile described above (males, smokers, low HDL-C) should be considered as being at increased risk of cardiovascular disease and their management should be individualized.

Figure 3. Risk assessment algorithm based on CEIPC recommendations; identification of discrepant profiles. CEIPC indicates Interdisciplinary Spanish Committee for Cardiovascular Prevention; HDL-C, high density lipoprotein cholesterol; PAPPS, Health Promotion and Preventive Activities Program of the Spanish Society for Family and Community Medicine.

Limitations

The study's main limitation was its cross-sectional design which meant that patients were not followed-up over time. Nevertheless, such a design is appropriate for assessing agreement between the charts and the study was performed in a large sample of the general population aged between 40 and 65 years using a rigorous methodology applied in conditions of usual clinical practice. The sample analyzed could be reasonably representative of the population in the region studied, ie, the Valencian Community, and of the age range studied, although we have no information regarding the characteristics of patients who did not accept the invitation to participate. There may also be variations between autonomous communities. The proposed modifications may help to detect individuals whose true level of cardiovascular risk is higher than that indicated by their SCORE assessment and who may require individualized management. The impact of applying these proposed modifications requires a longitudinal, follow-up study.

CONCLUSIONS

There are discrepancies in risk assessment and the identification of high risk individuals between the SCORE chart for countries with low cardiovascular risk (using a cut-point of 5%) and the REGICOR chart (using a cut-point of 10%). The 2 charts also identify different populations as being at high risk. Defining the type of patient in which discrepancies occur between the 2 charts may help to improve the clinical assessment of cardiovascular risk in patients which the SCORE chart identifies as not being at high risk but whose true cardiovascular risk could be higher than that estimated. The importance of these findings and the impact of their application in clinical practice should be confirmed in future longitudinal studies.

ABBREVIATIONS
BP: blood pressure
CEIPC: Comité Español Interdisciplinario para la Prevención Cardiovascular (Interdisciplinary Spanish Committee for Cardiovascular Prevention)
DM: diabetes mellitus
HBP: high blood pressure
HDL-C: high density lipoprotein cholesterol
LDL-C: low density lipoprotein cholesterol
PAPPS: Programa de Actividades Preventivas y Promoción de la Salud de la Sociedad Española de Medicina Familiar y Comunitaria (Health Promotion and Preventive Activities Program of the Spanish Society for Family and Community Medicine)

---

Correspondence: Dr. V.F. Gil Guillén.
Departamento de Medicina Clínica. Facultad de Medicina. Universidad Miguel Hernández. Campus Universitario de San Juan.
Ctra. de Valencia N-332, km 87. 03550 San Juan. Alicante. España.
E-mail: [e-mail]

Received March 6, 2007.
Accepted for publication June 29, 2007.