Accuracy of Self-Reported Diabetes, Hypertension and Hyperlipidemia in the Adult Spanish Population. DINO Study Findings

INTRODUCTION

The need to study large, representative samples of the overall population usually obliges epidemiological researchers to rely on self-reported diagnoses of disease (generally gathered through questionnaires, interviews, or telephone surveys) rather than clinical examinations and biometrical analyses, more accurate yet much more expensive and time-consuming methods. The use of questionnaires, on the other hand, has many advantages for epidemiological investigation, such as low cost and high efficiency in data collection. For these reasons, they have been extensively used in epidemiological surveys and are regarded as valid tools by most authors.^1,2 Nevertheless, they are not devoid of measurement error (either random or systematic), and the literature shows considerable differences in the accuracy of these methods depending on the nature of the disease, the characteristics of the population, presence of symptoms or health status.^2-9 The accuracy of self-reports depends on the respondent's awareness and understanding of the pathological condition, ability to recall it and willingness to do so,⁴ and evidence exists that patients tend to underreport chronic conditions.¹⁰

Validation studies are aimed at determining whether self-reported data agree with reliable diagnoses obtained by means of clinical examinations, biometrical measurements or structured interviews, whereas validation itself consists of measuring the actual level of this agreement. The aim of the present analysis was to validate self-reported diagnoses of diabetes, hypertension, and hyperlipidemia as compared to biochemical determinations and blood pressure measurements in a regionally-representative sample of 1556 adult Spanish subjects participating in the DINO (DIabetes, Nutrition, and Obesity) study.¹¹

METHODS

Study Design

Details on the setting and sample selection have been published elsewhere.¹¹ Briefly, the DINO study comprised the overall adult population (≥20 years) of Murcia, a Mediterranean Region in the southeast of Spain. Selection of a regionally-representative sample was carried out by randomized sampling, stratifying by health area, type of residence (urban or rural), age and gender. Exclusion criteria were pregnancy or severe mental or physical impairment. The field work took place between July 2000 and June 2003. From a total eligible sample of 2562 subjects, 2094 (81.7%) agreed to complete the questionnaire and 1570 (61.3%), to provide a blood sample. Validation studies were carried out in a final sample of 1556 participants with complete biochemical data. Those participants who refused to provide a blood sample did not differ in terms of age group or educational attainment, although they were significantly more likely to be men, smokers and living in an urban residence. Ethical approval of the study protocol was obtained from the Ethics Committee of Hospital Universitario Virgen de la Arrixaca in Murcia, Spain. All participants signed an informed consent form.

Survey Protocol and Data Collection

Subjects were contacted by mail, telephone, or a home visit. Participants were administered a questionnaire by trained interviewers who collected information on the history of chronic disease, education (maximum level of completed studies), smoking habit, and leisure-time activity (number of hours dedicated to exercise and watching television). Questions on leisure-time physical activity were based on a previously validated questionnaire.¹² In order to assess patients' self-reports (SR) of chronic conditions, participants were asked a series of questions: "Have you ever been told that you are diabetic or have high blood sugar?," "Have you ever been told that you have high blood pressure or that you are hypertensive?," and "Has a doctor ever told you that you have elevated plasma lipids?" Subjects were also asked about family history of diabetes, and whether their blood lipids and blood pressure had been checked during the previous year. Because of the high prevalence of obesity in the region and its documented association with the chronic conditions studied, the subjects were asked the question "Is obesity a disease for you?" When participants self-reported a disease, information on treatment was collected. Possible answers were "yes" (SR+), and "no" or "unknown" (SR-).

Subsequently, during a physical examination conducted by the interviewers, the subjects had their blood pressure checked and height and weight measured. Arterial blood pressure was assessed according to the MONICA protocol¹³ using a digital sphygmomanometer (BOSO Oscillomat^®). Blood pressure values were estimated as the mean of 2 consecutive measures. Height and weight of the participants were recorded and body mass index (BMI) was calculated as the weight divided by the squared height, in kg/m². The interviewers were trained in the standard procedures for collection of data, and the measurement equipment had been previously calibrated. Both the interview and the physical examination took place in the facilities of health centers, except when participants expressed their preference for their home, as was the case of a small number of older subjects.

Specimen Collection and Biochemical Analyses

Two blood samples were drawn from each participant by venepuncture after a 12-hour fast, and maintained cool (<10ºC) in darkness until centrifuged at 1200 g and 4ºC within 6 hours of being drawn. Serum glucose, total cholesterol, and triglyceride concentrations were determined in the automated analyzer Advia 1650 (Siemens Medical Solutions, Tarrytown, New York, USA). Coefficients of variation for glucose, total cholesterol, and triglycerides were 4.0%, 3.8%, and 4.0%, respectively. Analytical procedures were included in external quality assessment programs and met the specifications for acceptable imprecision and bias. All measurements were carried out in the Department of Clinical Biochemistry of Hospital Universitario Morales Meseguer in Murcia, Spain.

Validation Criteria and Analyses

The participants' SR of diabetes, hypertension, and hyperlipidemia were compared to the corresponding biometrical gold standard (GS). Criteria for the definition of chronic disease were those accepted for primary health care at the time of recruitment. Diabetes was characterized by the presence of a fasting glucose concentration ≥126 mg/ dL, treatment (insulin, hypoglycemic drugs, or diet).¹⁴ Hypertensive participants were defined as those having a mean systolic blood pressure ≥140 mm Hg or a mean diastolic blood pressure ≥90 mm Hg (based on 2 consecutive readings) or taking antihypertensive medication.¹⁵ Subjects were considered to be hyperlipidemic when they had a total cholesterol level ≥200 mg/dL, triglycerides ≥200 mg/dL, or were undergoing treatment (drugs, diet) to reduce blood lipids. For all 3 conditions, diet was the sole diagnostic criterion in less than 3% of cases.

A 2×2 table of true positives (SR+/GS+), true negatives (SR-/GS-), false positives (SR+/GS-), and false negatives (SR-/GS+) was built for each chronic condition to calculate sensitivity, specificity and positive and negative predictive values. Cohen's kappa (κ) coefficients were calculated to estimate overall agreement between self-reports and gold standards. In terms of the κ value, the level of agreement was considered to be: slight (≤0.20), fair (0.21-0.40), moderate (0.41-0.60), substantial (0.61-0.80), or almost perfect (≥0.81).¹⁶

Differences in the proportion of subjects self-reporting a condition across categories of studied variables were analyzed by c² tests. Logistic regression was used to calculate the probability that a subject with a positive diagnosis according to the gold standard responded "yes" in the questionnaire. For the analyses, "unknown" responses were coded as "no." To search for factors independently associated with a true positive report for each condition, several multivariate models were built. The decision concerning the final set of adjustment variables (sex, age, BMI, smoking, and educational level) was based on criteria of statistical significance and consistency in the literature. Participants with missing or unknown data in any of the variables considered were excluded only from those analyses involving that variable. Statistical analyses were carried out using SPSS for Windows v12.0 (SPSS Inc., Chicago, IL, USA), and the probability of error was set at 5% (a=.05).

RESULTS

Table 1 shows the distribution of relevant characteristics in the sample population. Participants were predominantly middle-aged or older, with those with primary or higher education and non-smokers being most prevalent. Remarkably, 64% of the sample was overweight or obese; roughly two thirds did not practice any kind of sport on a regular basis and one third spent three or more hours a day watching television. A family history of diabetes was reported by 40% of the participants, and most of them had had their blood pressure measured and their blood lipids analyzed during the previous year.

The proportion of positive self-reports for each condition is presented in Table 2 according to levels of the variables considered. The rates of positive self-reports of diabetes, hypertension, or hyperlipidemia were always higher with increasing age and BMI and decreasing educational level. They were also more frequent among those subjects with a positive response for any of the other 2 conditions. Subjects with a family history of diabetes reported more frequently that they had diabetes, but less frequently that they had hypertension. Other relevant factors with regard to these conditions were the measurement of arterial blood pressure or blood lipids during the previous year, the smoking habit, or the level of physical activity. Nevertheless, it is important to bear in mind that the data are not adjusted for age, a variable underlying many of these associations.

The prevalence, sensitivity, specificity, predictive values, and kappa indices for the 3 chronic conditions are presented in Table 3. The prevalence of diabetes according to the gold standard was approximately 11%, whereas the prevalence was much higher for hypertension (approximately 35%) and hyperlipidemia (around 59%). The self-reported prevalence was always lower than that estimated by the gold standard criteria. The sensitivity ranged from 35% for self-reported hyperlipidemia to nearly 70% for self-diagnoses of diabetes, and the specificity and positive predictive value were, in all cases, above 96% and 89%, respectively. The negative predictive values ranged from 52% to about 96%. According to the kappa values, the overall agreement between patients' self-reports and the gold standard diagnoses was substantial for diabetes (κ=0.78), moderate for hypertension (κ=0.51), and fair (κ=0.27) for hyperlipidemia.

Table 4 presents the results of multivariate logistic regression analyses carried out to search for factors related to the validity of self-reported chronic disease among affected subjects. Women were more likely to report hypertension correctly, whereas middle-aged and older subjects were more accurate in reporting hyperlipidemia. A family history of diabetes positively determined a correct self-diagnosis of diabetes, as well as hyperlipidemia. Self-reports of the 3 conditions were more accurate if the subject had been checked for arterial blood pressure during the previous year. Additionally, both hyperlipidemic and hypertensive subjects who had been tested for blood lipids in the past year were more likely to give a true positive response for these conditions.

The sensitivity of self-reports is shown according to sex, individually, and for every combination of chronic disease, in Figure 1. In general, the percentage of true-positive respondents was higher when reporting only one condition. This percentage decreased for any combination. Among hypertensive patients, a higher percentage of women reported the disease, whereas the sensitivity of self-reports of diabetes or hyperlipidemia did not show marked sex-related differences.

Figure. Positive agreement (sensitivity or percentage of patients reporting the disease) between self-reports and gold standards in subjects with chronic disease, by sex. In every category, all the subjects with the indicated pathology were included, regardless of whether they also suffered from either or both of the other 2 conditions.

DISCUSSION

In this study, we have measured the level of overall agreement between self-reported information on chronic conditions and biometrical measures, considered as the gold standard for validation. Among participants in the DINO study (a population-based survey of the prevalence of diabetes and related factors in Southern Spain), we found agreement to be "substantial" for diabetes, "moderate" for hypertension and just "fair" for hyperlipidemia. As shown in Table 3, sensitivity was relatively high for self-reported diabetes and lower for hypertension and hyperlipidemia. Specificity was high overall, and again it was higher for diabetes.

The prevalence of diabetes in the sample was approximately 11%. As a major cause of death and disability in the general population, the prevention of diabetes has received a great deal of attention in public health, being the focus of educational and screening programs. In the region of Murcia, the implementation of primary care service for the control of diabetes reaches 90% of patients, and patients undergo glycohemoglobin testing at least once every 6 months.¹⁷ These efforts have led to a significant level of awareness in the population, as shown by the high Cohen's kappa coefficient and high positive and negative predictive values. The sensitivity in our study is within the low range of published data.^{1,2,4,9,18-22} In spite of the large heterogeneity in the literature, there are studies comparable to ours in terms of design and validation criteria. Among those, a Dutch study reported a lower sensitivity value of 59%,⁶ whereas 2 other surveys carried out in Finnish population found sensitivity values of approximately 80%.^2,19

Hypertension was reported by 20% of the participants, but the actual prevalence according to the study diagnostic criteria was higher (35%). However, this difference was much more significant when considering the prevalence data for hyperlipidemia in the sample, 22% according to self-reports and almost 59% according to the gold standard. First, this may reflect a recall bias or actual unawareness of the condition due to the failure to undergo testing, but it might also suggest that there is no consensus among physicians concerning the criteria for reporting these conditions, and that some of them may be using higher threshold values to define hypertension (and hyperlipidemia), as previously suggested.^23,24 Only 2 studies have validated self-reported hypertension in Spanish populations.^24,25 The first one comprised a small sub-sample of the EPIC-Murcia cohort recruited between 1992 and 1996.²⁴ When comparing data for the same age range (35 to 65 years), we found lower a sensitivity (48.3% vs 62.5%) and kappa value (0.48 vs 0.58) for self-reported hypertension in the period from 2001 to 2003 than between 1992 and 1996 (data not shown). While these results may be partly explained by the characteristics of the EPIC cohort (a high proportion of blood donors and mostly women), they also point out the lack of improvement in patient awareness of hypertension in recent years. It is conceivable, as suggested by Tormo et al,²⁴ that the use of the previous criterion of 160/95 mm Hg by some physicians might have led us to define as hypertensives patients who had not been diagnosed as such. In the one other study carried out by Alonso et al in a Spanish population,²⁵ almost all the false negatives disappeared when the authors increased the cut-off point to 160/95 mm Hg. In our analysis, the number of false negatives decreased by 68% when the criterion of 160/95 mm Hg was used, raising the sensitivity and kappa value to 73% and 0.71, respectively (data not shown). A certain amount of underrecording, together with underreporting, might be accounting for the poor values reported. Overall, Alonso et al found substantial agreement for self-reported hypertension, but a lower degree of agreement for self-reported blood pressure. Data from other countries vary considerably; even in healthy adult populations, sensitivity ranged from 35% to 90%, specificity from 52% to 98%, and kappa values between 0.38 and 0.88.^{1,6,9,18,19,26-28}

The question in validating hyperlipidemia refers to elevated blood lipids, but it makes no explicit mention of cholesterol or triglycerides, which may have misled some participants (especially those with lower education levels). In support of this idea, we found a higher percentage of false negatives (49.0% vs 35.1%) and a lower degree of overall agreement (κ=0.19 vs κ=0.30) in those who had not completed studies as compared to the rest of the group (data not shown). However, the probability of a true positive response did not differ significantly across levels of education among hyperlipidemic subjects (Table 4). The possibility exists that different cutoffs for defining hyperlipidemia (especially hypercholesterolemia) may have produced an effect similar to that described for hypertension. Actually, setting the threshold at 250 mg/dL, we found an increase in both sensitivity (61.1%) and negative predictive value (87.3%), although the positive predictive value decreased to 72.3%; on the whole, the kappa value improved notably (κ=0.56). Studies in the literature have generally focused on validating self-reported hypercholesterolemia, but not hypertriglyceridemia. One study which has analyzed both indices, albeit separately, showed a lower sensitivity for triglycerides than for cholesterol (50% vs 64%).²⁷ Accordingly, the low sensitivity in our study may depend more on false negative cases of elevated triglycerides than on false negative cases of high cholesterol. Our reported sensitivity value for hyperlipidemia is the lowest among published the data (34.5%), only comparable to those published in rural populations.^1,29 The few studies that reported kappa values showed that overall agreement was low and similar to our result (approximately 0.3).^9,29

This analysis of factors associated with a correct case of chronic disease revealed interesting results. Among other factors, having undergone either a blood pressure test or a determination of blood lipids in the last year significantly increased the accuracy of self-reports. This increase was present not only for the disease involved but also for either (or both) of the other diseases. However, positive agreement did not improve in any comorbid situation on the whole (Figure 1). It was remarkable that hypertensive women were much more likely than men to self-report their condition (59% vs 41%). A probable explanation may be that women are more likely to undergo voluntary analyses of blood pressure, along with a greater health consciousness among women. Our results point out the importance of integrated screening for chronic conditions. The current paradigm of focusing on overall cardiovascular risk,³⁰ instead of considering risk factors separately, reflects a far more coherent view of cardiovascular disease and will help to improve the identification of the populations at risk and the efficiency of medical care.

This study has some limitations worth considering. The use of biometrical data to diagnose a disease may be disadvantageous when there is not a universal cut-off point for defining a particular condition, and some misclassification related to differences in clinical criteria, and not to inaccuracy in the ability of subjects to recall or report the condition, may result from the use of such data. In additional, biometrical data collected during a single examination may not be sufficient to detect all the subjects affected by a disease (which would bias the estimation of disease prevalence), or it also may generate false positive results. This limitation may be more relevant when considering hypertension (defined on the basis of two consecutive readings) than diabetes (the definition of which included HbA_1c, a mid-term index of exposure to high blood glucose levels). Another consideration refers to Cohen's kappa coefficient. Although this statistic is generally accepted and widely used, it is severely affected by imbalances in marginal totals and differences in disease prevalence.^21,31 Regardless of these limitations, the present study also has important strengths. The analyses were based on a large and regionally-representative sample, and the response rate was high (82% for the questionnaire and 61% for the questionnaire and blood sample). Finally, and most remarkably, this investigation contributes by mitigating the paucity of validation studies conducted on self-reported chronic conditions in the European population. The necessity for having upto-date studies of this kind lies in the chronological evolution of clinical diagnostic criteria, the attitude of the physicians and the knowledge of patients concerning health topics. Future investigations should study in depth the precise causes for the lack of agreement among self-reports of chronic disease, especially hypertension and hyperlipidemia: lack of analysis, improper reporting by the physician or insufficient health consciousness of the patient.

CONCLUSIONS

We have found substantial agreement for self-reported diabetes in a sample of adults from Southern Spain. The degree of agreement was lower for hypertension and hyperlipidemia, for which self-reports may not be valid estimates of disease prevalence. Further efforts are necessary in the future to ensure that chronic patients become aware of their disease.

ACKNOWLEDGEMENTS

The authors would like to express their gratitude to the participating centers from the different health areas of the region: Murcia (Santa María de Gracia and Santa Cruz health centers), Cartagena (Cartagena Oeste and Pozo Estrecho health centers), Lorca (Lorca Centro and Purias health centers), Caravaca (Caravaca and Barranda health centers), Yecla (Yecla and Campo Abajo health centers), and Molina de Segura (La Consolación and El Llano health centers). The authors would like to gratefully acknowledge the personnel in these centers for their kind collaboration and support, and all those subjects who kindly accepted to take part in the study.

ABBREVIATIONS
DINO: DIabetes, Nutrition, and Obesity

The DINO study was partially supported by the Spanish Health Research Fund (FIS 01/0711) and a grant for public health centers from Instituto de Salud Carlos III (C03/09).

---

Correspondence: Dr. J.M. Huerta Castaño.
Servicio de Epidemiología. Consejería de Sanidad. Ronda de Levante, 11. 30008 Murcia. España.
E-mail: [e-mail]

Received March 11, 2008.
Accepted for publication October 23, 2008.