We read the study by Huerta et al1 with considerable interest. We feel that validating the precision of diagnostic questionnaires is important from an epidemiological point of view because of their advantages (comfort, time, and low cost) compared with biometric and clinical methods, but we would like to make some comments on the study in order to contribute to the scientific debate and the exchange of ideas:
1. Although the authors state that "the interviewers are well qualified," we believe that the use of a procedure (for example, the Kappa index) to establish concordance among them would have helped make the study more reliable.2
2. As for the prevalence rates for the different diseases, the authors state that "all of the reported prevalence rates are lower than the estimates given by the reference model." However, in the case of diabetes, the 95% confidence intervals (CI) for the specific model and questionnaire overlap, so it cannot be stated that the specific resulting measurements are different from each other.
3. The statement in the article, based on the Kappa index calculation, is that "we have reached a degree of concordance between the diagnosis in question and the proper reference model for diabetes, with moderate AHT and low hyperlipaemia." This is not correct; although a lesser-known use of the Kappa index is evaluating discrepancies in diagnostic test, the indispensable requirement for using the index in this way is for the interpretation of the tests to be subject to the human factor (ie, radiographies, electrocardiograms, anatomical pathology samples, etc),3 which does not occur in this case, since the measurements for the reference model are biometric, calculated by automatic devices, calibrated, and therefore each measurement is not subject to human variability. Furthermore, the declared diagnosis (the sum of the data contributed by several researchers and not previously subjected to any test of concordance among observers) is compared with data from the test of reference (which is objective from the viewpoint of variability for each ascertainment), and for this reason that comparison cannot be made. In addition, the sensitivities for all three diseases are below 70%. We know from mathematical models that with sensitivities below 70% the Kappa index will necessarily be <0.4 because low sensitivity itself leads to discordance.2
4. Given that the purpose of the study is to evaluate the usefulness of the diagnostic questionnaire in question, using diagnoses from biometric tests as a standard, we believe that in addition to calculating sensitivity, specificity and predictive values, the coefficients of positive and negative probability (CPP and CPN respectively) should have been calculated as well. This is because the sensitivity and specificity values, despite completely defining validity (the degree to which a test measures what it is supposed to measure) for the diagnostic test, have the disadvantage of not providing relevant information when it comes to making a clinical decision when faced with a certain test result. The CPP and CPN express a unified summary of sensitivity and specificity, and therefore do not depend on the disease prevalence in each location and permit comparison between different studies, unlike predictive values, which are only valid for the location at which they were calculated.4,5 In this study, the CPP and CPN with 95% CI for each disease (CPP for diabetes, 161.46 [72.21-361.06]; AHT, 15.41 [10.84-21.9]; hyperlipaemia, 10.98 [7.07-17.05]; CPN for diabetes, 0.30 [0.24-0.38]; AHT, 0.52 [0.48-0.57]; hyperlipaemia, 0.68 [0.64-0.71]) indicate that the questionnaire is useful for diagnosing diabetes and hypertension (relevant CPP >10 with 95% CI) but they are not useful for ruling them out (CPN are not <0.1). With regard to hyperlipaemia, it will be necessary to design more studies (coefficients of probability are statistically significant, but not clinically conclusive).
5. When we refer to diagnostic test studies, parameters ≥80% are considered to be optimum or high, and for this reason, we cannot claim the sensitivities of the study as such.
To conclude, we must be careful and methodical with all details involved in a research study in order to contribute efficiently to its comprehensibility and usefulness to the researchers and clinics for which it is intended.