pdf (Español (España))
FLIP
How to Cite
Simancas Pallares, M. ., & Arévalo Tovar, L. . (2017). Performance of four statistical methods for the assessment of test re-test reliability of continuous variables in a sample. Biosalud, 16(1), 19–29. https://doi.org/10.17151/biosa.2017.16.1.4
More Citation Formats
Authors
Abstract
To compare the performance of four statistical tests in continuous variables test/retest reliability assessment. Methods: Statistical simulation study developed in the framework of an in vitro diagnostic test study including 120 teeth which met the inclusion criteria. Each tooth was positioned in a standardization device and was taken two digital x-rays (T0 and T1) in which we assessed tooth-length. Data were analyzed with descriptive statistics and then a statistical comparison was done with paired Student’s “t” test, intraclass correlation coefficient, Pearson correlation coefficient and Lin’s concordance correlation coefficient in Stata v.13.2 for Windows (StataCorp., TX., USA). Results: The average dental length for T0 was 21.15 mm and for T1 21.07 mm. Student’s “t” test revealed an average difference of 0.089 (P=0.00). The intraclass correlation coefficient 0.877 (95% CI: 0.43 – 0.98), Pearson’s productmoment correlation coefficient 0.93, and Lin’s concordance correlation coefficient 0.93 (95% CI: 0.908 – 0.956). Conclusions: Selection of a statistical test for test/re-test reliability assessment should be made having in mind the research objectives in any context and the possibility of each method for error assessment. Thus, a method that currently complies with this essential requirement is Lin’s concordance correlation coefficient, which is recommended for future test re-test research studies.
References
1. Bahrololoomi Z, Ezoddini F, Halvani N. Comparison of Radiography, Laser Fluorescence and Visual Examination for Diagnosing Incipient Occlusal Caries of Permanent First Molars. Journal of Dentistry. 2015; 12(5):324-32.
2. Gomez J. Detection and diagnosis of the early caries lesion. BMC oral health. 2015; 15 Suppl 1:S3.
3. Niederman R. Manual and electronic probes have similar reliability in the measurement of untreated periodontitis. Evidence-based dentistry. 2009; 10(2):39.
4. Kumar LV, Sreelakshmi N, Reddy ER, Manjula M, Rani ST, Rajesh A. Clinical Evaluation of Conventional Radiography, Radiovisiography, and an Electronic Apex Locator in Determining the Working Length in Primary Teeth. Pediatric Dentistry. 2016; 38(1):37-41.
5. Leonardi Dutra K, Haas L, Porporatti AL, Flores-Mir C, Nascimento Santos J, Mezzomo LA, et al. Diagnostic Accuracy of Cone-beam Computed Tomography and Conventional Radiography on Apical Periodontitis: A Systematic Review and Meta-analysis. Journal of Endodontics. 2016;42(3):356-64.
6. Tadinada A, Mahdian M, Sheth S, Chandhoke TK, Gopalakrishna A, Potluri A, et al. The reliability of tablet computers in depicting maxillofacial radiographic landmarks. Imaging science in dentistry. 2015; 45(3):175-80.
7. Cortés-Reyes E, Rubio-Romero JA, Gaitán-Duarte H. Statistical methods for evaluating diagnostic test agreement and reproducibility. Revista Colombiana de Obstetricia y Ginecología. 2010; 61(3):247-55.
8. Atkinson G, Nevill AM. Statistical methods for assessing measurement error (reliability) in variables relevant to sports medicine. Sports Medicine. 1998; 26(4):217-38.
9. Kramer MS, Feinstein AR. Clinical biostatistics. LIV. The biostatistics of concordance. Clinical Pharmacology & Therapeutics. 1981; 29(1):111-23.
10. Aravena PC, Moraga J, Cartes-Velásquez R, Manterola R. Validity and Reliability in Dental Research. Int J Odontostomat. 2014; 8(1):69-75.
11. Alarcón A, Muñoz S. Medición en salud: algunas consideraciones metodológicas. Rev Med Chile. 2008;1 36(1):125-30.
12. Fleiss JL. The design and analysis of clinical experiments. New York: John Wiley and Sons; 1986.
13. Lin LI. A concordance correlation coefficient to evaluate reproducibility. Biometrics. 1989; 45(1):255- 68.
14. Gómez-Gómez M, Danglot-Banck C, Vega-Franco L. Choosing a statistical test. Second part. Revista Mexicana de Pediatría. 2013; 80(2):81-5.
15. Student. The probable error of a mean. Biometrika. 1908; 6(1):1-25.
16. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychological bulletin. 1979; 86(2):420-8.
17. Pita-Fernández S, Pértega-Díaz S. Relación entre variables cuantitativas. Cad Aten Primaria. 1997; 4:141-4.
18. Chuan A, Thillainathan S, Graham P, Jolly B, Wong D, Smith N, et al. Reliability of numerical scales used for direct observation of procedural skills. Anaesthesia and intensive care.
2016; 44(2):201-8.
19. Flores-Mir C, Rosenblatt MR, Major PW, Carey JP, Heo G. Measurement accuracy and reliability of tooth length on conventional and CBCT reconstructed panoramic radiographs. Dental press journal of orthodontics. 2014; 19(5):45-53.
20. Oznurhan F, Tuzuner T, Baygin O, Unal M, Kapdan A, Ozturk C. Accuracy of three different apex locators and visual exam in primary teeth with and without root resorption in vitro. European journal of paediatric dentistry: Official Journal of European Academy of Paediatric Dentistry. 2014;15(4):381-4.
21. McBride GB. A proposal for strenght-of-agreement criteria for Lin’s concordance correlation coefficient. National Institution of Water & Atmospheric Research Ltd, 2005 HAM2005-062.
22. Oliveira ML, Vieira ML, Cruz AD, Boscolo FN, De Almeida SM. Gray scale inversion in digital image for measurement of tooth length. Brazilian Dental Journal. 2012; 23(6):703-6.
23. Scaf G, Morihisa O, Loffredo L de C. Comparison between inverted and unprocessed digitized radiographic imaging in periodontal bone loss measurements. Journal of applied oral science: revista FOB. 2007; 15(6):492-4.
24. Feltz CJ, Miller GE. An asymptotic test for the equality of coefficients of variation from k populations. Statistics in Medicine. 1996; 15(6):646-58.
25. Carrasco J, Jover L. Métodos estadísticos para evaluar la concordancia. Medicina Clínica (Barcelona). 2004; 122(Supl 1):28-34.
26. Olds T. Five errors about error. Journal of Science and Medicine in Sport / Sports Medicine Australia. 2002; 5(4):336-40.
27. Weir JP. Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. Journal of Strength and Conditioning Research / National Strength & Conditioning Association. 2005; 19(1):231-40.
28. Bodur H, Odabas M, Tulunoglu O, Tinaz AC. Accuracy of two different apex locators in primary teeth with and without root resorption. Clinical Oral Investigations. 2008; 12(2):137-41.
29. Bland JM, Altman DG. Comparing two methods of clinical measurement: a personal history. International Journal of Epidemiology. 1995; 24 Suppl 1:S7-14.
30. Bates BT, Zhang S, Dufek JS, Chen FC. The effects of sample size and variability on the correlation coefficient. Medicine and Science in Sports and Exercise. 1996; 28(3):386-91.
31. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986; 1(8476):307-10.
32. Sánchez-Ayala A, Farias-Neto A, Vilanova LS, Costa MA, Paiva AC, Carreiro AD, et al. Reproducibility, Reliability, and Validity of Fuchsin-Based Beads for the Evaluation of Masticatory Performance. Journal of Prosthodontics: official journal of the American College of Prosthodontists. 2015.
33. Baumgartner TA. Norm-referenced measurement: reliability. Measurement concepts in physical education and exercise science. Champaing, IL: Human Kinetics; 1989. p. 45-72.
34. Bland JM, Altman DG. A note on the use of the intraclass correlation coefficient in the evaluation of agreement between two methods of measurement. Computers in Biology and Medicine. 1990; 20(5):337-40.
35. Cepeda MS, Africano JM, Polo R, Alcala R, Carr DB. Agreement between percentage pain reductions calculated from numeric rating scores of pain intensity and those reported by patients with acute or cancer pain. Pain. 2003; 106(3):439-42.
36. Bland JM, Altman DG. Comparing methods of measurement: why plotting difference against standard method is misleading. Lancet. 1995; 346(8982):1085-7.
37. Carrasco JL, Jover L, King TS, Chinchilli VM. Comparison of concordance correlation coefficient estimating approaches with skewed data. Journal of Biopharmaceutical Statistics. 2007; 17(4):673- 84.
38. Altman DG. Practical statistics for medical research. London: Chapman and Hall; 1991.
39. Bishop D. Reliability of a 1-h endurance performance test in trained female cyclists. Medicine and science in sports and exercise. 1997; 29(4):554-9.
2. Gomez J. Detection and diagnosis of the early caries lesion. BMC oral health. 2015; 15 Suppl 1:S3.
3. Niederman R. Manual and electronic probes have similar reliability in the measurement of untreated periodontitis. Evidence-based dentistry. 2009; 10(2):39.
4. Kumar LV, Sreelakshmi N, Reddy ER, Manjula M, Rani ST, Rajesh A. Clinical Evaluation of Conventional Radiography, Radiovisiography, and an Electronic Apex Locator in Determining the Working Length in Primary Teeth. Pediatric Dentistry. 2016; 38(1):37-41.
5. Leonardi Dutra K, Haas L, Porporatti AL, Flores-Mir C, Nascimento Santos J, Mezzomo LA, et al. Diagnostic Accuracy of Cone-beam Computed Tomography and Conventional Radiography on Apical Periodontitis: A Systematic Review and Meta-analysis. Journal of Endodontics. 2016;42(3):356-64.
6. Tadinada A, Mahdian M, Sheth S, Chandhoke TK, Gopalakrishna A, Potluri A, et al. The reliability of tablet computers in depicting maxillofacial radiographic landmarks. Imaging science in dentistry. 2015; 45(3):175-80.
7. Cortés-Reyes E, Rubio-Romero JA, Gaitán-Duarte H. Statistical methods for evaluating diagnostic test agreement and reproducibility. Revista Colombiana de Obstetricia y Ginecología. 2010; 61(3):247-55.
8. Atkinson G, Nevill AM. Statistical methods for assessing measurement error (reliability) in variables relevant to sports medicine. Sports Medicine. 1998; 26(4):217-38.
9. Kramer MS, Feinstein AR. Clinical biostatistics. LIV. The biostatistics of concordance. Clinical Pharmacology & Therapeutics. 1981; 29(1):111-23.
10. Aravena PC, Moraga J, Cartes-Velásquez R, Manterola R. Validity and Reliability in Dental Research. Int J Odontostomat. 2014; 8(1):69-75.
11. Alarcón A, Muñoz S. Medición en salud: algunas consideraciones metodológicas. Rev Med Chile. 2008;1 36(1):125-30.
12. Fleiss JL. The design and analysis of clinical experiments. New York: John Wiley and Sons; 1986.
13. Lin LI. A concordance correlation coefficient to evaluate reproducibility. Biometrics. 1989; 45(1):255- 68.
14. Gómez-Gómez M, Danglot-Banck C, Vega-Franco L. Choosing a statistical test. Second part. Revista Mexicana de Pediatría. 2013; 80(2):81-5.
15. Student. The probable error of a mean. Biometrika. 1908; 6(1):1-25.
16. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychological bulletin. 1979; 86(2):420-8.
17. Pita-Fernández S, Pértega-Díaz S. Relación entre variables cuantitativas. Cad Aten Primaria. 1997; 4:141-4.
18. Chuan A, Thillainathan S, Graham P, Jolly B, Wong D, Smith N, et al. Reliability of numerical scales used for direct observation of procedural skills. Anaesthesia and intensive care.
2016; 44(2):201-8.
19. Flores-Mir C, Rosenblatt MR, Major PW, Carey JP, Heo G. Measurement accuracy and reliability of tooth length on conventional and CBCT reconstructed panoramic radiographs. Dental press journal of orthodontics. 2014; 19(5):45-53.
20. Oznurhan F, Tuzuner T, Baygin O, Unal M, Kapdan A, Ozturk C. Accuracy of three different apex locators and visual exam in primary teeth with and without root resorption in vitro. European journal of paediatric dentistry: Official Journal of European Academy of Paediatric Dentistry. 2014;15(4):381-4.
21. McBride GB. A proposal for strenght-of-agreement criteria for Lin’s concordance correlation coefficient. National Institution of Water & Atmospheric Research Ltd, 2005 HAM2005-062.
22. Oliveira ML, Vieira ML, Cruz AD, Boscolo FN, De Almeida SM. Gray scale inversion in digital image for measurement of tooth length. Brazilian Dental Journal. 2012; 23(6):703-6.
23. Scaf G, Morihisa O, Loffredo L de C. Comparison between inverted and unprocessed digitized radiographic imaging in periodontal bone loss measurements. Journal of applied oral science: revista FOB. 2007; 15(6):492-4.
24. Feltz CJ, Miller GE. An asymptotic test for the equality of coefficients of variation from k populations. Statistics in Medicine. 1996; 15(6):646-58.
25. Carrasco J, Jover L. Métodos estadísticos para evaluar la concordancia. Medicina Clínica (Barcelona). 2004; 122(Supl 1):28-34.
26. Olds T. Five errors about error. Journal of Science and Medicine in Sport / Sports Medicine Australia. 2002; 5(4):336-40.
27. Weir JP. Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. Journal of Strength and Conditioning Research / National Strength & Conditioning Association. 2005; 19(1):231-40.
28. Bodur H, Odabas M, Tulunoglu O, Tinaz AC. Accuracy of two different apex locators in primary teeth with and without root resorption. Clinical Oral Investigations. 2008; 12(2):137-41.
29. Bland JM, Altman DG. Comparing two methods of clinical measurement: a personal history. International Journal of Epidemiology. 1995; 24 Suppl 1:S7-14.
30. Bates BT, Zhang S, Dufek JS, Chen FC. The effects of sample size and variability on the correlation coefficient. Medicine and Science in Sports and Exercise. 1996; 28(3):386-91.
31. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986; 1(8476):307-10.
32. Sánchez-Ayala A, Farias-Neto A, Vilanova LS, Costa MA, Paiva AC, Carreiro AD, et al. Reproducibility, Reliability, and Validity of Fuchsin-Based Beads for the Evaluation of Masticatory Performance. Journal of Prosthodontics: official journal of the American College of Prosthodontists. 2015.
33. Baumgartner TA. Norm-referenced measurement: reliability. Measurement concepts in physical education and exercise science. Champaing, IL: Human Kinetics; 1989. p. 45-72.
34. Bland JM, Altman DG. A note on the use of the intraclass correlation coefficient in the evaluation of agreement between two methods of measurement. Computers in Biology and Medicine. 1990; 20(5):337-40.
35. Cepeda MS, Africano JM, Polo R, Alcala R, Carr DB. Agreement between percentage pain reductions calculated from numeric rating scores of pain intensity and those reported by patients with acute or cancer pain. Pain. 2003; 106(3):439-42.
36. Bland JM, Altman DG. Comparing methods of measurement: why plotting difference against standard method is misleading. Lancet. 1995; 346(8982):1085-7.
37. Carrasco JL, Jover L, King TS, Chinchilli VM. Comparison of concordance correlation coefficient estimating approaches with skewed data. Journal of Biopharmaceutical Statistics. 2007; 17(4):673- 84.
38. Altman DG. Practical statistics for medical research. London: Chapman and Hall; 1991.
39. Bishop D. Reliability of a 1-h endurance performance test in trained female cyclists. Medicine and science in sports and exercise. 1997; 29(4):554-9.
Downloads
Download data is not yet available.
