Alternative Agreement Approaches in Testing the Agreement of Three-dimensional Computerized Tomography Measurements of Styloid Processes by Two Observers at Two Different Times*

Year 2019, Volume: 24 Issue: 2, 93 - 100, 13.06.2019

Semra Erdoğan , Kaan Esen , Yüksel Balcı Sermin Tok Umay , Gülhan Temel , Anıl Özgür

Abstract

Aim: This study aimed to evaluate the intraobserver and interobserver
agreement of styloid process (SP) measurements on three-dimensional (3D)
computerized tomography (CT) images between two experienced radiologists by
using the concordance correlation coefficient (CCC) and coefficient of
individual agreement (CIA) agreement statistical methods.

Materials and Methods: Contrast-enhanced carotid CT angiography images
of 68 patients, per­formed between June 2015 and December 2015, were evaluated
retrospectively. The length between the attachment point of the SP to the
temporal bone and the distal end was measured on 3D CT images performed with a
64-slice CT scanner. Both the agreements between the two radiologists and the
replicated measurements of each radiologist were calculated with the help of
CIA and CCC agreement indexes.

Results: When CIA was used for agreement statistics, the two
radiologists disagreed in the right and left measurements of the individual.
When the CCC agreement statistic was used, there was a perfect agreement
between the measurements of the two radiologists.

Discussion and Conclusion: The variance values
of between-subject and within-subject should be taken into consideration for
each observer in cases of two replicated measurements. If these values show
very large differences from each other, CCCtotal values may have been
calculated larger than the real value. However, the CIA statistics value is
more stable and when such a case is encountered, researchers are advised that
between-subject and within-subject variances should be calculated due to the
differences between the two replicated measurements of each observer.
Therefore, the 2 2 W B rate should
absolutely be taken into consideration

Keywords

disaggrement, interobserver agreement, intraobserver agreement, replicated measurement, styloid processes

Stiloid Proseslerin İki Farklı Zamanda İki Değerlendirici Tarafından Alınan Üç Boyutlu Bilgisayarlı Tomografi Ölçümlerinin Uyumunun Test Edilmesinde Alternatif Uyum Yaklaşımları

Abstract

 Amaç: Bu çalışmada üç boyutlu
bilgisayarlı tomografi (BT) görüntüleri üzerinde iki deneyimli radyolog tarafından
alınan stiloid proses (SP) ölçümlerinin değerlendiriciler arası ve değerlendirici
içi uyumunu, uyum istatistik yöntemleri olan konkordans korelasyon katsayısı
(concordance correlation coeffici­ent—CCC) ve birey uyum katsayısı (coefficient
of individual agreement—CIA) metotları kullanılarak değerlendirilmesi amaçlanmıştır.

Gereç ve Yöntemler: Haziran 2015—Aralık 2015 döneminden, 68 hastaya ait
kontrastlı karotis BT anjiyografi görüntüleri geriye dö­nük olarak incelendi.
SP temporal kemiğe tutunma noktası ve dis­tal ucu arasındaki mesafe, 64 kesitli
bir BT cihazı ile gerçekleştirilen üç boyutlu BT görüntülerinde ölçüldü.
Radyologlar arasındaki ve radyologların kendi tekrarlı ölçümleri arasındaki
uyum, CIA ve CCC uyum indekslerinden yararlanılarak hesaplandı.

Bulgular: Uyum istatistikleri olarak CIA kullanıldığında, bireylerin sağ
ve sol ölçümlerinde iki radyolog arasında uyum bulunmadı. CCC uyum istatistiği
kullanıldığında ise iki radyoloğun tekrarlı öl­çümleri arasında mükemmel bir
uyum bulundu.

Tartışma ve Sonuç: Her bir değerlendiriciden iki tekrarlı ölçüm alındığında,
denekler arası ve denekler içi varyans değerlerinin dikkate alınması
gerekmektedir. Eğer bu değerler birbirinden ol­dukça büyük farklılıklar gösteriyorsa,
CCCtotal değerleri gerçek değerden daha büyük bir değer olarak hesaplanacak ve
elde edilen bu sonuçlar araştırmacıları yanıltacaktır. Böyle bir durumla karşı­laşıldığında,
CIA istatistiğinin çok daha kararlı olduğu ve denekler arası ve denekler içi
varyans değerlerinin, her bir değerlendiricinin iki tekrarlı ölçümleri arasındaki
farklılıklar üzerinden hesaplanma­sı gerektiği araştırmacılara önerilir. Böylece,
2 2 W B oranı mutlaka dikkate alınmalıdır. 

Keywords

değerlendirici içi uyum, değerlendiriciler arası uyum, ; stiloid proses, tekrarlı ölçümler, uyumsuzluk

References

• 1. Gao J, Pan Y, Haber M. Assessment of observer agreement for matched repeated binary measurements. Computational Statistics & Data Analysis 2012; 56 (5):1052-60.
• 2. Barnhart HX, Haber MJ, Lin LI. An overview on assessing agreement with continuous measurements. J Biopharm Stat 2007; 17(4): 529-69.
• 3. Barnhart HX, Lokhnygina Y, Kosinski AS, Haber M. 2007-b. Comparison of concordance correlation coefficient and coefficient of individual agreement in assessing agreement. J Biopharma Stat 2007; 17(4): 721-38.
• 4. Bland J, Altman D. Measuring agreement in method comparison studies. Stat Methods Med Res 1999; 8(2): 135-160.
• 5. Bland J, Altman D. Agreement between methods of measurement with multiple observations per individual. J Biopharm Stat 2007; 17(4): 571-82.
• 6. Başekim CC, Mutlu H, Güngör A, Silit E, Pekkafali Z, Kutlay M, et al. Evaluation of styloid process by three-dimensional computed tomography. Eur Radiol 2005; 15(1): 134-9.
• 7. Kosar MI, Atalar MH, Sabancioğullari V, Tetiker H, Erdil FH, Cimen M, et al. Evaluation of the length and angulation of the styloid process in the patient with pre-diagnosis of Eagle syndrome. Folia Morphol (Warsz) 2011; 70(4): 295-9.
• 8. Oztunç H, Evlice B, Tatli U, Evlice A. Cone-beam computed tomographic evaluation of styloid process: a retrospective study of 208 patients with orofacial pain. Head Face Med 2014;10(5): 1-7.
• 9. Okur A, Ozkırış M, Serin HI, Gencer ZK, Karaçavuş S, Karaca L, et al. Is there a relationship between symptoms of patients and tomographic characteristics of styloid process? Surg Radiol Anat 2014; 36(7): 627-32.
• 10. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. International Journal of Nursing Studies 2010; 47 (8): 931-6.
• 11. Van Stralen KJ, Dekker FW, Zoccali C, Jager KJ. Measuring agreement, more complicated than it seems. Nephron Clin Pract 2012; 120(3):c162-7.
• 12. Lin L, Hedayet AS, Wu W. (2012), A Unified Model for Continuous and Categorical Data. Statistical Tools for Measuring Agreement. 1st Ed. New York Springer-Verlag.
• 13. Vangeneugden T, Laenen A, Geys H, Renard D, Molenberghs G. Applying linear mixed model to estimate reliability in clinical trials with repeated measurement. Controlled Clinical Trials 2004; 25: 13-30.
• 14. Molenbergs G, Vangeneugden T, Laenen A. Estimating reliability and generalizability from hierarchical biomedical data. Journal of Biopharmaceutical Statistics 2007; 17(4): 595-627.
• 15. King TS, Chinchilli VM, Carrasco JL, Wang K. A Class of repeated measures concordance correlation coefficients. Journal of Biopharmaceutical Statistics 2007; 17(4): 653-72.
• 16. Quiroz J. Assessment of equivalence using a concordance correlation coefficient in repeated measurement design. Journal of Biopharmaceutical Statistics 2005; 15: 913-28.
• 17. Chen CC, Barnhart HX. Assessing agreement with intraclass correlation coefficient and concordance correlation coefficient for data with repeated measures. Computational Statistics and Data Analysis 2013; 60: 132-45.
• 18. Haber M, Barnhart HX. A general approach to evaluating agreement between two observers or methods of measurement. Stat Methods Med Res 2008; 17(2):151–69.
• 19. Haber M, Barnhart HX, Song J, Gruden J. Observer Variability: A new approach in evaluating interobserver agreement. Journal of Data Science 2005; 3: 69-83.
• 20. Lin L, Hedayat AS, Wu W. A Unified approach for assessing agreement for continuous and categorical data. J Biopharm Stat 2007;17(4): 629-52.
• 21. Vieira EM, Guedes OA, Morais SD, Musis CR, Albuquerque PA, Borges AH. Prevalence of Elongated Styloid Process in a Central Brazilian Population. J Clin Diagn Res 2015; 9(9): ZC90-2.
• 22. Jung T, Tschernitschek H, Hippen H, Schneider B, Borchers L. Elongated styloid process: when is it really elongated? Dentomaxillofac Radiol 2004; 33(2):119-24.
• 23. Scaf G, Freitas DQ, Loffredo Lde C. Diagnostic reproducibility of the elongated styloid process. J Appl Oral Sci 2003; 11(2):120-4.
• 24. Nayak DR, Pujary K, Aggarwal M, Punnoose SE, Chaly VA. Role of three-dimensional computed tomography reconstruction in the management of elongated styloid process: a preliminary study. J Laryngol Otol 2007; 121(4):349-53.
• 25. Pan Y, Haber M, Gao J, Barnhart HX. A new permutation-based method for assessing agreement between two observers making replicated quantitative readings. Stat Med 2012; 31(20):2249-61.