The effect of anchors on reliability of endoscopic tremor ratings

Soren Lowell, Richard T. Kelley, Lauren Busekroos, Ramani Voleti, Carly J. Hosbach-Cannon, Raymond H. Colton, Dragos Mihaila

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Objectives/Hypothesis: The purpose of this study was to determine the effects of anchors and training on intrarater and inter-rater reliability for visual-perceptual, endoscopic tremor ratings. Study Design: Prospective cohort study. Methods: Nasoendoscopy recordings of 10 participants with a diagnosis of essential voice tremor were evaluated by five voice specialists using the Vocal Tremor Scoring System. Ratings were performed before, immediately after, and 4 weeks after implementation of a training program with anchor stimuli. Immediate and long-term post-training ratings were performed with simultaneous use of anchor samples for each rating. Results: Intrarater reliability showed significant improvement from pretraining to immediate and long-term post-training. Mean correlation coefficients (Spearman's rho) increased from 0.71 at pretraining to 0.84 and 0.90 at immediate and long-term post-training, respectively. Inter-rater reliability was not affected by training with anchors, with mean correlation coefficients ranging from 0.62 at pretraining to 0.58 and 0.64 at immediate and long-term post-training, respectively. Conclusions: Consistent, reproducible ratings are critical for the interpretation and comparison of endoscopic tremor data. Reliability findings from this study indicate that the use of anchor samples as referents for making ordinal judgments about the severity of tremor in oropharyngeal and laryngeal regions was helpful for improving internal standards and consistency but less useful for calibrating across different raters.

Original languageEnglish (US)
JournalLaryngoscope
DOIs
StateAccepted/In press - 2016

Fingerprint

Tremor
Essential Tremor
Reproducibility of Results
Cohort Studies
Prospective Studies
Education

Keywords

  • Endoscopic ratings
  • Laryngoscopy
  • Reliability
  • Tremor
  • Voice

ASJC Scopus subject areas

  • Otorhinolaryngology

Cite this

Lowell, S., Kelley, R. T., Busekroos, L., Voleti, R., Hosbach-Cannon, C. J., Colton, R. H., & Mihaila, D. (Accepted/In press). The effect of anchors on reliability of endoscopic tremor ratings. Laryngoscope. https://doi.org/10.1002/lary.26034

The effect of anchors on reliability of endoscopic tremor ratings. / Lowell, Soren; Kelley, Richard T.; Busekroos, Lauren; Voleti, Ramani; Hosbach-Cannon, Carly J.; Colton, Raymond H.; Mihaila, Dragos.

In: Laryngoscope, 2016.

Research output: Contribution to journalArticle

Lowell, S, Kelley, RT, Busekroos, L, Voleti, R, Hosbach-Cannon, CJ, Colton, RH & Mihaila, D 2016, 'The effect of anchors on reliability of endoscopic tremor ratings', Laryngoscope. https://doi.org/10.1002/lary.26034
Lowell S, Kelley RT, Busekroos L, Voleti R, Hosbach-Cannon CJ, Colton RH et al. The effect of anchors on reliability of endoscopic tremor ratings. Laryngoscope. 2016. https://doi.org/10.1002/lary.26034
Lowell, Soren ; Kelley, Richard T. ; Busekroos, Lauren ; Voleti, Ramani ; Hosbach-Cannon, Carly J. ; Colton, Raymond H. ; Mihaila, Dragos. / The effect of anchors on reliability of endoscopic tremor ratings. In: Laryngoscope. 2016.
@article{b2f5c1329adb4708af5d08c4e9bf9ce0,
title = "The effect of anchors on reliability of endoscopic tremor ratings",
abstract = "Objectives/Hypothesis: The purpose of this study was to determine the effects of anchors and training on intrarater and inter-rater reliability for visual-perceptual, endoscopic tremor ratings. Study Design: Prospective cohort study. Methods: Nasoendoscopy recordings of 10 participants with a diagnosis of essential voice tremor were evaluated by five voice specialists using the Vocal Tremor Scoring System. Ratings were performed before, immediately after, and 4 weeks after implementation of a training program with anchor stimuli. Immediate and long-term post-training ratings were performed with simultaneous use of anchor samples for each rating. Results: Intrarater reliability showed significant improvement from pretraining to immediate and long-term post-training. Mean correlation coefficients (Spearman's rho) increased from 0.71 at pretraining to 0.84 and 0.90 at immediate and long-term post-training, respectively. Inter-rater reliability was not affected by training with anchors, with mean correlation coefficients ranging from 0.62 at pretraining to 0.58 and 0.64 at immediate and long-term post-training, respectively. Conclusions: Consistent, reproducible ratings are critical for the interpretation and comparison of endoscopic tremor data. Reliability findings from this study indicate that the use of anchor samples as referents for making ordinal judgments about the severity of tremor in oropharyngeal and laryngeal regions was helpful for improving internal standards and consistency but less useful for calibrating across different raters.",
keywords = "Endoscopic ratings, Laryngoscopy, Reliability, Tremor, Voice",
author = "Soren Lowell and Kelley, {Richard T.} and Lauren Busekroos and Ramani Voleti and Hosbach-Cannon, {Carly J.} and Colton, {Raymond H.} and Dragos Mihaila",
year = "2016",
doi = "10.1002/lary.26034",
language = "English (US)",
journal = "Laryngoscope",
issn = "0023-852X",
publisher = "John Wiley and Sons Inc.",

}

TY - JOUR

T1 - The effect of anchors on reliability of endoscopic tremor ratings

AU - Lowell, Soren

AU - Kelley, Richard T.

AU - Busekroos, Lauren

AU - Voleti, Ramani

AU - Hosbach-Cannon, Carly J.

AU - Colton, Raymond H.

AU - Mihaila, Dragos

PY - 2016

Y1 - 2016

N2 - Objectives/Hypothesis: The purpose of this study was to determine the effects of anchors and training on intrarater and inter-rater reliability for visual-perceptual, endoscopic tremor ratings. Study Design: Prospective cohort study. Methods: Nasoendoscopy recordings of 10 participants with a diagnosis of essential voice tremor were evaluated by five voice specialists using the Vocal Tremor Scoring System. Ratings were performed before, immediately after, and 4 weeks after implementation of a training program with anchor stimuli. Immediate and long-term post-training ratings were performed with simultaneous use of anchor samples for each rating. Results: Intrarater reliability showed significant improvement from pretraining to immediate and long-term post-training. Mean correlation coefficients (Spearman's rho) increased from 0.71 at pretraining to 0.84 and 0.90 at immediate and long-term post-training, respectively. Inter-rater reliability was not affected by training with anchors, with mean correlation coefficients ranging from 0.62 at pretraining to 0.58 and 0.64 at immediate and long-term post-training, respectively. Conclusions: Consistent, reproducible ratings are critical for the interpretation and comparison of endoscopic tremor data. Reliability findings from this study indicate that the use of anchor samples as referents for making ordinal judgments about the severity of tremor in oropharyngeal and laryngeal regions was helpful for improving internal standards and consistency but less useful for calibrating across different raters.

AB - Objectives/Hypothesis: The purpose of this study was to determine the effects of anchors and training on intrarater and inter-rater reliability for visual-perceptual, endoscopic tremor ratings. Study Design: Prospective cohort study. Methods: Nasoendoscopy recordings of 10 participants with a diagnosis of essential voice tremor were evaluated by five voice specialists using the Vocal Tremor Scoring System. Ratings were performed before, immediately after, and 4 weeks after implementation of a training program with anchor stimuli. Immediate and long-term post-training ratings were performed with simultaneous use of anchor samples for each rating. Results: Intrarater reliability showed significant improvement from pretraining to immediate and long-term post-training. Mean correlation coefficients (Spearman's rho) increased from 0.71 at pretraining to 0.84 and 0.90 at immediate and long-term post-training, respectively. Inter-rater reliability was not affected by training with anchors, with mean correlation coefficients ranging from 0.62 at pretraining to 0.58 and 0.64 at immediate and long-term post-training, respectively. Conclusions: Consistent, reproducible ratings are critical for the interpretation and comparison of endoscopic tremor data. Reliability findings from this study indicate that the use of anchor samples as referents for making ordinal judgments about the severity of tremor in oropharyngeal and laryngeal regions was helpful for improving internal standards and consistency but less useful for calibrating across different raters.

KW - Endoscopic ratings

KW - Laryngoscopy

KW - Reliability

KW - Tremor

KW - Voice

UR - http://www.scopus.com/inward/record.url?scp=84963791996&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84963791996&partnerID=8YFLogxK

U2 - 10.1002/lary.26034

DO - 10.1002/lary.26034

M3 - Article

JO - Laryngoscope

JF - Laryngoscope

SN - 0023-852X

ER -