Development and Psychometric Testing of the Rate and Accumulation Concept Inventory

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Background: Because a fundamental understanding of rate and accumulation principles is important for engineering students across all subdisciplines, a method is needed to assess their understanding of these principles. Purpose/Hypothesis: This article discusses the development of the Rate and Accumulation Concept Inventory (RACI) instrument and provides an analysis of its validity and reliability. The RACI is designed to test students’ (1) overall mastery of rate and accumulation concepts and (2) mastery of specific rate and accumulation concepts (e.g., mathematics, mass flow, and heat flow). Design/Methods: Exploratory findings of students’ conceptual understanding prompted the development of a preliminary RACI instrument. Two pilot studies were conducted, with adjustments made to the instrument between each. Data from the second pilot were used to assess evidence of validity and reliability through classical test theory, item response theory, and structural equation modeling. Results: Validity and reliability evidence indicates that the RACI can be used to measure students’ overall understanding of the concepts identified, though several items require revision or removal. Issues of potential construct underrepresentation were found in two of the conceptual categories. Conclusions: The evidence of reliability and validity indicates the appropriateness of the RACI for assessing engineering students’ overall understanding of rate and accumulation principles. Additional research is needed to further enhance the validity and reliability of the RACI as a diagnostic tool, particularly in measuring students’ understanding of rate and accumulation concepts in particular physical contexts.

Original languageEnglish (US)
Pages (from-to)491-520
Number of pages30
JournalJournal of Engineering Education
Volume107
Issue number3
DOIs
StatePublished - Jul 1 2018

Fingerprint

psychometrics
Students
Testing
student
engineering
evidence
Heat transfer
test theory
heat
diagnostic
mathematics

Keywords

  • concept inventories
  • mathematics
  • misconceptions
  • physical science
  • psychometric analysis

ASJC Scopus subject areas

  • Education
  • Engineering(all)

Cite this

Development and Psychometric Testing of the Rate and Accumulation Concept Inventory. / Flynn, Carli D.; Davidson, Cliff Ian; Dotger, Sharon.

In: Journal of Engineering Education, Vol. 107, No. 3, 01.07.2018, p. 491-520.

Research output: Contribution to journalArticle

@article{2b3025d921ba4028a608cfbd1fc44635,
title = "Development and Psychometric Testing of the Rate and Accumulation Concept Inventory",
abstract = "Background: Because a fundamental understanding of rate and accumulation principles is important for engineering students across all subdisciplines, a method is needed to assess their understanding of these principles. Purpose/Hypothesis: This article discusses the development of the Rate and Accumulation Concept Inventory (RACI) instrument and provides an analysis of its validity and reliability. The RACI is designed to test students’ (1) overall mastery of rate and accumulation concepts and (2) mastery of specific rate and accumulation concepts (e.g., mathematics, mass flow, and heat flow). Design/Methods: Exploratory findings of students’ conceptual understanding prompted the development of a preliminary RACI instrument. Two pilot studies were conducted, with adjustments made to the instrument between each. Data from the second pilot were used to assess evidence of validity and reliability through classical test theory, item response theory, and structural equation modeling. Results: Validity and reliability evidence indicates that the RACI can be used to measure students’ overall understanding of the concepts identified, though several items require revision or removal. Issues of potential construct underrepresentation were found in two of the conceptual categories. Conclusions: The evidence of reliability and validity indicates the appropriateness of the RACI for assessing engineering students’ overall understanding of rate and accumulation principles. Additional research is needed to further enhance the validity and reliability of the RACI as a diagnostic tool, particularly in measuring students’ understanding of rate and accumulation concepts in particular physical contexts.",
keywords = "concept inventories, mathematics, misconceptions, physical science, psychometric analysis",
author = "Flynn, {Carli D.} and Davidson, {Cliff Ian} and Sharon Dotger",
year = "2018",
month = "7",
day = "1",
doi = "10.1002/jee.20226",
language = "English (US)",
volume = "107",
pages = "491--520",
journal = "Journal of Engineering Education",
issn = "1069-4730",
publisher = "American Society for Engineering Education",
number = "3",

}

TY - JOUR

T1 - Development and Psychometric Testing of the Rate and Accumulation Concept Inventory

AU - Flynn, Carli D.

AU - Davidson, Cliff Ian

AU - Dotger, Sharon

PY - 2018/7/1

Y1 - 2018/7/1

N2 - Background: Because a fundamental understanding of rate and accumulation principles is important for engineering students across all subdisciplines, a method is needed to assess their understanding of these principles. Purpose/Hypothesis: This article discusses the development of the Rate and Accumulation Concept Inventory (RACI) instrument and provides an analysis of its validity and reliability. The RACI is designed to test students’ (1) overall mastery of rate and accumulation concepts and (2) mastery of specific rate and accumulation concepts (e.g., mathematics, mass flow, and heat flow). Design/Methods: Exploratory findings of students’ conceptual understanding prompted the development of a preliminary RACI instrument. Two pilot studies were conducted, with adjustments made to the instrument between each. Data from the second pilot were used to assess evidence of validity and reliability through classical test theory, item response theory, and structural equation modeling. Results: Validity and reliability evidence indicates that the RACI can be used to measure students’ overall understanding of the concepts identified, though several items require revision or removal. Issues of potential construct underrepresentation were found in two of the conceptual categories. Conclusions: The evidence of reliability and validity indicates the appropriateness of the RACI for assessing engineering students’ overall understanding of rate and accumulation principles. Additional research is needed to further enhance the validity and reliability of the RACI as a diagnostic tool, particularly in measuring students’ understanding of rate and accumulation concepts in particular physical contexts.

AB - Background: Because a fundamental understanding of rate and accumulation principles is important for engineering students across all subdisciplines, a method is needed to assess their understanding of these principles. Purpose/Hypothesis: This article discusses the development of the Rate and Accumulation Concept Inventory (RACI) instrument and provides an analysis of its validity and reliability. The RACI is designed to test students’ (1) overall mastery of rate and accumulation concepts and (2) mastery of specific rate and accumulation concepts (e.g., mathematics, mass flow, and heat flow). Design/Methods: Exploratory findings of students’ conceptual understanding prompted the development of a preliminary RACI instrument. Two pilot studies were conducted, with adjustments made to the instrument between each. Data from the second pilot were used to assess evidence of validity and reliability through classical test theory, item response theory, and structural equation modeling. Results: Validity and reliability evidence indicates that the RACI can be used to measure students’ overall understanding of the concepts identified, though several items require revision or removal. Issues of potential construct underrepresentation were found in two of the conceptual categories. Conclusions: The evidence of reliability and validity indicates the appropriateness of the RACI for assessing engineering students’ overall understanding of rate and accumulation principles. Additional research is needed to further enhance the validity and reliability of the RACI as a diagnostic tool, particularly in measuring students’ understanding of rate and accumulation concepts in particular physical contexts.

KW - concept inventories

KW - mathematics

KW - misconceptions

KW - physical science

KW - psychometric analysis

UR - http://www.scopus.com/inward/record.url?scp=85054178632&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85054178632&partnerID=8YFLogxK

U2 - 10.1002/jee.20226

DO - 10.1002/jee.20226

M3 - Article

VL - 107

SP - 491

EP - 520

JO - Journal of Engineering Education

JF - Journal of Engineering Education

SN - 1069-4730

IS - 3

ER -