Evaluating Data Science Project Agility by Exploring Process Frameworks Used by Data Science Teams

Sucheta Lahiri, Jeffrey Saltz

Research output: Chapter in Book/Entry/PoemConference contribution

Abstract

The lack of effective team process is often noted as one of the key drivers of data science project inefficiencies and failures. To help address this challenge, this research reports on semi-structured interviews, across 16 organizations, which explored data science agile framework usage. While 62% of the organizations reported using an agile framework, none actually followed the Scrum Guide (or any other published framework), but rather, each organization had defined their own process that incorporated one or more aspects of Scrum. The other organizations used a proprietary/ad-hoc approach, often based on a proprietary data science life cycle. In short, while many data science teams are trying to be agile, they are adapting existing frameworks to work within a data science context. Future research could explore how data science teams can best achieve agility, perhaps via new agile frameworks that address the unique data science project management challenges.

Original languageEnglish (US)
Title of host publicationProceedings of the 56th Annual Hawaii International Conference on System Sciences, HICSS 2023
EditorsTung X. Bui
PublisherIEEE Computer Society
Pages6538-6547
Number of pages10
ISBN (Electronic)9780998133164
StatePublished - 2023
Event56th Annual Hawaii International Conference on System Sciences, HICSS 2023 - Virtual, Online, United States
Duration: Jan 3 2023Jan 6 2023

Publication series

NameProceedings of the Annual Hawaii International Conference on System Sciences
Volume2023-January
ISSN (Print)1530-1605

Conference

Conference56th Annual Hawaii International Conference on System Sciences, HICSS 2023
Country/TerritoryUnited States
CityVirtual, Online
Period1/3/231/6/23

Keywords

  • Agile
  • Data Science
  • Team Process

ASJC Scopus subject areas

  • General Engineering

Fingerprint

Dive into the research topics of 'Evaluating Data Science Project Agility by Exploring Process Frameworks Used by Data Science Teams'. Together they form a unique fingerprint.

Cite this