Not all software engineers can become good data engineers

Jeffrey S. Saltz, Sibel Yilmazel, Ozgur Yilmazel

Research output: Chapter in Book/Entry/PoemConference contribution

7 Scopus citations

Abstract

The amount of data that businesses collect and analyze has been rapidly increasing, which has triggered an increase in big data teams. With the growth of both the number and size of big data teams, specialized roles are starting to be defined. One such role is the data engineer, who focuses on ensuring that the data is easily available for advanced analytics. Via a case study, this paper explores the role of the data engineer and the key characteristics that enable someone to be a good data engineer. The paper also explores if good software engineers could become good data engineers. Our findings show that the knowledge and skills required to be a data engineer are significantly different from those required to be a software engineer. Hence, not surprisingly, we found that that not all software engineers could become good data engineers.

Original languageEnglish (US)
Title of host publicationProceedings - 2016 IEEE International Conference on Big Data, Big Data 2016
EditorsRonay Ak, George Karypis, Yinglong Xia, Xiaohua Tony Hu, Philip S. Yu, James Joshi, Lyle Ungar, Ling Liu, Aki-Hiro Sato, Toyotaro Suzumura, Sudarsan Rachuri, Rama Govindaraju, Weijia Xu
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages2896-2901
Number of pages6
ISBN (Electronic)9781467390040
DOIs
StatePublished - 2016
Event4th IEEE International Conference on Big Data, Big Data 2016 - Washington, United States
Duration: Dec 5 2016Dec 8 2016

Publication series

NameProceedings - 2016 IEEE International Conference on Big Data, Big Data 2016

Other

Other4th IEEE International Conference on Big Data, Big Data 2016
Country/TerritoryUnited States
CityWashington
Period12/5/1612/8/16

Keywords

  • Big Data
  • Data Science
  • Process Methodology

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Information Systems
  • Hardware and Architecture

Fingerprint

Dive into the research topics of 'Not all software engineers can become good data engineers'. Together they form a unique fingerprint.

Cite this