G-Storm: GPU-enabled high-throughput online data processing in Storm

Zhenhua Chen; Jielong Xu; Jian Tang; Kevin Kwiat; Charles Kamhoua

doi:10.1109/BigData.2015.7363769

G-Storm: GPU-enabled high-throughput online data processing in Storm

Zhenhua Chen, Jielong Xu, Jian Tang, Kevin Kwiat, Charles Kamhoua

Department of Electrical Engineering & Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

17 Scopus citations

Abstract

The Single Instruction Multiple Data (SIMD) architecture of Graphic Processing Units (GPUs) makes them perfect for parallel processing of big data. In this paper, we present the design, implementation and evaluation of G-Storm, a GPU-enabled parallel system based on Storm, which harnesses the massively parallel computing power of GPUs for high-throughput online stream data processing. G-Storm has the following desirable features: 1) G-Storm is designed to be a general data processing platform as Storm, which can handle various applications and data types. 2) G-Storm exposes GPUs to Storm applications while preserving its easy-to-use programming model. 3) G-Storm achieves high-throughput and low-overhead data processing with GPUs. We implemented G-Storm based on Storm 0.9.2 and tested it using two different applications: continuous query and matrix multiplication. Extensive experimental results show that compared to Storm, G-Storm achieves over 7x improvement on throughput for continuous query, while maintaining reasonable average tuple processing time. It also leads to 2.3x throughput improvement for the matrix multiplication application.

Original language	English (US)
Title of host publication	Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	307-312
Number of pages	6
ISBN (Print)	9781479999255
DOIs	https://doi.org/10.1109/BigData.2015.7363769
State	Published - Dec 22 2015
Event	3rd IEEE International Conference on Big Data, IEEE Big Data 2015 - Santa Clara, United States Duration: Oct 29 2015 → Nov 1 2015

Other

Other	3rd IEEE International Conference on Big Data, IEEE Big Data 2015
Country	United States
City	Santa Clara
Period	10/29/15 → 11/1/15

ASJC Scopus subject areas

Computer Networks and Communications
Computer Science Applications
Information Systems
Software

Access to Document

10.1109/BigData.2015.7363769

Fingerprint Dive into the research topics of 'G-Storm: GPU-enabled high-throughput online data processing in Storm'. Together they form a unique fingerprint.

Cite this

G-Storm : GPU-enabled high-throughput online data processing in Storm. / Chen, Zhenhua; Xu, Jielong; Tang, Jian; Kwiat, Kevin; Kamhoua, Charles.

Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015. Institute of Electrical and Electronics Engineers Inc., 2015. p. 307-312 7363769.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Chen, Z, Xu, J, Tang, J, Kwiat, K & Kamhoua, C 2015, G-Storm: GPU-enabled high-throughput online data processing in Storm. in Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015., 7363769, Institute of Electrical and Electronics Engineers Inc., pp. 307-312, 3rd IEEE International Conference on Big Data, IEEE Big Data 2015, Santa Clara, United States, 10/29/15. https://doi.org/10.1109/BigData.2015.7363769

@inproceedings{fd9fba9383ea431c90fd9b75c3e9d133,

title = "G-Storm: GPU-enabled high-throughput online data processing in Storm",

abstract = "The Single Instruction Multiple Data (SIMD) architecture of Graphic Processing Units (GPUs) makes them perfect for parallel processing of big data. In this paper, we present the design, implementation and evaluation of G-Storm, a GPU-enabled parallel system based on Storm, which harnesses the massively parallel computing power of GPUs for high-throughput online stream data processing. G-Storm has the following desirable features: 1) G-Storm is designed to be a general data processing platform as Storm, which can handle various applications and data types. 2) G-Storm exposes GPUs to Storm applications while preserving its easy-to-use programming model. 3) G-Storm achieves high-throughput and low-overhead data processing with GPUs. We implemented G-Storm based on Storm 0.9.2 and tested it using two different applications: continuous query and matrix multiplication. Extensive experimental results show that compared to Storm, G-Storm achieves over 7x improvement on throughput for continuous query, while maintaining reasonable average tuple processing time. It also leads to 2.3x throughput improvement for the matrix multiplication application.",

author = "Zhenhua Chen and Jielong Xu and Jian Tang and Kevin Kwiat and Charles Kamhoua",

year = "2015",

month = dec,

day = "22",

doi = "10.1109/BigData.2015.7363769",

language = "English (US)",

isbn = "9781479999255",

pages = "307--312",

booktitle = "Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

note = "3rd IEEE International Conference on Big Data, IEEE Big Data 2015 ; Conference date: 29-10-2015 Through 01-11-2015",

}

TY - GEN

T1 - G-Storm

T2 - 3rd IEEE International Conference on Big Data, IEEE Big Data 2015

AU - Chen, Zhenhua

AU - Xu, Jielong

AU - Tang, Jian

AU - Kwiat, Kevin

AU - Kamhoua, Charles

PY - 2015/12/22

Y1 - 2015/12/22

N2 - The Single Instruction Multiple Data (SIMD) architecture of Graphic Processing Units (GPUs) makes them perfect for parallel processing of big data. In this paper, we present the design, implementation and evaluation of G-Storm, a GPU-enabled parallel system based on Storm, which harnesses the massively parallel computing power of GPUs for high-throughput online stream data processing. G-Storm has the following desirable features: 1) G-Storm is designed to be a general data processing platform as Storm, which can handle various applications and data types. 2) G-Storm exposes GPUs to Storm applications while preserving its easy-to-use programming model. 3) G-Storm achieves high-throughput and low-overhead data processing with GPUs. We implemented G-Storm based on Storm 0.9.2 and tested it using two different applications: continuous query and matrix multiplication. Extensive experimental results show that compared to Storm, G-Storm achieves over 7x improvement on throughput for continuous query, while maintaining reasonable average tuple processing time. It also leads to 2.3x throughput improvement for the matrix multiplication application.

AB - The Single Instruction Multiple Data (SIMD) architecture of Graphic Processing Units (GPUs) makes them perfect for parallel processing of big data. In this paper, we present the design, implementation and evaluation of G-Storm, a GPU-enabled parallel system based on Storm, which harnesses the massively parallel computing power of GPUs for high-throughput online stream data processing. G-Storm has the following desirable features: 1) G-Storm is designed to be a general data processing platform as Storm, which can handle various applications and data types. 2) G-Storm exposes GPUs to Storm applications while preserving its easy-to-use programming model. 3) G-Storm achieves high-throughput and low-overhead data processing with GPUs. We implemented G-Storm based on Storm 0.9.2 and tested it using two different applications: continuous query and matrix multiplication. Extensive experimental results show that compared to Storm, G-Storm achieves over 7x improvement on throughput for continuous query, while maintaining reasonable average tuple processing time. It also leads to 2.3x throughput improvement for the matrix multiplication application.

UR - http://www.scopus.com/inward/record.url?scp=84963761145&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84963761145&partnerID=8YFLogxK

U2 - 10.1109/BigData.2015.7363769

DO - 10.1109/BigData.2015.7363769

M3 - Conference contribution

AN - SCOPUS:84963761145

SN - 9781479999255

SP - 307

EP - 312

BT - Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 29 October 2015 through 1 November 2015

ER -