Deferred lightweight indexing for log-structured key-value stores

Yuzhe Tang, Arun Iyengar, Wei Tan, Liana Fong, Ling Liu, Balaji Palanisamy

Research output: Chapter in Book/Report/Conference proceedingConference contribution

10 Scopus citations

Abstract

The recent shift towards write-intensive workload on bigdata (e.g., financial trading, social user-generated data streams)has pushed the proliferation of log-structured key-value stores, represented by Google's BigTable [1], Apache HBase [2] andCassandra [3]. While providing key-based data access with aPut/Get interface, these key-value stores do not support value-based access methods, which significantly limits their applicability in modern web and database applications. In this paper, we present DELI, a DEferred Lightweight Indexing scheme on the log-structured key-value stores. To index intensively updated bigdata in real time, DELI aims at making the index maintenance as lightweight as possible. The key idea is to apply an append-only design for online index maintenance and to collect index garbage at carefully chosen time. DELI optimizes the performance of index garbage collection through tightly coupling its execution with a native routine process called compaction. The DELI'ssystem design is fault-tolerant and generic (to most key-valuestores), we implemented a prototype of DELI based on HBasewithout internal code modification. Our experiments show that the DELI offers significant performance advantage for the write-intensive index maintenance.

Original languageEnglish (US)
Title of host publicationProceedings - 2015 IEEE/ACM 15th International Symposium on Cluster, Cloud, and Grid Computing, CCGrid 2015
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages11-20
Number of pages10
ISBN (Electronic)9781479980062
DOIs
StatePublished - Jul 7 2015
Event15th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing, CCGrid 2015 - Shenzhen, China
Duration: May 4 2015May 7 2015

Publication series

NameProceedings - 2015 IEEE/ACM 15th International Symposium on Cluster, Cloud, and Grid Computing, CCGrid 2015

Other

Other15th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing, CCGrid 2015
CountryChina
CityShenzhen
Period5/4/155/7/15

Keywords

  • Indexing
  • Key-value stores
  • Log-structured
  • NoSQL
  • Secondary index

ASJC Scopus subject areas

  • Computer Science (miscellaneous)
  • Computer Networks and Communications
  • Software

Fingerprint Dive into the research topics of 'Deferred lightweight indexing for log-structured key-value stores'. Together they form a unique fingerprint.

  • Cite this

    Tang, Y., Iyengar, A., Tan, W., Fong, L., Liu, L., & Palanisamy, B. (2015). Deferred lightweight indexing for log-structured key-value stores. In Proceedings - 2015 IEEE/ACM 15th International Symposium on Cluster, Cloud, and Grid Computing, CCGrid 2015 (pp. 11-20). [7152467] (Proceedings - 2015 IEEE/ACM 15th International Symposium on Cluster, Cloud, and Grid Computing, CCGrid 2015). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CCGrid.2015.150