High-Performance Concurrency Control Mechanisms for Main-Memory Databases

📝 Abstract

A database system optimized for in-memory storage can support much higher transaction rates than current systems. However, standard concurrency control methods used today do not scale to the high transaction rates achievable by such systems. In this paper we introduce two efficient concurrency control methods specifically designed for main-memory databases. Both use multiversioning to isolate read-only transactions from updates but differ in how atomicity is ensured: one is optimistic and one is pessimistic. To avoid expensive context switching, transactions never block during normal processing but they may have to wait before commit to ensure correct serialization ordering. We also implemented a main-memory optimized version of single-version locking. Experimental results show that while single-version locking works well when transactions are short and contention is low performance degrades under more demanding conditions. The multiversion schemes have higher overhead but are much less sensitive to hotspots and the presence of long-running transactions.

💡 Analysis

A database system optimized for in-memory storage can support much higher transaction rates than current systems. However, standard concurrency control methods used today do not scale to the high transaction rates achievable by such systems. In this paper we introduce two efficient concurrency control methods specifically designed for main-memory databases. Both use multiversioning to isolate read-only transactions from updates but differ in how atomicity is ensured: one is optimistic and one is pessimistic. To avoid expensive context switching, transactions never block during normal processing but they may have to wait before commit to ensure correct serialization ordering. We also implemented a main-memory optimized version of single-version locking. Experimental results show that while single-version locking works well when transactions are short and contention is low performance degrades under more demanding conditions. The multiversion schemes have higher overhead but are much less sensitive to hotspots and the presence of long-running transactions.

📄 Content

High-Performance Concurrency Control Mechanisms for Main-Memory Databases Per-Åke Larson1, Spyros Blanas2, Cristian Diaconu1,
Craig Freedman1, Jignesh M. Patel2, Mike Zwilling1
Microsoft1, University of Wisconsin – Madison2 {palarson, cdiaconu, craigfr,mikezw}@microsoft.com, {sblanas, jignesh}@cs.wisc.edu

ABSTRACT A database system optimized for in-memory storage can support much higher transaction rates than current systems. However, standard concurrency control methods used today do not scale to the high transaction rates achievable by such systems. In this pa- per we introduce two efficient concurrency control methods spe- cifically designed for main-memory databases. Both use multiver- sioning to isolate read-only transactions from updates but differ in how atomicity is ensured: one is optimistic and one is pessimistic. To avoid expensive context switching, transactions never block during normal processing but they may have to wait before com- mit to ensure correct serialization ordering. We also implemented a main-memory optimized version of single-version locking. Ex- perimental results show that while single-version locking works well when transactions are short and contention is low perfor- mance degrades under more demanding conditions. The multiver- sion schemes have higher overhead but are much less sensitive to hotspots and the presence of long-running transactions.

1. INTRODUCTION Current database management systems were designed assuming that data would reside on disk. However, memory prices continue to decline; over the last 30 years they have been dropping by a factor of 10 every 5 years. The latest Oracle Exadata X2-8 system ships with 2TB of main memory and it is likely that we will see commodity servers with multiple terabytes of main memory with- in a few years. On such systems the majority of OLTP databases will fit entirely in memory, and even the largest OLTP databases will keep the active working set in memory, leaving only cold, infrequently accessed data on external storage.
   A DBMS optimized for in-memory storage and running on a many-core processor can support very high transaction rates.
   Efficiently ensuring isolation between concurrently executing transactions becomes challenging in such an environment. Current DBMSs typically rely on locking but in a traditional implementa- tion with a separate lock manager the lock manager becomes a bottleneck at high transaction rates as shown in experiments by Johnson et al [15]. Long read-only transactions are also problem- atic as readers may block writers.
   This paper investigates high-performance concurrency control mechanisms for OLTP workloads in main-memory databases. We found that traditional single-version locking is “fragile”. It works well when all transactions are short and there are no hotspots but performance degrades rapidly under high contention or when the workload includes even a single long transaction.
   Decades of research has shown that multiversion concurrency control (MVCC) methods are more robust and perform well for a broad range of workloads. This led us to investigate how to con- struct MVCC mechanisms optimized for main memory settings. We designed two MVCC mechanisms: the first is optimistic and relies on validation, while the second one is pessimistic and relies on locking. The two schemes are mutually compatible in the sense that optimistic and pessimistic transactions can be mixed and access the same database concurrently. We systematically ex- plored and evaluated these methods, providing an extensive ex- perimental evaluation of the pros and cons of each approach. The experiments confirmed that MVCC methods are indeed more robust than single-version locking.
   This paper makes three contributions. First, we propose an opti- mistic MVCC method designed specifically for memory resident data. Second, we redesign two locking-based concurrency control methods, one single-version and one multiversion, to fully exploit a main-memory setting. Third, we evaluate the effectiveness of these three different concurrency control methods for different workloads. The insights from this study are directly applicable to high-performance main memory databases: single-version locking performs well only when transactions are short and contention is low; higher contention or workloads including some long transac- tions favor the multiversion methods; and the optimistic method performs better than the pessimistic method. The rest of the paper is organized as follows. Section 2 covers preliminaries of multiversioning and describes how version visi- bility and updatability are determined based on version timestamps. The optimistic scheme and the pessimistic scheme are described in Section 3 and Section 4, respectively. Section 5 reports performance results. Related work is discussed in Section 6, and Section 7 offers concluding remarks. Proofs of correctness

This content is AI-processed based on ArXiv data.