A Hop, A Skip, and A Jump - part the fifth

Yet another carrot for the stew, a paper from last year directly discussing RDBMS on PM. Of all the references so far in this series, this one is *a must read*. Really. Not just because it validates every notion flowing from my noggin. No, not at all.

In this tutorial, we provide an outline on how to build a new DBMS given the changes to hardware landscape due to NVM.

Yum. And it's from what was Carnegie Tech (which accepted me, but I ended up going elsewhere), one of the first Computer Science curriculums!

In the case of memory-oriented DBMSs (e.g., VoltDB, MemSQL), they contain components for overcoming the volatility of DRAM. Such components may be unnecessary in a system with byte-addressable NVM with fast random access.

Exactly why the announced kludges for *nix/Windows aren't the end. The OS/400 turns out to be the winner. In some form.

Peloton is an open-source HTAP DBMS that we are building that is designed from the groundup to use NVM. Our intended audience are developers, researchers, and practitioners with knowledge of DBMS internals. They do not need any in-depth background or experience with NVM.

Here we see another step closer to the OS/400 model:

To cope with these shorter NVM latencies, Microsoft and Linux are adding support for direct access storage (DAX) in Windows Server 2016 [48] and Linux 4.7 [1], respectively. With DAX, a DBMS directly allocates and uses NVM without an intervening filesystem. This requires only one copy between the file and the user buffers, thus improving the file I/O performance by an order of magnitude compared to block-oriented filesystems.
[my bold]

Recall an earlier part of this series' mention that transaction semantics will be different with PM? Make it so:

The write-ahead logging (WAL) protocol supports efficient transaction processing when memory is volatile and durable storage cannot support fast random writes [58, 39, 33]. But this assumption causes unnecessary performance degradations in a DBMS with NVM storage [14]. Consider a transaction that inserts a tuple into a table. A DBMS first records the tuple's contents in the log, and it later propagates the change to the database. With NVM, a DBMS can employ a logging protocol that avoids this unnecessary data duplication. The reason why NVM enables a better logging protocol than WAL is two-fold. The write throughput of NVM is more than an order of magnitude higher than that of an SSD or HDD. Further, the gap between sequential and random write throughput of NVM is smaller than that in SSD and HDD. Hence, a DBMS can flush changes directly to the database in NVM during regular transaction processing [15, 14, 12, 64, 40, 62, 80].

The Future is Now.