28 February 2020

One Classy Dame - part the third

Found another paper, only two pages, that makes the case for non-filesystem storage with SCM.
The existing OS structure of file systems as a kernel-level service may no longer be necessary with storage class memory, and causes unneeded complexity and lower performance.

The paper is undated (another paper by these authors notes it's from 2011), so the idea of using SCM the way I want it used has been living for some time. As mentioned in these missives, with regard to RDBMS use of storage, I've often said that industrial strength RDBMS have all provided access to 'raw' devices (in *nix terms), although most vendors have been discouraging that approach for some time.
In addition, some applications, such as databases, bypass the file system but still rely on the lower levels of the storage stack.

The latter paper is more complete, in that it documents a PoC implementation on linux. And is somewhat different from a single-level store on SCM.

It seems that there are a few identifiable issues:
1 - a mix of DRAM and SCM makes the most sense, since code need not be persistent and need not be hobbled by slower access
2 - kernel code that knows how to handle access to both types of 'RAM'
3 - application code that knows how to write transactions to SCM, rather than files
4 - limitation of SCM by the amount of memory available on a X86 motherboard; currently 256GB, modulo https://www.quora.com/What-is-the-most-RAM-a-computer-has-ever-hadbespoke> boards not in general circulation.
5 - unlike disk based storage, expansion has a (nearly?) hard limit, so RDBMS applications would have to result to evicting aged data to SSD/HDD on a regular basis

Here is a Master's paper from 2019, so rather current, on transaction semantics. The paper isn't geared toward RDBMS or ACID, rather to general data consistency. A bit dense, but worth the effort.

No comments: