Skytable supports the persistent storage of data, which is an inherently obvious need for any database. In this document we explore how Skytable's persistence works.
Whenever you start Skytable, it will create a number of directories under a root 'data' directory. This is what the data directory structure looks like:
|__user-files (your other files)|__data |__data.bin |__snapshots |__<YYYYMMDD>-<HHMMSS>.snapshot (many) |__remote |___<snapshotname>.snapshot (many)
data.bin file is primarily used for persistence while the snapshots folder contains automatically created
snapshots and remotely created snapshots.
As soon as you start Skytable, it will look for a
data.bin file in the data directory;
if the data file is found and successfully read, then the previous data is moved into the in-memory table. If no
file was found, then the database server will create one. Once you terminate the daemon, it will flush all data
to this file. All writes are
fsynced by the time they complete.
The server would attempt to do a final save operation before it terminates and if this fails, the server would enter into a retry loop. It will try the save operation after every 10 seconds. Exponential backoff was not chosen because it could increase to extremely large values that may hurt a sysadmin's time and productivity.
Skytable supports automated saving of data in the background, via
BGSAVE, runs every two minutes to flush all the data in the in-memory table onto disk, unless customized through the configuration file. BGSAVE is enabled by default and we don't recommend disabling it until you're sure that
your hardware will never fail; it is likely that this will never be the case. First BGSAVE will create a temporary
file and then flush the current in-memory table onto disk. It will then replace the old
data.bin file. The daemon automatically
fsyncs after every successful write (whether to the buffers or
to the actual disk).
It can happen that BGSAVE fails to flush data to the disk due to some unforeseen system issues (such as lack of empty disk space, permission errors, etc). But if we continue to accept modifications to the data, it is a bad idea because this data may never get updated! This is why if BGSAVE fails, it will automatically poison the in-memory table preventing it from accepting any write/update operations. Poisioning is nothing but a global flag set in the database that indicates that the DB shouldn't accept any more updates/writes to data and in such a poisoned state, only reads are permitted.
BGSAVE will automatically try to recover on every 120s (or whatever duration was set). If the problem was corrected (say it was a permissions issue), then the database server will automatically resume writes and unpoison the database.