Skip to main content
Version: 0.5.2

Persistence

Skytable supports the persistent storage of data, which is an inherently obvious need for any database. In this document we explore how Skytable's persistence works.

Data directory structure#

Whenever you start Skytable, it will create a number of directories under a root 'data' directory. This is what the data directory structure looks like:

|__user-files (your other files)
|__data
|__data.bin
|__snapshots
|__<YYYYMMDD>-<HHMMSS>.snapshot (many)
|__remote
|___<snapshotname>.snapshot (many)

The data.bin file is primarily used for persistence while the snapshots folder contains automatically created snapshots and remotely created snapshots.

How Skytable stores data#

As soon as you start Skytable, it will look for a data.bin file in the data directory and then fall back to the data.bin file in the current directory for backwards compatibility. The database daemon will then attempt to migrate the older data into the structure noted above if required.

note

This backward compatibility will possibly be removed in future releases

If the data file is found and successfully read, then the previous data is moved into the in-memory table. If no file was found, then the database server will create one. Once you terminate the daemon, it will flush all data to this file. There are more features like BGSAVE and snapshots that can be configured and used by users.

Automated background saving#

Skytable supports automated saving of data in the background, via BGSAVE. BGSAVE, runs every two minutes to flush all the data in the in-memory table onto disk, unless customized through the configuration file. BGSAVE is enabled by default and we don't recommend disabling it until you're sure that your hardware will never fail; it is likely that this will never be the case.

Reliability of BGSAVE#

It can happen that BGSAVE fails to flush data to the disk due to some unforeseen system issues (such as lack of empty disk space, permission errors, etc). But if we continue to accept modifications to the data, it is a bad idea because this data may never get updated! This is why if BGSAVE fails, it will automatically poison the in-memory table preventing it from accepting any write/update operations. Poisioning is nothing but a global flag set in the database that indicates that the DB shouldn't accept any more updates/writes to data and in such a poisoned state, only reads are permitted.