Sunday, May 1, 2016

OrientDB incremental backup API


Since 2.2 version we support an incremental backup feature. So what perks do you get if you use incremental backup instead of classical backup:
  1. The database stores on disk only data which was changed after the previous backup.
  2. The database is not switched to read-only mode and users may continue to work with it.
Let's look how does it work inside of OrientDB.

Every time when we change data, together with user data we put some kind of timestamp which will show us when this change happened, we call it LSN. Despite normal timestamp, every time, when we make an update, LSN will continuously grow and can not be equal to previous LSN. In each snippet of incremental backup ( the file which is created during single incremental backup operation), we put maximum LSN of changes which we added into this snippet. So merge of different snippets and finding of changes happened after the previous backup is kinda simple. We iterate over all data in a sequential way and compare latest stored LSN and LSN of processed change if the last one is bigger we put it in the new backup file.

To prevent the situation when we lose changes happened during iteration over all database data, we log all operation since the start of an incremental backup process into database journal and at the end of the backup process we append a list of those operations to the backup file.

During restore process we import all data added into backup files and apply operations from database journal, so at the end of the process, we get the database in the state which it had at the end of last incremental backup.

To perform an incremental backup from Java you can call the method ODatabase#incrementalBackup(path-to-incremental-backup-directory) or from the console the same will look like backup database path-to-incremental-backup-directory -incremental.

To create database from incremental backup you can call from Java ODatabase#create(path-to-incremental-backup-directory) or from console: create database root root plocal graph -restore=path-to-incremental-backup-directory

Please note that incremental backup feature is provided only with the enterprise edition of OrientDB.