Versioning and Reproducibility

Learn how to implement versioning and ensure reproducibility in LanceDB. Includes version control, data snapshots, and audit trails.

This page shows the core table-versioning APIs used in the code snippets for Python, TypeScript, and Rust. Each operation below maps directly to methods shown in the examples.

Basic Versioning Example

Let’s create a table with sample data to demonstrate LanceDB’s versioning capabilities:

Set Up the Table

First, let’s create a table with some sample data:

Check Initial Version

After creating the table, let’s check the initial version information:

Modify Data

When you modify data through operations like update or delete, LanceDB automatically creates new versions.

Update Existing Data

Let’s update some existing records to see versioning in action:

Add New Data

Now let’s add more records to the table:

Check Version Changes

Let’s see how the versions have changed after our modifications:

Rollback to Previous Versions

LanceDB supports fast rollbacks to any previous version without data duplication.

View All Versions

First, let’s see all the versions we’ve created:

Restore a Version Snapshot

Now let’s restore a captured version snapshot:

Tag-Based Versioning

Numeric table versions like v3 or v17 are precise but hard to remember. Tags let you attach human-readable labels (e.g., "prod", "baseline", "q3-evaluation") to specific versions and check those out by name. They are conceptually similar to git tags, and unlike numeric versions, tagged versions are preserved when old versions are pruned (see the cleanup note at the bottom of this page).

The tags API supports the standard CRUD operations — create, list, update, delete — plus checking out by tag name.

Deleting a tag only removes the label, not the version it points to. After deletion, the underlying table version becomes eligible for cleanup again.

Delete Data From the Table

Let’s demonstrate how deletions also create new versions:

Go Back to Latest Version

First, let’s return to the latest version:

Delete Data

Now let’s delete some data to see how it affects versioning:

Version History and Operations

On a fresh table, the snippets in this guide produce this version sequence:

1. v1: create table (create_table / createTable / create_table)
2. v2: update rows (update)
3. v3: add rows (add)
4. v4: restore snapshot (restore) from version_after_mod/versionAfterMod
5. v5: delete rows (delete)

Read-only and checkout operations shown here (list_versions/listVersions, version, checkout, checkout_latest/checkoutLatest) do not create new versions.

System Operations

System operations like optimize(), index updates, and table compaction also increment table version numbers. In LanceDB OSS and Enterprise, optimize() can prune older versions based on its retention setting (cleanup_older_than, 7 days by default), which is when old-version files are removed and disk space is reclaimed.

Tagged versions are exempt from cleanup. A version with a tag pointing at it is retained regardless of age, and its files are not removed by optimize(). To make a tagged version eligible for pruning, delete the tag first.