Delta Lake

Open storage framework that brings reliability and performance to data lakes.

MCPCLI

About Delta Lake

Open storage framework that brings reliability and performance to data lakes. Explore how Delta Lake integrates with the agentic data stack ecosystem and supports autonomous data operations.

Key Features

  • ACID transactions with optimistic concurrency control on object storage
  • Schema enforcement on write and schema evolution support
  • Time travel and data versioning for auditing, rollbacks, and reproducibility
  • Unified batch and streaming processing on the same tables
  • MERGE/UPDATE/DELETE operations (upserts) for incremental data management
  • Broad engine support (Spark, Flink, Trino, Presto, Hive, Athena, Snowflake, BigQuery)
  • Small file compaction and Z-Order indexing for query performance
  • Change Data Feed for capturing row-level changes between table versions

Agent Integration

CLI — spark-sql / deltalake (Python)

$ pip install deltalake
CLI Documentation

External Links

Databricks Labs MCP Server

Official Databricks MCP server exposing Unity Catalog including Delta tables, vector search, and Genie

delta-rs (Rust/Python SDK)

Native Rust library with Python bindings for Delta Lake — no Spark required, ideal for lightweight agent access

Python API Reference

Complete Python API docs for the deltalake package (read, write, optimize, vacuum, time-travel)

Databricks MCP Documentation

Official Databricks docs on hosting and using MCP servers with Unity Catalog permissions

Awesome Lakehouse Guide

Curated repo covering open table formats and lakehouse architecture

← Back to Lake Format