Delta Lake Blogs

RSS Feed

How to drop columns from a Delta Lake table

By Matthew Powers

This post shows you two ways to drop columns from Delta Lake tables.

Apache Flink Source Connector for Delta Lake tables

By Krzysztof Chmielewski , Scott Sandre , Denny Lee

We are excited to announce the release of Delta Connectors 0.5.0, which introduces the new Flink/Delta Source Connector on Apache Flink™ 1.13 that can read directly from Delta tables using Flink’s DataStream API.

Delta 2.0 - The Foundation of your Data Lakehouse is Open

By Tathagata Das , Denny Lee

We are happy to announce the release of the Delta Lake 2.0 on Apache Spark™ 3.2! The significance of Delta Lake 2.0 is not just a number - though it is timed quite nicely with Delta Lake’s 3rd birthday. It reiterates our collective commitment to the open-sourcing of Delta Lake, as announced by Michael Armbrust’s Day 1 keynote at Data + AI Summit 2022.

Multi-cluster writes to Delta Lake Storage in S3

By Scott Sandre , Denny Lee , Mariusz Kryński

While Delta Lake has supported concurrent reads from multiple clusters since its inception, there were limitations for multi-cluster writes specifically to Amazon S3. Note, this was not a limitation for Azure ADLSgen2 nor Google GCS, as S3 currently lacks “put-If-Absent” consistency guarantees. Thus, to guarantee ACID transactions on S3, one would need to have concurrent writes originating from the same Apache Spark™ driver. This was one of the most requested issues by the community and we are excited to announce that Delta Lake 1.2 (release notes, blog) now supports writing data from multiple clusters to S3 while maintaining the transactionality of the writes.

Delta Lake 1.2 - More Speed, Efficiency and Extensibility Than Ever

By Venki Korukanti , Scott Sandre , Tathagata Das , Allison Portis , Denny Lee , Vini Jaiswal

Introducing performance optimizations that will supercharge your data pipelines at any scale.

Writing to Delta Lake from Apache Flink

By Fabian Paul , Pawel Kubit , Scott Sandre , Tathagata Das , Denny Lee

Learn more about how you can write from Apache Flink to Delta Lake about the latest release of the open-source project Delta Sharing and how it enables sharing on Google Cloud Storage, among other enhancements.

Extending Delta Sharing to Google Cloud Storage

By Will Girten , Shixiong Zhu

Learn more about the latest release of the open-source project Delta Sharing and how it enables sharing on Google Cloud Storage, among other enhancements.

Delta Connectors 0.3.0 Released

By Allison Portis

We are excited to announce the release of Delta Connectors 0.3.0.

Delta Lake 1.1.0 Released

By Scott Sandre

We are excited to announce the release of Delta Lake 1.1.0.

Delta Sharing 0.3.0 Released

By Lin Zhou

We are excited to announce the release of Delta Sharing 0.3.0.

Power BI Delta Sharing Connector

By Denny Lee

We are excited about the recently announced preview of the Power BI Delta Sharing connector

Delta Lake User Survey (2021 H2)

By Denny Lee

We would like to invite you to provide your feedback on Delta Lake OSS.