Delta Lake Blogs
Multi-cluster writes to Delta Lake Storage in S3
By Scott Sandre , Denny Lee , Mariusz Kryński
While Delta Lake has supported concurrent reads from multiple clusters since its inception, there were limitations for multi-cluster writes specifically to Amazon S3. Note, this was not a limitation for Azure ADLSgen2 nor Google GCS, as S3 currently lacks “put-If-Absent” consistency guarantees. Thus, to guarantee ACID transactions on S3, one would need to have concurrent writes originating from the same Apache Spark™ driver. This was one of the most requested issues by the community and we are excited to announce that Delta Lake 1.2 (release notes, blog) now supports writing data from multiple clusters to S3 while maintaining the transactionality of the writes.
Delta Lake 1.2 - More Speed, Efficiency and Extensibility Than Ever
By Venki Korukanti , Scott Sandre , Tathagata Das , Allison Portis , Denny Lee , Vini Jaiswal
Introducing performance optimizations that will supercharge your data pipelines at any scale.
Writing to Delta Lake from Apache Flink
By Fabian Paul , Pawel Kubit , Scott Sandre , Tathagata Das , Denny Lee
Learn more about how you can write from Apache Flink to Delta Lake about the latest release of the open-source project Delta Sharing and how it enables sharing on Google Cloud Storage, among other enhancements.
Extending Delta Sharing to Google Cloud Storage
By Will Girten , Shixiong Zhu
Learn more about the latest release of the open-source project Delta Sharing and how it enables sharing on Google Cloud Storage, among other enhancements.
Delta Connectors 0.3.0 Released
We are excited to announce the release of Delta Connectors 0.3.0.
Delta Lake 1.1.0 Released
By Scott Sandre
We are excited to announce the release of Delta Lake 1.1.0.
Delta Sharing 0.3.0 Released
By Lin Zhou
We are excited to announce the release of Delta Sharing 0.3.0.
Power BI Delta Sharing Connector
By Denny Lee
We are excited about the recently announced preview of the Power BI Delta Sharing connector
Delta Lake User Survey (2021 H2)
By Denny Lee
We would like to invite you to provide your feedback on Delta Lake OSS.
Delta Lake 1.0.0 Released
We are excited to announce the release of Delta Lake 1.0.0 on Apache Spark 3.1.
Salesforce Engineering: Delta Lake Tech Talk Series
By Denny Lee
We are happy to announce the Salesforce Engineering Delta Lake Tech Talk Series for March and April 2021.
AMA: Growing the Delta Lake ecosystem
By Denny Lee
On March 11th, 2021 9:00 am PT, join us for this fun Delta Lake AMA session where we discuss with QP Hou, Christian Williams, and Alexander Kushnir from Scribd on growing the Delta Lake open-source ecosystem.