Delta Lake Online Tech Talks

February 21, 2020
Denny Lee

Delta Lake Online Tech Talks

Delta Lake Online Tech Talks

News

February 21, 2020


We’re excited to announce the next series of Delta Lake online tech talks over the next few weeks. This will be a fun set of tech talks with live demos and Q&A. Check them out!

Getting Data Ready for Data Science with Delta Lake and MLflow

February 27, 2020 | 9:00 am PT
Watch Now!

One must take a holistic view of the entire data analytics realm when it comes to planning for data science initiatives. Data engineering is a key enabler of data science helping furnish reliable, quality data in a timely fashion. Delta Lake, an open-source storage layer that brings reliability to data lakes can help take your data reliability to the next level. In this session you will learn about:

  • The data science lifecycle
  • The importance of data engineering to successful data science
  • Key tenets of modern data engineering
  • How Delta Lake can help make reliable data ready for analytics
  • The ease of adopting Delta Lake for powering your data lake
  • How to incorporate Delta Lake within your data infrastructure to enable Data Science


Beyond Lambda: Introducing Delta Architecture

March 5, 2020 | 9:00 am PT
Watch Now!

Lambda architecture is a popular technique where records are processed by a batch system and streaming system in parallel. The results are then combined during query time to provide a complete answer. Strict latency requirements to process old and recently generated events made this architecture popular. The key downside to this architecture is the development and operational overhead of managing two different systems. There have been attempts to unify batch and streaming into a single system in the past. Organizations have not been that successful though in those attempts. But, with the advent of Delta Lake, we are seeing a lot of our customers adopting a simple continuous data flow model to process data as it arrives. We call this architecture, The Delta Architecture. In this session, we cover the major bottlenecks for adopting a continuous data flow model and how the Delta architecture solves those problems.


Simplify and Scale Data Engineering Pipelines with Delta Lake

March 12, 2020 | 10:00 am PT
Watch Now!

A common data engineering pipeline architecture uses tables that correspond to different quality levels, progressively adding structure to the data: data ingestion (“Bronze” tables), transformation/feature engineering (“Silver” tables), and machine learning training or prediction (“Gold” tables). Combined, we refer to these tables as a “multi-hop” architecture. It allows data engineers to build a pipeline that begins with raw data as a “single source of truth” from which everything flows. In this session, we will show how to build a scalable data engineering data pipeline using Delta Lake. Delta Lake is an open-source storage layer that brings reliability to data lakes. Delta Lake offers ACID transactions, scalable metadata handling, and unifies streaming and batch data processing. It runs on top of your existing data lake and is fully compatible with Apache Spark APIs In this session you will learn about:

  • The data engineering pipeline architecture
  • Data engineering pipeline scenarios
  • Data engineering pipeline best practices
  • How Delta Lake enhances data engineering pipelines
  • The ease of adopting Delta Lake for building your data engineering pipelines


Starting March 26th, join us for the Delta Lake Internals Online Tech Talks Series

Join the Delta Lake Community

Communicate with fellow Delta Lake users and contributors, ask questions and share tips
Slack ChannelGoogle Group


Project Governance

Delta Lake is an independent open-source project and not controlled by any single company. To emphasize this we joined the Delta Lake Project in 2019, which is a sub-project of the Linux Foundation Projects.

 

Within the project, we make decisions based on these rules.

Copyright © 2020 Delta Lake, a Series of LF Projects, LLC. For web site terms of use, trademark policy and other project policies please see https://lfprojects.org.
twitterstack-overflow