Juliana Hirt

Camp Counselor

Modern Data Independence - Data Ingestion Using ELT Patterns for Diverse Workloads

Event Logo

Tuesday, July 26, 2022 - 3:30 PM UTC, for 1 hour.

ON THAT (Online Only) Regular, 60 minute presentation

etl
infrastructure

Since the introduction of the relational database model by E.F. Codd in 1970, there has been a standardized thought process on how data should be ingested, prepared, and stored. This technique is often referred to as ETL, or extract-transform-load. ETL processes were optimized to take advantage of the available hardware resources and the overall shape and type of the data that was being ingested. Evolving with the everchanging data topology, a new design pattern has been emerging; ELT, or extract-load-transform. Data engineers are now challenged to build frameworks which can handle standard row and column data alongside more modern data such as image, video, and compressed files. In this section we will be reviewing the new ELT pattern using a sample architecture, and the benefits it provides to today’s data engineers

Take Aways

  • Understand the key differences between ETL and ELT architectures.
  • Understand the benefits of ELT workloads for diverse data sets.
  • Understand the necessary cloud resources to implement this pattern.
favorited by:
Chris Johnson Timothy Tyson