Video explainers
Data engineering, explained in under a minute each
Eight short animated explainers covering the questions that come up first: what data engineering actually is, how ETL and ELT differ, when batch beats streaming, where analytical data should live, how analytical models are shaped, what an orchestrator buys you, how quality is measured, and what the modern stack is made of. There is no narration and no audio track at all, so every point is on screen. They read the same muted, in a meeting, or embedded in your own docs.
-
01 What Is Data Engineering
Data engineering builds and runs the systems that move, store, model and serve data, plus the reliability work that keeps the results trustworthy.
- Move data from where it is created to where it gets used
- Four responsibilities show up in every data platform
- Correct once is not the job. Correct every run is.
-
02 ETL vs ELT
ETL reshapes data on a separate tier before loading it. ELT lands raw data in the platform first and transforms it in place. What changes is where compute runs and whether you keep the raw copy.
- Transform before you load
- Load first, transform in place
- What you gain, and what you take on
-
03 Batch vs Streaming
Batch jobs run on a schedule over everything that arrived since the last run. Streaming processes each record as it lands. Micro-batching sits between them, and most platforms end up running more than one.
- Work piles up, then runs on a schedule
- Each record is processed as it arrives
- Micro-batching sits between them, and most platforms run both
-
04 Warehouse, Lake, Lakehouse
The warehouse gave analytics governed tables in a closed format. The lake gave open files with none of the guarantees. The lakehouse puts a table format over open storage so the guarantees come back.
- Governed tables, in someone else's format
- Open files, and none of the guarantees
- A table format puts the guarantees back
- One copy of the data, many engines on top of it
-
05 Data Modeling for Analytics
Operational schemas normalize to make writes safe. Analytical models denormalize to make scans fast and questions readable. Dimensional modeling is how that second shape is built, and slowly changing dimensions are how it survives change.
- Two schemas, optimized for opposite things
- Facts in the middle, dimensions around them
- Dimensions change, and history has to survive it
-
06 Orchestration and DAGs
Pipeline steps have dependencies, and a directed acyclic graph is how you state them. An orchestrator turns that graph into scheduling, retries, backfills and run history you can actually see.
- Steps have dependencies, so they need a graph
- Schedule, execute, observe, recover
- A graph of tasks, or a graph of assets
-
07 Data Quality and Observability
Quality is a set of measured dimensions with thresholds. Write-audit-publish keeps a failed check from ever becoming a visible table. Observability watches freshness, volume, schema, distribution and lineage for the rest.
- Quality is measured, not asserted
- Run the checks before anyone can read the table
- Five signals for the failures nobody wrote a test for
-
08 The Modern Data Stack
The stack unbundled into ingestion, storage, transformation and serving, each purchasable on its own. Open table formats moved the lock-in question from storage to the catalog, and the semantic layer is where shared definitions now live.
- Ingestion, storage, transformation, serving
- The lock-in question moved from storage to the catalog
- Definitions belong above the tables, not inside each tool
Silent by design: no narration, no music. Hover to preview, click to play, or download any MP4 and reuse it. Every topic here has a longer write-up in the knowledge base.