Skip to content

Data from Scratch

The plain-English path from "what is a table?" to working as a data engineer, analytics engineer, or database administrator. Eight phases, each ending with something you can actually do.

No exam required. Certifications are named where they help, but every phase stands on its own.

How to use this path

Work through the phases in order. Each one lists:

  • Learn - the concept pages on this site to read first
  • Do - a small exercise or build that proves you understood it
  • After this phase - what you can now do that you could not before
  • Certify (optional) - exams that test the same ground

Most people spend a few weeks per phase alongside a job. Phases 1 to 4 are shared by every data role. Phase 7 is where the three roles split.

flowchart LR
    P0[0. Tech basics] --> P1[1. SQL and tables]
    P1 --> P2[2. Modeling for analytics]
    P2 --> P3[3. Pipelines]
    P3 --> P4[4. Streaming]
    P4 --> P5[5. Quality and governance]
    P5 --> P6[6. Data in the cloud]
    P6 --> P7{7. Pick a lane}
    P7 --> DE[Data engineer]
    P7 --> AE[Analytics engineer]
    P7 --> DBA[Database administrator]
    DE --> P8[8. Get hired]
    AE --> P8
    DBA --> P8

The diagram shows the eight phases in order, with the path branching into three roles at phase 7 and joining again for interview prep.

Phase 0 - Tech basics (skip if you already use a terminal)

Data work happens in terminals, notebooks and cloud consoles. If those are new, start on the sister site.

  • Learn: Day One on Cloud Zero to Hero - terminal, git, HTTP, servers, JSON and YAML, assuming nothing.
  • Do: install Python, open a terminal, and run python3 --version. Work through the first few chapters of the official Python tutorial.
  • After this phase: you can run a script, read an error message, and save your work in git.

Phase 1 - SQL and relational thinking

SQL is the one skill every data role uses every day. Learn it before any tool.

  • Learn: SQL vs NoSQL for the big picture, then the Databases topic index for where each engine fits.
  • Do: load a public CSV into SQLite or PostgreSQL. Write queries with SELECT, WHERE, GROUP BY, JOIN, and one window function (ROW_NUMBER() or SUM() OVER).
  • After this phase: you can answer a business question ("which product sold best last month, by region?") straight from raw tables.
  • Certify (optional): Azure Data Fundamentals (DP-900) covers relational, non-relational and analytics basics at an entry level.

Phase 2 - Modeling data for analytics

Operational databases are shaped for writing one order at a time. Analytics wants the opposite: reading millions of rows at once. Modeling is how you bridge the two.

  • Learn: Warehouses, lakes, and lakehouses, then Data modeling for analytics (star schemas, facts and dimensions, slowly changing dimensions).
  • Do: take the tables from phase 1 and redesign them as one fact table and three dimension tables. Rewrite your phase 1 queries against the new model.
  • After this phase: you can explain why a warehouse is laid out differently from an app database, and design a star schema for a simple business.

Phase 3 - Pipelines: moving and transforming data

Data rarely starts where it is needed. A pipeline extracts it, cleans it, and lands it somewhere useful, on a schedule, without a human pressing a button.

Phase 4 - Streaming and event data

Batch pipelines run every hour or every night. Some data cannot wait that long: payments, sensor readings, clickstreams.

Phase 5 - Quality, lineage and governance

A pipeline that runs on time with wrong numbers is worse than one that fails loudly. This phase is what separates hobby projects from production data.

  • Learn: Data quality and lineage.
  • Read: the Data mesh pattern, which treats each team's data as a product with an owner and a contract.
  • Do: add three checks to your phase 3 pipeline (row count not zero, no null primary keys, totals match the source) and make the pipeline stop when one fails.
  • After this phase: you can say where a number on a dashboard came from, and catch bad data before a stakeholder does.
  • Related: access control and data privacy are covered on Security Zero to Hero.

Phase 6 - Data in the cloud

Most data platforms now run on a cloud provider or a cloud data platform. You do not need to master a whole cloud, but you need its storage, identity and compute basics.

Phase 7 - Pick a lane

The first six phases are shared. Here the three roles diverge. Pick the one closest to the work you enjoyed most.

Data engineer

You build and run the pipelines and platforms everyone else depends on.

Analytics engineer

You turn raw tables into trusted models and the dashboards the business reads. More SQL and modeling, less infrastructure.

Database administrator (DBA)

You keep databases fast, available, backed up and secure. Fewer pipelines, more depth in one or two engines.

Phase 8 - Get hired

  • Read: Data engineer interview prep - pipeline design, data modeling and SQL questions.
  • Quick refresher: Data engineer in 1 hour, a curated reading order through this site.
  • Show your work: put the pipeline from phases 3 to 6 on GitHub with a README that explains the design, the checks from phase 5, and what you would change at ten times the data.
  • Practice: every cert above has a practice question bank with explained answers.

Where this path stops

  • Machine learning and AI are on the sister site, AI Zero to Hero. Start with its zero to hero path. The pipelines and models you built here are what ML systems are fed from.
  • Cloud infrastructure beyond what a data platform needs (networking, Kubernetes, DevOps) is on Cloud Zero to Hero.
  • Security (identity, encryption, compliance such as GDPR and HIPAA) is on Security Zero to Hero.
  • Concepts - every data concept page on one list
  • Study Hub - every data certification, by provider
  • Topic indexes - databases and data engineering, across every pillar