Your Portfolio & Online Presence

Why a portfolio matters more than a degree in data

In a lot of fields, a credential is the ticket. In data engineering, the credential is increasingly the work itself. Hiring managers have been burned by candidates who can recite the difference between a star schema and a snowflake schema but have never actually built a pipeline that runs. So they've learned to discount "trust me" and reward "show me."

That's an enormous advantage for a career-changer. You may not have a CS degree or years on the job, but you can have something a fresh graduate often doesn't: a real, working, documented system that someone can click into and inspect. A portfolio collapses the distance between claiming a skill and demonstrating it.

The capstone (mini-GridDP) as your centerpiece

Your capstone — the mini-GridDP platform — is the single strongest asset in your entire search. Most applicants have toy notebooks; you have an end-to-end system. The job now is presentation: making it trivially easy for a stranger to understand what it does, see it working, and respect the judgment behind it. A strong centerpiece has four layers:

1. A clean public repo. Open-source it on GitHub. Before you do, sweep for embarrassments: no committed API keys or .env files (add them to .gitignore and rotate any that leaked), no 2 GB of raw data, no folder called final_FINAL_v3. A tidy structure signals you've worked on a real team's codebase before, even if you haven't.

2. A great README. This is the most-read file in your portfolio and the one most beginners neglect. The README is the project as far as a skimming recruiter is concerned. It should answer, in order: what is this, what does it do, what does the architecture look like, and how do I run it. Lead with the architecture diagram — a single image that shows data flowing from source → ingestion → warehouse → transformation → dashboard communicates more in three seconds than three paragraphs.

3. A short demo. Code in a repo is abstract; seeing it run is visceral. You don't need a polished video — pick whichever is fastest:

• Screenshots of the dashboard and a successful pipeline run, embedded directly in the README.
• A GIF of the dashboard updating or a DAG turning green — it autoplays in the README and instantly proves "this is real."
• A 2–3 minute Loom walkthrough where you click through the running system and narrate it. This is the single highest-leverage artifact you can make; it's the next best thing to a live demo and recruiters can forward it.

4. A "decisions & tradeoffs" writeup. This is what separates a portfolio that proves a project runs from one that proves you can think. Add a short section (in the README or a linked DECISIONS.md) explaining a few choices: why you used Dagster instead of Airflow, why you partitioned a table the way you did, what breaks at 100× the data volume and how you'd fix it. Naming a limitation honestly is a strength signal, not a weakness — it shows engineering maturity.

Two or three smaller projects that show range

The capstone proves depth. A couple of small, sharp side projects prove range — that you can pick up an unfamiliar tool and ship something focused. Resist the urge to build another giant platform; each of these should be a weekend, not a month, and each should do one thing cleanly. Quality and a good README beat scope.

Writing about your work

A repo shows what you built. A writeup shows how you think — and thinking is what gets hired. A short blog post or project writeup is one of the highest-return things you can produce, because it works while you sleep: it shows up in searches, it gives recruiters something to share, and it gives you a ready-made interview narrative.

A good writeup follows a simple four-beat arc:

• What I built — one or two plain sentences and the architecture diagram.
• Why — the problem it solves, or the skill you wanted to prove.
• What I learned — the surprise, the bug that took a day, the thing the docs didn't tell you. This is the most credible part; specificity reads as real experience.
• What I'd do next — the honest limitations and where you'd take it. This shows you see beyond the demo.

A recruiter-friendly GitHub

Your GitHub profile is, for many roles, the most-clicked link on your resume. A recruiter — often non-technical — spends seconds on it, so optimize for the at-a-glance impression. Here's what they actually look at, and what to do about each:

Here's the kind of README structure that reads well — clean headings, a diagram up top, run instructions, and a decisions section:

# mini-GridDP — an end-to-end data platform for a GPU-rental marketplace

A data platform for a GPU-rental marketplace: it ingests rentals from the app's
Postgres database and GPU telemetry from a stream, models them with dbt, and
serves revenue and utilization metrics on a dashboard.

![Architecture](docs/architecture.png)

## What it does
- Ingests rentals from Postgres and telemetry events from Redpanda, on a Dagster schedule
- Lands raw data in DuckDB (bronze), then models it silver -> gold with dbt
- Validates row counts and freshness; surfaces metrics in a Metabase dashboard

## Demo
![Dashboard](docs/dashboard.gif)
▶️ 2-minute walkthrough: https://loom.com/share/...

## Run it locally
```bash
git clone https://github.com/you/mini-griddp.git
cd mini-griddp
cp .env.example .env        # no real secrets committed
make setup && make run      # spins up the pipeline end to end
```

## Decisions & tradeoffs
- **Dagster over Airflow** — asset-based model fit the dependency graph better.
- **DuckDB for the warehouse** — zero-ops and fast for this data size; I'd swap
  in BigQuery/Snowflake past ~50 GB.
- **Known limits** — single-node ingestion; at 100x volume I'd partition by day
  and parallelize the extracts.

## Tech
Python · SQL · dbt · Dagster · DuckDB · Redpanda · Metabase · Docker

A LinkedIn recruiters can find

GitHub proves you can do the work; LinkedIn is how recruiters discover you in the first place. Most of them search a keyword database and message whoever matches. Your job is to be the profile that surfaces and reads as credible. Four things move the needle:

• Headline — the single most important line, shown everywhere. Make it a keyword-rich role statement, not a vague aspiration: Data Engineer | Python · SQL · dbt · Airflow/Dagster · BigQuery. Recruiters search those exact terms.
• About — three short paragraphs that tell your story: where you're coming from, the skills you've built (name the tools), and what you're looking for. A career-change arc ("operations analyst who taught myself to build the pipelines I kept wishing existed") is memorable, not a liability.
• Skills & keywords — fill the Skills section with the tools from the postings you're targeting. This section is literally what recruiter searches match against; an empty one makes you invisible.
• Contactable & open — turn on "Open to work" (recruiter-only visibility if you prefer it discreet), add an email or link, and use a real, friendly headshot. An unreachable profile wastes every other bit of effort.

✓ Check yourself

Answer from memory before opening each one.

1. Your capstone repo has clean code, a README with an architecture diagram and run instructions, and a GIF of the DAG turning green. What has a reviewer still not seen evidence of, and what do you add to show it?

   Show answer

   Judgement. Repo and README prove the project exists, the demo proves it runs, but nothing yet proves you can think. Add a decisions & tradeoffs section (or a linked DECISIONS.md): why Dagster over Airflow, why a table is partitioned the way it is, what breaks at 100× the volume and how you would fix it. See The capstone as your centerpiece.

2. You are drafting the decisions section and know your ingestion is single-node and would not survive 100× the data. A friend says to leave that out because it makes the project look weak. What does the chapter say?

   Show answer

   Put it in. The tempting view is that a portfolio should show only strengths; the chapter's is that naming a limitation honestly, with how you would fix it, is a strength signal — it shows engineering maturity and that you see beyond the demo. It is also exactly what an interviewer will probe, so writing it down pre-loads your answer. See the decisions & tradeoffs layer and Writing about your work.

3. A recruiter with no engineering background opens your GitHub profile for about ten seconds. Name three things they actually register, and the fix for a language bar that reads 100% Jupyter Notebook.

   Show answer

   Pinned repos (pin 3–6, capstone first — nobody scrolls past them), whether each has a README (none reads as abandoned, however good the code), and the contribution graph and profile (real regular commits, a name, a bio, a link onward). Balance an all-notebook bar with Python, SQL and YAML repos so the profile reads as "engineer". See A recruiter-friendly GitHub.

4. Your LinkedIn headline is "Aspiring data professional passionate about insights", and your Skills section is empty. No recruiter has ever messaged you. Explain the mechanism, and rewrite the headline.

   Show answer

   Recruiters search a keyword database and message whoever matches; the headline and Skills section are what those searches hit, so with no tool names you never appear in the results. Make it a keyword-rich role statement — Data Engineer | Python · SQL · dbt · Airflow/Dagster · BigQuery — and fill Skills with the tools from the postings you are targeting. See A LinkedIn recruiters can find.

5. Your capstone is batch-only, and the postings you decoded in Chapter 01 keep mentioning Kafka and dbt. You have three free weekends. What do you build, how big, and what do you not build?

   Show answer

   One or two small projects that cover the gap and do one thing cleanly: a streaming mini-project (a producer plus a consumer that aggregates into a table), and if dbt is not already loud in the capstone, a dbt-only project with staging → marts, tests and dbt docs. Each is a weekend with a good README, not a second platform — the capstone already proves depth, these prove range, and three projects is plenty. See Smaller projects that show range.

6. You have one spare evening before you start applying: record a 2–3 minute Loom walkthrough of the running capstone, or write a blog post about building it. Which first, and what would reverse the order?

   Show answer

   The Loom, if the README has no demo yet: the chapter calls it the single highest-leverage artifact, because seeing the system run is the proof every visitor to the repo looks for, and recruiters can forward it. If screenshots or a GIF already cover "it runs", the writeup wins — it shows how you think, surfaces in searches, and becomes your interview narrative, and its value compounds over a months-long search. See A short demo and Writing about your work.

# mini-GridDP — an end-to-end data platform for a GPU-rental marketplace

mini-GridDP ingests rentals from a marketplace's Postgres database and GPU
telemetry from a Redpanda stream, lands them in a DuckDB warehouse, and models
them with dbt into clean gold tables. A Dagster schedule runs the whole pipeline
nightly and a Metabase dashboard serves revenue and utilization on top.

I built mini-GridDP to prove I could ship a real end-to-end data platform, not
just a notebook. It pulls rentals from Postgres and GPU telemetry from a
stream, lands them in DuckDB, and models them bronze-to-gold with dbt, all
orchestrated nightly by Dagster with row-count and freshness checks. The hardest part was making the
load idempotent — my first version double-counted rows on re-runs until I keyed
the merge on rental_id. I learned that orchestration is mostly
about handling failure and late data gracefully, which the happy-path tutorials
skip entirely. Next I'd partition the warehouse table by day and swap DuckDB for
BigQuery to see how the design holds up past 50 GB.

Takeaway

• In data engineering the credential is increasingly the work itself, so a documented, running system answers the hiring manager's doubt better than a degree line.
• Present the capstone as a clean public repo with no committed secrets, a README that says what it is and shows the architecture, and a short demo that proves it runs.
• A decisions-and-trade-offs writeup is what separates a portfolio that proves the project runs from one that proves you can think.
• Spotlight the signs of production work that interviewers probe: idempotent loads, tests, an orchestrated run, quality checks and handling of failure and late data.
• Add one or two small projects that cover a gap in the capstone, such as streaming for a batch-only build, matched to what target postings keep asking for.
• Make LinkedIn searchable with a keyword headline and a filled-in Skills section, and link the profile, the repo, the resume and the writeups to each other.

Your work is now visible and credible, so the next chapter builds the document that actually gets submitted. → The Resume & Applying

Further reading

• GitHub: About the repository README file — GitHub docs, accessed 2026. What a README should contain and where GitHub displays it.
• GitHub: Managing your profile README — GitHub docs, accessed 2026. Creating the README that shows at the top of your profile.
• GitHub: Creating diagrams — GitHub docs, accessed 2026. Drawing the architecture diagram in a README with Mermaid.
• GitHub: What is GitHub Pages? — GitHub docs, accessed 2026. A free home for project writeups.