Skip to content
Valentina Vanegas Aristizábal

Work / Public sector

Data and economics · Public sector · 2026

Sisbén IV data pipeline

The first Power BI dashboard of Antioquia's Planning Department and the Python pipeline behind it, built on ~25M records of Sisbén, the survey Colombia uses to target social programs.

Status
Covers all 125 municipalities
Role
Data & Economics Intern Planning Department, Government of Antioquia
Period
Feb – Jun 2026
Stack
Python · DuckDB · PyArrow · Power BI · DAX · TMDL
Links
Code ↗ My post on LinkedIn (in Spanish) ↗ Internship certificate, in Spanish (PDF, 573 KB) My supervisor's letter of recommendation (PDF, 148 KB)

Abstract

Sisbén is the survey Colombia uses to decide who can receive social programs, so its numbers have to be right before anyone reads a chart. I built the path from six raw annual snapshots to a dashboard by municipality: a Python pipeline that cleans and audits the data, and a Power BI model the department’s Technical Director used with all 125 municipalities of Antioquia.

What I built

  • The first Power BI dashboard of Antioquia's Planning Department, with Sisbén data by municipality (2021–2026), used by the Technical Director with all 125 municipalities.
  • A batch ETL in Python with DuckDB and PyArrow over ~25M records across six annual snapshots (2021–2026).
  • 24 derived columns moved into the batch ETL, plus automated data-quality audits.
  • A Power BI semantic model in TMDL and DAX, with 19 tables.
Table 1 The pipeline in numbers.
WhatCount
Records~25M
Annual snapshots6 (2021–2026)
Derived columns24
Tables in the semantic model19
Municipalities in the dashboard125
Pipeline: six annual snapshots, 2021 to 2026; a Python ETL with DuckDB and PyArrow; about 25 million records with 24 derived columns and quality audits; a Power BI semantic model with 19 tables; a dashboard by municipality for all 125 municipalities. 6 annual snapshots 2021–2026 Python ETL DuckDB · PyArrow ~25M records 24 derived columns · quality audits Power BI semantic model 19 tables · TMDL, DAX Dashboard by municipality all 125 municipalities
Fig. 1 From six raw annual snapshots to a dashboard by municipality, with the data audited before anyone reads a chart.