Back to Learning: Starting Data Engineer in Python

Building on my SQL foundation

I am working through the Data Engineer in Python track. My recent portfolio work has focused on SQL, Power BI and a PostgreSQL ETL workflow. Python is the next part of that learning.

I am interested in how data reaches a database in the first place, particularly when it comes from an API or a collection of files. I also want more practice with transformations that are easier to express in a general purpose programming language.

What I want to develop

• API ingestion. Understanding authentication, pagination and how to handle incomplete responses.
• Data preparation. Working with Python and pandas while keeping the original data and transformation steps traceable.
• Version control. Using Git to keep changes to a workflow understandable.
• Orchestration. Learning how dependencies, scheduling and failure handling fit together, including the role of Airflow.

How this relates to my existing work

The PostgreSQL project gives me a starting point for thinking about repeatable processing. Python will let me explore more types of sources and more flexible workflows. I am treating it as a skill in progress rather than presenting course topics as professional experience.

Keeping the learning visible

I plan to use these field notes to record what I build and what I still need to understand. For now, the completed SQL and Power BI projects are the clearest examples of my work. New Python work will belong in the portfolio when there is an implementation to examine.