For hockey fans, it’s a familiar story. As the clock runs down in the final (3rd) period, teams losing by a goal or two will look to pull their goalie and send out an extra skater in their place. …
Pandas groupby is a powerful function that groups distinct sets within selected columns and aggregates metrics from other columns accordingly.
Performing these operations results in a pivot table, something that’s very useful in data analysis.
In this article, I share a technique for computing ad-hoc aggregations that can involve multiple…
Personal knowledge graphs are rapidly growing in popularity as benefits emerge. There are lots to chose from, but here’s why I love Obsidian.
Roam Research was the first tool like this that I learned about — their revolutionary graph approach to note taking blew my mind a bit.
Databases are like Pokémon. Gotta pass data between em’ all!
There are many methods of sending data to and from BigQuery. Some are even documented!
In this post I’ll share a simple method that I use to copy tables from Redshift into BigQuery using only SQL and command line tools.
In Python it’s okay to make assumptions, as long as you’re able to clean up the mess if they turn out to be wrong. In fact, this is not only okay but considered good practice.
This section is for people who want to learn about the basic try/except control structure…
In this post, I introduce the concept of dynamic DAG creation and explain the significance of Python global variables for Airflow.
Dynamic DAG creation is important for scalable data pipeline applications.
When confined to the realm of static DAG scripts, we find ourselves duplicating code in order to create pipelines.
Symbolic links are handy for shortcuts in your file explorer.
Maybe you have shortcuts on your desktop to your favorite folders, or maybe you symlink your active projects in your home directory for quick bash access.
ln -s /User/alex/Apps/2020/appthing /User/alex/appthing
This will create a shortcut to
If you’re like me then you try to avoid using the mouse whenever possible. This post will help with that.
My “normal” workflow involves opening new application windows pretty frequently.
For example, I might open a new terminal window to start working on a project, or I might open a…
Last night I fell down the rabbit hole of different ways to configure docker apps with runtime arguments.
Somehow it ended with me searching for ASCII art of a d20.. I settled on this:
If you’re thinking “what does ASCII art of dice have to do with docker?” Nothing really…
Why you should think about using JSON Line format in your data processing workflow. We’ll look at some jsonl examples and discuss how I use it day to day.
It’s a file type specification where each line is a JSON object. Just imagine a bunch of stacked up dictionaries. …