A small TIL about Python profiling.

SQL is an amazing language, it lets you declaratively say what you want, and the engine figures out for you the best way to return it to you. Or should I say, it figures out the best way to return it to you given the information it has and the capabilities of the engine itself.

In this post, we’ll discuss a performance optimization technique for BigQuery (also other advanced enough Enteprise Data Warehouses and databases support SEMI JOINS, but I'll focus on BigQuery since it's the one I use the most these days): using semi joins.

IN and EXISTS often produce the same result, but when negated and dealing with NULL values, they behave differently.

Today I learned how to run a webserver in Google Colab, without needing external services like ngrok. I'm using Dagster here as an example but any webserver should work.

Presentation I gave at the Dagster Community Meeting on the 21st of September, 2021 on how to use Dagster for machine learning purposes, with an explanation on why we chose that instead of Airflow.