SQL is an amazing language, it lets you declaratively say what you want, and the engine figures out for you the best way to return it to you. Or should I say, it figures out the best way to return it to you given the information it has and the capabilities of the engine itself.

In this post, we’ll discuss a performance optimization technique for BigQuery (also other advanced enough Enteprise Data Warehouses and databases support SEMI JOINS, but I'll focus on BigQuery since it's the one I use the most these days): using semi joins.

IN and EXISTS often produce the same result, but when negated and dealing with NULL values, they behave differently.

Today I learned how to run a webserver in Google Colab, without needing external services like ngrok. I'm using Dagster here as an example but any webserver should work.

Presentation I gave at the Dagster Community Meeting on the 21st of September, 2021 on how to use Dagster for machine learning purposes, with an explanation on why we chose that instead of Airflow.

1. Not letting it double. I was impatient and kept refreshing it even if i didn't see any (or very little) activity. I had to wait longer.
2. Put it in the fridge too early. I put it in the fridge on day 7, while there was still no activity (probably due to point one). I had to put it back to ambient temperature to fix it.
3. Little oxygen available. I was putting a lid on top of the container. While it was just resting there unscrewed, I think that it reduced the amount of air going inside the container too much. I fixed it by replacing the lid with a kitchen towel secured with an elastic.
4. Too much water. I was hydrating it too much. While 100-110% hydration should be ok, it seems to grow much stronger now that i reduced it to 90%.
5. Changed flour too early. Changing flour is ok, but i did it one of the first days of the starter. I think that might have messed up things as I went from a 100% rye to a mix of whole wheat and strong white flour.