NOT IN and NOT EXISTS don't always produce the same result

2024-06-01T00:00:00+00:00

IN</code> and EXISTS</code> often produce the same result, but when negated and dealing with NULL</code> values, they behave differently.</p> </span>

Let's see an example, assume we have these two tables:</p>

orders</strong></p>

order_id</th> customer_id</th> amount</th></tr></thead>

1</td> 100</td> 50.00</td></tr>

2</td> 101</td> 75.00</td></tr>

3</td> 102</td> 30.00</td></tr>

4</td>

NULL</td>

20.00</td></tr> </tbody></table>

customers</strong></p>

customer_id</th> name</th></tr></thead>

100</td> Bob</td></tr>

101</td> Alice</td></tr>

102</td> Martha</td></tr>

NULL</td>

John</td></tr> </tbody></table>

Ignore the fact that a well formed customers table would need to have the id always specified and ideally as a constraint. This is just an example.</em></p>

Now, let’s say we want to find orders where the customer_id is not present in the customers table.</p>

Using NOT EXISTS</code> we would write something like:</p>

SELECT </span>*
</span>FROM</span> orders o
</span>WHERE</span> NOT EXISTS (</span>SELECT </span>1 </span>FROM</span> customers c </span>WHERE </span>c</span>.</span>customer_id </span>= </span>o</span>.</span>customer_id</span>);
</span></code></pre>
The output, would probably be what we expect:</p>
order_id</th> customer_id</th> amount</th></tr></thead>

4</td> NULL</td> 20.00</td></tr>
</tbody></table>
Using NOT IN</code> we would write something like:</p>
SELECT </span>*
</span>FROM</span> orders
</span>WHERE</span> customer_id NOT IN (</span>SELECT</span> customer_id </span>FROM</span> customers);
</span></code></pre>
The output here will be empty!</p>
When we use NOT IN</code>, SQL checks each value in the orders table against the list of values returned by the subquery. If any value in the subquery result is NULL</code>, the entire NOT IN</code> comparison will result in NULL</code> for each row in the orders table. This is because any comparison with NULL</code> yields NULL</code> and NOT IN</code> needs all comparisons</strong> to be TRUE</code> for a row to be included in the result.</p>
This is not a problem with NOT EXISTS</code>, because the  NOT EXISTS</code> clause checks for the non-existence of rows that meet the criteria specified in the subquery. It does not perform direct comparisons with NULL</code> in the same way NOT IN</code> does. Instead, it simply checks if there are any rows that match the condition. If no such rows exist, the condition is true.</p>


How to run and serve a webserver in Google Colab without ngrok
2024-05-20T00:00:00+00:00
Today I learned how to run a webserver in Google Colab, without needing external services like ngrok. 
I'm using Dagster here as an example but any webserver should work.</p>
</span>
If you just want to create a public url for a webserver running in colab you can jump to step 4</strong></p>
Step 1: Install the Required Packages#</a>
</h2>
!pip install dagster dagster-webserver
</span></code></pre>
Step 2: Scaffold a New Dagster Project#</a>
</h2>
!dagster project scaffold --name test
</span></code></pre>
Step 3: Run Dagster in the Background#</a>
</h2>
To run Dagster's webserver, you'll need to start it in the background. This can be done using Python's subprocess module. The following code navigates to the project directory and starts the webserver on port 3000:</p>
import </span>subprocess
</span>
</span>subprocess.</span>Popen</span>(
</span>    [
</span>        "</span>bash</span>",
</span>        "</span>-c</span>",
</span>        "</span>cd /content/test && dagster-webserver -h 0.0.0.0 -p 3000 &</span>"
</span>    ]
</span>)
</span></code></pre>
Step 4: Create a Public URL for the Webserver#</a>
</h2>
Google Colab provides a way to create a public URL for your webserver. Use the output.serve_kernel_port_as_window function to expose the webserver running on port 3000.</p>
This does the real magic. Note that the URL is authenticated, and only the user running the notebook has access to the webserver.</p>
from </span>google.colab </span>import </span>output
</span>output.</span>serve_kernel_port_as_window</span>(</span>3000</span>, </span>path</span>='</span>/</span>')
</span></code></pre>
you can also run it in an iframe with</p>
output.</span>serve_kernel_port_as_iframe</span>(</span>3000</span>)
</span></code></pre>
Putting it all together#</a>
</h2>
You can check out a full example in this Colab Notebook</a></p>


Avoiding ZIP does not support timestamps before 1980 in Nix with Python
2020-09-15T00:00:00+00:00
By default, nix sets the epoch to be 1 (which corresponds to 1970-01-01), this causes some issues when building some Python packages. 
Since the zip might contain files from the nix store, those are dated 1970 breaking the installation of the python package.</p>
</span>
To quickly fix the issue add this to your nix-shell</p>
export </span>SOURCE_DATE_EPOCH</span>=</span>315532800
</span></code></pre>
Related:</p>

Github issue on Nixpkgs</a></li>
Nix python docs</a></li>
</ul>

Alessandro Marrella - til

Avoiding select * in BigQuery

How to quickly profile python imports and runtime

NOT IN and NOT EXISTS don't always produce the same result

How to run and serve a webserver in Google Colab without ngrok

Colab Notebook</a></p>

Colab Notebook</a></p>

Colab Notebook</a></p>

Avoiding ZIP does not support timestamps before 1980 in Nix with Python

Alessandro Marrella - til

Avoiding select * in BigQuery

How to quickly profile python imports and runtime

NOT IN and NOT EXISTS don't always produce the same result

How to run and serve a webserver in Google Colab without ngrok

Putting it all together#</a> </h2> You can check out a full example in this Colab Notebook</a></p>

Avoiding ZIP does not support timestamps before 1980 in Nix with Python

Colab Notebook</a></p>