sql magic not limited by jupyterlab-limit-output

[jameshowison]

Context

Students often run SQL queries with large output, forgetting to always add a LIMIT statement. This can produce so much output that it locks up the server. We encountered this with the xsql notebooks

We are now mostly working in python but we don't have a database browser (I never managed to introduce pgadmin since the time was past by the time we got it working). So it's convenient for students to use the %%sql magic cells to directly run SQL (to see results of python driven inserts, etc).

%%sql
SELECT *
FROM performances

Problem is that those cells aren't limited by jupyterlab-limit-output (so that query returns 140 rows, just an example).

On the other hand jupyterlab-limit-output does limit other output. So if I do that query like this

import pandas

pd = %sql SELECT * FROM performances
pd.DataFrame() # or pd.DataFrame().to_dict

then it's limited to 20 rows (using to_dict is also limited).

Proposal

Need a way to have jupyterlab-limit-output apply to all cell types (or at least apply to both the xsql and the sql-magic cell types. I'm also posting at https://github.com/deshaw/jupyterlab-limit-output

Updates and actions

No response

[jameshowison]

Hey, you guys are awesome and already tried to push a fix for xues-sql upstream to deshaw: deshaw/jupyterlab-limit-output#2

I wonder if sql-magic just uses a different and additional MIME type?

[jameshowison]

Also, if they aren't going to merge PRs, can we specify the merged PR in the image. ie using something like

pip install git+https://github.com/user/repo.git@refs/pull/123/merge

got that from here: https://stackoverflow.com/questions/13561618/pip-how-to-install-a-git-pull-request

So would that be something like replacing jupyterlab-limit-output in environment.yml with

https://github.com/deshaw/jupyterlab-limit-output.git@refs/pull/2/merge