Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for allowMissingColumns flag in nodejs-polars client #284

Open
mostopalove opened this issue Oct 8, 2024 · 3 comments
Open

Comments

@mostopalove
Copy link

Description:
I would like to request the addition of support for the allowMissingColumns flag in the Node.js client, similar to its current implementation in the Python client.

This feature is documented in the Python API here:
Polars Python Documentation - allow_missing_columns

In the Node.js client, the current available options for ScanParquetOptions can be found here:
Node.js Polars Documentation - ScanParquetOptions

Feature Request:
Please consider adding the allowMissingColumns flag to the Node.js client's ScanParquetOptions to provide consistency across different client implementations.

In addition, this feature should help resolve errors like "Error: schemas contained differing number of columns", which arise when loading parquet files with schema inconsistencies between different files.

Thank you for considering this request!

@Bidek56
Copy link
Collaborator

Bidek56 commented Oct 8, 2024

allow_missing_columns was just added to to Polars, we need to let it bake a bit and be added to a RS release first. :)

@liadhorovitz
Copy link

Thanks @Bidek56
Any estimation for the "baking" time? :)

@universalmind303
Copy link
Collaborator

the rust release cadence is a bit slower than Python. Historically, it looks like there's a release every 1-2 months

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants