Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Operations] replace operation #74

Open
nfelnlp opened this issue May 20, 2023 · 0 comments
Open

[Operations] replace operation #74

nfelnlp opened this issue May 20, 2023 · 0 comments
Labels
enhancement New feature or request

Comments

@nfelnlp
Copy link
Member

nfelnlp commented May 20, 2023

A possible adaptation of the deprecated TTM operation change (actions/perturbation/what_if.py)
is replace where any string can be replaced with a custom input.

This would require two custom inputs, in fact. It would modify the temp_dataset. It is also similar to cfe, but here the user has full control of what they would insert in place of the original string from the data.

A major limitation is that even prediction is not easily possible, because the entire dataset would have to be recomputed. However, this might still be relevant for a small temp_dataset, e.g. you filter by "spider" and are left with 13 instances for BoolQ. With this scale, it would certainly be possible to compute predictions and explanations on the fly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant