Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fail instead of looping when data cannot be generated (bad_test hit too often) #63

Open
smelc opened this issue Mar 3, 2021 · 0 comments

Comments

@smelc
Copy link

smelc commented Mar 3, 2021

If a test is too restrictive:

let is_bad_data : ... -> bool

f data =
  if is_bad_data data
  then Crowbar.bad_test ()
  else ... (* do test *)

let () =
  Crowbar.add_test ~name:"test f" [generator] f

I mean if Crowbar.bad_test is hit some number of times, it would be better to fail at some point, to give feedback to the caller. At the time of writing, Crowbar doesn't terminate. For example Haskell's quickcheck does this, along the lines of giving up generating data after X trials and only Y successes.

As this is a breaking change (it could make tests that pass now (slowly though) fail), care should be taken regarding adoption.

Thanks for your work on crowbar ❤️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant