-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RETRY implements a backoff strategy that differs from Google recommended strategy - customise retry strategy? #661
Comments
As a point of reference, gargle now has a branch where I implement a Google-ish retry strategy: |
And to address one of your points more specifically: I'm not sure there's really a "Google party line" on how exponential backoff should be implemented. They are pretty clear that they want exponential backoff, morally speaking.
I dove into all these links in the
My memory is a bit foggy, but I'm pretty sure that I saw examples of a exponentially increasing deterministic wait + random jitter AND random wait with an exponentially increasing support, even within the Google links. And then, in my case, there was a very special important case for Google Sheets (quota violation), where I have to dig the wait time out of a message 😬 |
Wow lots of thought on it already, great to see. I did wonder if this would be Google specific so look at it in gargle but it seems httr would be a nice place to have a more generic solution. Perhaps being able to pass a numeric sequence or a function that makes a sequence to |
I think any substantial innovation / upgrading might be a task for httr2 (https://github.com/r-lib/httr2/issues), which is mostly just an issue target at this point. I definitely thought of my gargle retry work as 50% for gargle-using packages today and 50% research for more flexible retry behaviour in httr or its successor. |
httr has been superseded in favour of httr2, so is no longer under active development. If this problem is still important to you in httr2, I'd suggest filing an issue offer there 😄 — but |
This issue has been raised on googleAuthR: MarkEdmondson1234/googleAuthR#186
Is it possible to have some more customisation on how the retry strategy occurs? At the moment it means that too many requests happen for Google APIs in particular, and perhaps other APIs may also have differing backoff rules.
In the Google API case, the stipulation is: "2^attempt + random(0, pause_base)" whereas
httr
's RETRY uses "randomly wait between 0 and pause_base * 2 ^ attempt seconds"The difference means that by the 3rd retry the difference is minimum 8s vs between 0-8seconds which means the 3rd retry is always too soon.
The text was updated successfully, but these errors were encountered: