Add APIs in DynamicTokenBucket to be able to return excess tokens and to borrow from future.
Summary: The common use case for token buckets to implement resource usage throttling/smoothing also implies that callers need to either implement their own backoff+retry mechanism. The simple exponential backoff approach is prone to starvation and over-throttling (under-utilized resource). Many cases will simply be okay with being told at what point in future their allocations can be met effectively doing the starvation free FIFO scheduling for them. Added a consumeWithBorrow API to enable this behavior. Added checks in existing API to account for the possibility of the internal clock of the bucket being 'in future' wrt to the 'now' argument passed in the call. Also added an API to be able to return previously allocated tokens effectively moving the clock back on the bucket. This is largely for completeness and there isn't an immediate use case in mind for this. Reviewed By: yfeldblum, usumeet Differential Revision: D13166472 fbshipit-source-id: 4d6f93eedcc75eb07c1250026e57806c96974c96
Showing
Please register or sign in to comment