facebook / facebook/folly

DynamicTokenBucket::consumeWithBorrowNonBlocking will return incorrect value with very low rate

Open
#2,430 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
30.5k
Forks
5.9k
PR merge metrics
No merged PRs in 30d

Description

The following unit test demonstrates the issue:

```
TEST(TokenBucketTest, CanConsumeFirstTokenLowRate) {
folly::DynamicTokenBucket tokenBucket;
ASSERT_EQ(tokenBucket.consumeWithBorrowNonBlocking(1.0 /** toConsume*/, .0001 /** rate*/, 1.0 /** burstSize*/), 0.0);
}
```

IIUC, this should succeed because the bucket is initialized with 1.0 token, and we are only attempting to consume this first token, and no wait should be required. However, for an excessively low rate (like .0001, this does not repro with .001), the test fails instead:

```
{"t":{"$date":"2025-05-02T12:53:32.953+00:00"},"s":"I", "c":"TEST", "id":8423378, "ctx":"main","msg":"Test Failed","attr":{"testName":"TokenBucketTest/CanConsumeFirstTokenLowRate","exception":"TestAssertionFailureException","error":"Expected tokenBucket.consumeWithBorrowNonBlocking(1.0, .0001, 1.0) == 0.0 ( 6241.23 == 0) @src/third_party/folly/test/token_bucket_test.cpp:47"}}
```

There may be a precision bug in the code-- I think it would also be acceptable to document the maximum precision of the rate if such a limit exists.

Contributor guide

Open the contributing guide

Research direction

Start by running the CanConsumeFirstTokenLowRate test in src/third_party/folly/test/token_bucket_test.cpp and tracing consumeWithBorrowNonBlocking with the very low rate shown. Compare the returned value with the expected 0.0, then verify the behavior for nearby rates and determine whether the fix should preserve precision or document a rate limit.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.