apache / apache/beam

Client-side throttling for Datastore connector

Open
#18,483 0 comments 0 reactions 0 assignees View on GitHub
gcp improvement io java P3
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
2d 5h
Merged PRs (30d)
204

Description

The Datastore connector currently has exponential backoff on errors, which is good. But it does not do any other throttling of its write load in response to errors; once a request succeeds, it resumes writing as quickly as it can.

Write loads will be more stable and more likely to compete if the client throttles itself in the event that it receives high rates of errors from the Datastore service; specifically
https://landing.google.com/sre/book/chapters/handling-overload.html#client-side-throttling-a7sYUg is a technique that Google has had success with on other services.

We (Datastore) have a patch in progress to add this behaviour to the connector.

Imported from Jira [BEAM-2556](https://issues.apache.org/jira/browse/BEAM-2556). Original Jira may contain additional context.
Reported by: cphipps.

Contributor guide

Open the contributing guide

Research direction

Start by locating the Datastore connector implementation and reviewing the linked Google SRE guidance on client-side throttling, then check the imported BEAM-2556 context for the patch already in progress. Done means the connector throttles its write load after high error rates while retaining exponential backoff and resumes appropriately after successful requests.

Written by the indexing model from the issue text.

Assessment

Tech stack
gcp, java
Domain
cloud, data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.