Issue in SimpleCatalog when accessing in a concurrent environment
- Dominant language
- C++
- Stars
- 2.6k
- Forks
- 260
- PR merge metrics
- No merged PRs in 30d
Description
We are using zetasql Java parser as an API on Cloud run. Each container receives multiple requests and the Catalog is declared in the global scope as a static final variable.
Each request has a query and we scan the query for tables and add it to the catalog at runtime. The idea is that as more queries come, the catalog will grow as needed, without it needing to be prefilled.
Now the issue is that I'm seeing random `Table not found` errors when requests are coming. Some how the parser is not seeing the tables in the catalog as it parses the query. This is only happening in concurrent environment. If I keep concurrency as 1, everything works perfectly.
My hypothesis is that the SimpleCatalog uses Hashmap to store tables and functions. As a hashmap is not thread safe, when multiple requests are coming quickly, it is facing issues/not being updated.
Can someone help in finding why this might be happening?
Contributor guide
Research direction
Start with SimpleCatalog and reproduce the reported difference between concurrency 1 and multiple concurrent requests using the Java parser on Cloud Run. Inspect how tables and functions are stored and updated, including the reported HashMap thread-safety hypothesis. Done means identifying the cause of the random “Table not found” errors and validating the behavior under concurrent requests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- gcp, java, sql
- Domain
- backend, cloud, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100