google / google/googlesql

Issue in SimpleCatalog when accessing in a concurrent environment

Open
#138 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
2.6k
Forks
260
PR merge metrics
No merged PRs in 30d

Description

We are using zetasql Java parser as an API on Cloud run. Each container receives multiple requests and the Catalog is declared in the global scope as a static final variable.
Each request has a query and we scan the query for tables and add it to the catalog at runtime. The idea is that as more queries come, the catalog will grow as needed, without it needing to be prefilled.

Now the issue is that I'm seeing random `Table not found` errors when requests are coming. Some how the parser is not seeing the tables in the catalog as it parses the query. This is only happening in concurrent environment. If I keep concurrency as 1, everything works perfectly.

My hypothesis is that the SimpleCatalog uses Hashmap to store tables and functions. As a hashmap is not thread safe, when multiple requests are coming quickly, it is facing issues/not being updated.

Can someone help in finding why this might be happening?

Contributor guide

Open the contributing guide

Research direction

Start with SimpleCatalog and reproduce the reported difference between concurrency 1 and multiple concurrent requests using the Java parser on Cloud Run. Inspect how tables and functions are stored and updated, including the reported HashMap thread-safety hypothesis. Done means identifying the cause of the random “Table not found” errors and validating the behavior under concurrent requests.

Written by the indexing model from the issue text.

Assessment

Tech stack
gcp, java, sql
Domain
backend, cloud, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.