conductor-oss / conductor-oss/conductor

Postgres freeText search configuration doesn't match depending on DB config

Open
#698 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
32.2k
Forks
1k
Avg merge
2d
Merged PRs (30d)
33

Description

### Describe the bug

I recently came across an issue where querying tasks with freeText search was returning different results depending on the default configuration of the database.

For context, we're using conductor with postgres as the indexing engine. The relevant config is here:
```
conductor.db.type=postgres
spring.datasource.url=
spring.datasource.username=
spring.datasource.password=
conductor.postgres.schema=
conductor.indexing.enabled=true
conductor.indexing.type=postgres
conductor.elasticsearch.version=0
```

We have a `HUMAN` task that accepts the following as its input parameters:
```
{
"waitForNotaEvent": "clip.transcode.completed/93c4d5a0-371e-45ec-b8a8-541c06619682"
}
```

I then use the freeText search to find tasks with this input using the following:
```
GET /api/tasks/search?freeText=waitForNotaEvent=clip.transcode.completed/93c4d5a0-371e-45ec-b8a8-541c06619682
```
Note that the query params end up URL encoded but I've left them as plain text here for ease of use.

In `postgres-persistence/src/main/java/com/netflix/conductor/postgres/util/PostgresIndexQueryBuilder.java` the `parseFreeText` method uses `jsonb_to_tsvector('english', json_data, '["all"]')` but `Condition.getQueryFragment` uses `to_tsquery(?)` and doesn't set a config. [The postgres docs](https://www.postgresql.org/docs/current/textsearch-controls.html#TEXTSEARCH-PARSING-QUERIES) mention that if no configuration is set ( e.g. `to_tsquery('english', 'query')` ) then it will use the default value for `default_text_search_config`.

In my local environment this was set to `pg_catalog.english` but on our hosted instance this was set to `pg_catalog.simple`. The DB is hosted on AWS and there's no way to change this particular value via config ( a known issue ), so I've resorted to running some SQL to set that.

The result of this is that queries on my local instance returned results as expected, but when we run the same query on our hosted instance it didn't execute the free text search correctly.

It would be ideal if `to_tsquery` used the same config as `jsonb_to_tsvector` to ensure that query always matched and didn't rely on default configuration values.

## Instructions to reproduce
- Set up a Postgres DB and ensure that `default_text_search_config` is set to `pg_catalog.simple`. You can use `ALTER ROLE [role] SET default_text_search_config TO 'pg_catalog.simple';`
- Use the above config to configure conductor to use postgres indexing.
- Create a workflow with a single task. This could be any task.
- Give the task the above input JSON.
- Try to query for that task using the `freeText` parameter, as above.

Expected behaviour is that these tasks are returned. Actual behaviour is that nothing is returned.

Now run `ALTER ROLE [role] SET default_text_search_config TO 'pg_catalog.english';` and try the search again. You should see the correct results.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.