awslabs / awslabs/graphrag-toolkit

[FEATURE] Support Amazon OpenSearch Service (managed, non-serverless) with SigV4/IAM auth

Open
#407 0 comments 0 reactions 1 assignee Claimed by @noel-improv View on GitHub
enhancement
Dominant language
Python
Stars
442
Forks
106
Avg merge
2d 3h
Merged PRs (30d)
41

Description

### Package

lexical-graph

### Problem statement

The `opensearch://` vector store (added in #397) connects to OpenSearch endpoints that don't use AWS authentication — HTTP basic auth from `OPENSEARCH_USERNAME`/`OPENSEARCH_PASSWORD`, or no auth. It's been validated against a localhost Docker/Finch container. Two deployment shapes aren't covered yet:

- A managed **Amazon OpenSearch Service** domain (not Serverless) that authenticates with IAM/SigV4.
- Any auth beyond basic/none (e.g. mutual TLS).

Raised in review of #397 (@acarbonetto): OpenSearch can run well beyond localhost (EC2, on-prem, managed Service), and we should support more of those once the localhost path is in.

### Proposed solution

Decouple the two concerns the current single flag ties together:

- **Auth mechanism** — `is_sigv4_auth` selects AWS SigV4 vs basic/no auth in `create_os_client` / `create_os_async_client`.
- **AOSS document handling** — the same flag currently also sets the llama-index client's `is_aoss` (custom `id` field, no post-ingest `refresh()`), via `DummyAuth(service='aoss')`.

A managed Amazon OpenSearch Service domain needs SigV4 auth **with** non-AOSS document handling (`_id` + `refresh()`), so these can't stay coupled. Scope:

- Separate the SigV4-auth signal from the AOSS-behavior signal so a managed OpenSearch Service domain can use SigV4 without AOSS document semantics.
- Add a connection path / config for managed Amazon OpenSearch Service (IAM/SigV4), distinct from both `aoss://` and the basic-auth `opensearch://`.
- Decide how the endpoint scheme is expressed (opensearch-py accepts an endpoint without an explicit `http://`/`https://`); document the default.
- Tests for the managed-Service auth path and document handling.

### Alternatives considered

Leave `opensearch://` as basic-auth/no-auth only and document managed Amazon OpenSearch Service as unsupported (current state in #397). Works for self-hosted OpenSearch but excludes managed non-serverless domains.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.