Support Managed Knowledge Bases in BedrockCreateKnowledgeBaseOperator
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 472
Description
### Description
Amazon Bedrock Managed Knowledge Bases reached general availability on 2026-06-17. It is a fully managed RAG offering: Bedrock owns the vector store, the ingestion pipeline and the retrieval infrastructure, so there is no self-managed vector store to configure.
It is created through the same `bedrock-agent` `CreateKnowledgeBase` operation as the existing self-managed base. The only differences are the shape of `KnowledgeBaseConfiguration` and the absence of `storageConfiguration`:
```python
bedrock_agent.create_knowledge_base(
name='my-managed-kb',
roleArn=role_arn,
knowledgeBaseConfiguration={
'type': 'MANAGED',
'managedKnowledgeBaseConfiguration': {'embeddingModeType': 'MANAGED'},
}
}
```
`BedrockCreateKnowledgeBaseOperato` currently cannot produce this call:
1. `knowledgeBaseConfiguration` is hardcorded in `execute()`:
````python
knowledge_base_configuration={
'type': 'VECTOR',
'vectorKnowledgeBaseConfiguration': {'embeddingModelArn': self.embedding_model_arn},
}
````
2. `storageConfiguration=self.storage_config` is always passed to the API call.
3. `embedding_model_arn` and `storage_config` are required `__init__` arguments, and neither applies to a managed knowledge base.
Overrading via `create_knowledge_nase_kwards` does not work either. Both `knoeledgeBaseConfiguration` and `storageConfiguration` are already supplied as explicit keyword arguments, so sypplying them again through the kwargs dict raises `TypeError: got multiple values for keyword argument`.
Separaetly, the `wait_for_indexing` / "no such index" retry loop exists purely to work around vector index propagation in a self-managed vector store. It has no meaning for a managed knowledge base, since there is no index for the user to create.
Net effect: there is currently no way to create a managed knowledge base from the amazon provider.
References:
- https://aws.amazon.com/about-aws/whats-new/2026/06/amazon-bedrock-managed-knowledge-base/
- https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent_CreateKnowledgeBase.html
- https://aws.amazon.com/blogs/machine-learning/build-enterprise-search-for-agents-with-amazon-bedrock-managed-knowledge-base/
### Use case/motivation
Refreshing a RAF corpus is a scheduled data pipeline concern: the source documents are usually produced or updated by upstream Airflow tasks, so provisioning the knowledge base and driving ingestion belongs in the same DAG. The provider already covers the self-managed path end to end(`BedrockCreateKnowledgeBaseOperator` -> `BedrockCreateDataSourceOperator` -> `BedrockIngestDataOperator` -> `BedrockKnowledgeBaseActiveSensor`). Managed knowledge bases are the missing half of an otherwise complete story, and they are the lower-friction option for teams that do not want to own vector srore.
### Related issues
Not found.
### Are you willing to submit a PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start with BedrockCreateKnowledgeBaseOperator's __init__ and execute() methods, then compare their arguments and call behavior with the linked Bedrock CreateKnowledgeBase API reference. Done means the operator can create both managed and self-managed knowledge bases, while managed bases avoid self-managed storage settings and the vector-index retry behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cloud, data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 67/100