[Bug] kafka_fdw foreign table returns zero rows: Feature not supported: Foreign Data
- Dominant language
- C
- Stars
- 1.4k
- Forks
- 247
- Avg merge
- 4d 3h
- Merged PRs (30d)
- 39
Description
### Apache Cloudberry version
PostgreSQL 14.4 (Cloudberry Database 1.0.0 build dev) on x86_64-pc-linux-gnu, compiled by gcc (GCC) 11.5.0 20240719 (Red Hat 11.5.0-5), 64-bit compiled on Mar 24 2025 12:34:11 (with assert checkin
g)
### What happened
Description:
I have successfully created and configured the kafka_fdw extension and a foreign table to read from a Kafka topic. However, when querying the foreign table, it returns zero rows with the following log message:
`NOTICE: "Feature not supported: Foreign Data"`
This seems to indicate the foreign table was created correctly but data retrieval is blocked or unsupported by the executor.
Environment:
OS: Docker container
Kafka: 5.5.0 (also tried 7.4.0)
kafka_fdw: latest build from GitHub
Database: CloudberryDB / GreenplumDB
FDW server created with broker localhost:9092
Additional Context:
Kafka topic contrib_regress is successfully created. Using command-line tools, I can publish and consume messages. Seems like the FDW is skipping execution due to unsupported features.
Please advise on how to enable execution or whether a fix is available.
### What you think should happen instead
I think cloudberry has to get data from foreign database system.
### How to reproduce
Steps to Reproduce:
Installed and enabled kafka_fdw:
`CREATE EXTENSION kafka_fdw;`
Set up Kafka/Zookeeper using Docker:
version: '2'
networks:
test:
services:
zookeeper:
image: confluentinc/cp-zookeeper:5.5.0
ports: [ "2181:2181" ]
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
kafka:
image: confluentinc/cp-kafka:5.5.0
ports: [ "9092:9092" ]
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://192.168.222.51:9092
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
Created server and foreign table:
CREATE SERVER kafka_server
FOREIGN DATA WRAPPER kafka_fdw
OPTIONS (brokers 'localhost:9092');
CREATE USER MAPPING FOR PUBLIC SERVER kafka_server;
CREATE USER MAPPING FOR gpadmin SERVER kafka_server;
CREATE FOREIGN TABLE kafka_test (
part int OPTIONS (partition 'true'),
offs bigint OPTIONS (offset 'true'),
some_int int,
some_text text,
some_date date,
some_time timestamp
)
SERVER kafka_server
OPTIONS (format 'csv', topic 'contrib_regress', batch_size '30', buffer_delay '100');
Ran query:
`SELECT * FROM kafka_test LIMIT 1;`
Expected Behavior:
Should return rows from the Kafka topic contrib_regress.
Actual Behavior:
Returns zero rows with notice:
`NOTICE: "Feature not supported: Foreign Data"`
### Operating System
OS: Docker container Linux mdw 4.18.0-553.el8_10.x86_64 #1 SMP Fri May 24 13:05:10 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes, I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/cloudberry/blob/main/CODE_OF_CONDUCT.md).
Contributor guide
Research direction
Start by reproducing SELECT * FROM kafka_test LIMIT 1 against the kafka_fdw foreign table and inspect the foreign-table execution path that emits “Feature not supported: Foreign Data.” Compare the Cloudberry executor’s handling of this query with kafka_fdw’s expected retrieval behavior; done means rows from the contrib_regress Kafka topic are returned without that notice.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kafka, postgresql
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100