awslabs / awslabs/amazon-kinesis-client-python
ERROR s.a.k.multilang.DrainChildSTDERRTask [NONE] - Received error line from subprocess for shard shardId-000000000000
- Dominant language
- Python
- Stars
- 377
- Forks
- 228
- PR merge metrics
- No merged PRs in 30d
Description
I am using the KCL library in Python and logging the information as below:
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(name)
logger.info("some logger info message")
From the above code, I am expecting to display the information as INFO in Log file, but it is showing as below:
[multi-lang-daemon-0001] ERROR s.a.k.multilang.DrainChildSTDERRTask [NONE] - Received error line from subprocess [INFO:some logger info message] for shard shardId-000000000000 some logger info message
It is logging the message twice. I have looked at the DrainChildSTDERRTask.HandleLineResult handleLine(String line) and understand that it logging to logger and JVM.
@Override protected HandleLineResult handleLine(String line) { log.error("Received error line from subprocess [{}] for shard {}", line, getShardId()); System.err.println(line); return new HandleLineResult(); }
I have two questions:
Why my logger.info is going to DrainChildSTDERRTask class though it is not an error?
Is logging twice necessary as it is filling the log quickly when it processes loads of streaming messages?
Not sure if this is an error in the KCL library or I am missing something when setting up the configuration in the python code.
Also in the sample_kcl_app you are writing to the sys.stderr is that the expected behaviour?
def log(self, message): sys.stderr.write(message)
It would be great if someone can help with this one!
Contributor guide
Research direction
Start with DrainChildSTDERRTask.handleLine(String line) and the Python logging.basicConfig setup described in the report. Reproduce the output with sample_kcl_app, including its sys.stderr logging, then determine whether the duplicate ERROR output is expected and what configuration or library behavior should change.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, python
- Domain
- stream-processing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100