awslabs / awslabs/amazon-kinesis-client-python

ERROR s.a.k.multilang.DrainChildSTDERRTask [NONE] - Received error line from subprocess for shard shardId-000000000000

Open
#116 2 comments 2 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
377
Forks
228
PR merge metrics
No merged PRs in 30d

Description

I am using the KCL library in Python and logging the information as below:

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(name)
logger.info("some logger info message")

From the above code, I am expecting to display the information as INFO in Log file, but it is showing as below:

[multi-lang-daemon-0001] ERROR s.a.k.multilang.DrainChildSTDERRTask [NONE] - Received error line from subprocess [INFO:some logger info message] for shard shardId-000000000000 some logger info message

It is logging the message twice. I have looked at the DrainChildSTDERRTask.HandleLineResult handleLine(String line) and understand that it logging to logger and JVM.

@Override protected HandleLineResult handleLine(String line) { log.error("Received error line from subprocess [{}] for shard {}", line, getShardId()); System.err.println(line); return new HandleLineResult(); }

I have two questions:

Why my logger.info is going to DrainChildSTDERRTask class though it is not an error?
Is logging twice necessary as it is filling the log quickly when it processes loads of streaming messages?

Not sure if this is an error in the KCL library or I am missing something when setting up the configuration in the python code.

Also in the sample_kcl_app you are writing to the sys.stderr is that the expected behaviour?
def log(self, message): sys.stderr.write(message)

It would be great if someone can help with this one!

Contributor guide

Open the contributing guide

Research direction

Start with DrainChildSTDERRTask.handleLine(String line) and the Python logging.basicConfig setup described in the report. Reproduce the output with sample_kcl_app, including its sys.stderr logging, then determine whether the duplicate ERROR output is expected and what configuration or library behavior should change.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, python
Domain
stream-processing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.