adobe / adobe/experience-platform-streaming-connect

Task is being killed and will not recover until manually restarted. Error: java.lang.NullPointerException after 407 proxy error

Open
#73 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
25
Forks
25
PR merge metrics
No merged PRs in 30d

Description

### Subject of the issue

when a proxy error happens the AEP sink connector flow dies with this message error.

```
{"debug_level":"WARN","debug_timestamp":"2024-06-19 04:36:09,350","debug_thread":"task-thread-onboarding-adobe-sink-1","debug_file":"HttpConnection.java", "debug_line":"161","debug_message":"attempt 1 of 3 failed with exception - Unable to tunnel through proxy. Proxy returns "HTTP/1.1 407 Proxy authentication required""}
{"debug_level":"ERROR","debug_timestamp":"2024-06-19 04:36:09,465","debug_thread":"task-thread-xxxx-adobe-sink-1","debug_file":"WorkerSinkTask.java", "debug_line":"623","debug_message":"WorkerSinkTask{id=xxxx-adobe-sink-1} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted. Error: java.lang.NullPointerException"}
```

### Your environment
Kafka confluentinc/cp-server-connect:7.4.5 with adobeinc-streaming-connect-sink v0.0.16

### Steps to reproduce
not sure how to reproduce, we have detected after proxy authentication happened
with this config

```
configs:
topics: "my_orig_topic"
tasks.max: "2"
aep.flush.interval.seconds: "1"
aep.flush.bytes.kb: "4"
key.converter: "org.apache.kafka.connect.storage.StringConverter"
value.converter: "org.apache.kafka.connect.json.JsonConverter"
key.converter.schemas.enable: "false"
value.converter.schemas.enable: "false"
aep.endpoint: "https://dcs.adobedc.net/collection/xxxxxxxxxxxxxxxxxxxxxx"
aep.connection.auth.enabled: "true"
aep.connection.auth.token.type: "jwt_token"
aep.connection.auth.client.id: "xxxxxxxxxxxxxxx"
aep.connection.auth.imsOrg: "xxxxxxxxxxxxxxxx"
aep.connection.auth.accountKey: "xxxxxxxxxxxxxxxx"
aep.connection.auth.filePath: "xxxxxxxxxxxxxxxxxx"
aep.connection.auth.endpoint: "https:/xxxxxxx.adobelogin.com"
aep.connection.endpoint.headers: "{xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx}"
aep.connection.auth.client.secret: "xxxxxxxxxxxxxxxxxxxx"
aep.connection.proxy.host: "proxy.myorg.com"
aep.connection.proxy.user: "my_user_proxy"
aep.connection.proxy.password: "my_user_password"
aep.connection.proxy.port: "8383"
errors.tolerance: "all"
errors.deadletterqueue.topic.name: "my_topic_error"
errors.log.enable: "true"
errors.log.include.messages: "true"
errors.retry.timeout: "-1"
errors.retry.delay.max.ms: "180000"
```

### Expected behaviour

We excpect connector trying reconnect several times until proxy error disapears.

### Actual behaviour

Connector taks stops until process restart.

Contributor guide

Open the contributing guide

Research direction

Start by inspecting HttpConnection.java around line 161 and trace how the proxy's HTTP 407 failure reaches the connector task. Use the supplied proxy settings and error logs as the starting case; done means the connector continues retrying after proxy authentication errors instead of stopping with a NullPointerException and requiring a process restart.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.