Blizzard / Blizzard/node-rdkafka

Major difference in size between v2.14.0 and v.15.0

Open
#1,030 2 comments 1 reaction 0 assignees View on GitHub
Dominant language
JavaScript
Stars
2.2k
Forks
403
PR merge metrics
No merged PRs in 30d

Description

**Environment Information**
- OS [Alpine]:
- Node Version [16.15.1]:
- NPM Version [8.11.0]:
- node-rdkafka version [2.14.0]:

**Steps to Reproduce**
when upgrading the library from v2.13 to v2.15 we were unable to deploy to lambda as the size of the package increased from 150MB to 340 MB. The reason we need this update: when using version 2.13 we get the following error on publishing:

Unsupported value "SASL_SSL" for configuration property "security.protocol": OpenSSL not available at build time

with the following dockerfile config (where the npm ci is taking place):
```

FROM public.ecr.aws/lambda/nodejs:16.2022.05.23.09

RUN yum update -y && \
yum install -y \
zip \
bash \
python3 \
g++ \
gcc \
libgcc-5-dev \
libstdc++6 \
linux-headers-$(uname -r) \
gcc-c++ \
make \
openssl-dev \
openssl \
cyrus-sasl-dev \
which \
ca-certificates \
lz4-dev \
musl-dev

ENTRYPOINT []
```

and producer config for auth looks like this:
```
{
'security.protocol': 'sasl_ssl' as const,
'sasl.mechanism': 'SCRAM-SHA-512',
'sasl.username': 'user',
'sasl.password': 'pass'
}
```

I have noticed there was an update of the librdkafka to v2.0.2 in : https://github.com/Blizzard/node-rdkafka/releases/tag/v2.15.0 so i suspect this is what may have fixed this openssl issue, how could we go around this issue while still being able to upload to lambda. The package is so large that i am unable to upload it neither as a lambda layer.

Contributor guide

Open the contributing guide

Research direction

Compare the installed package contents and native dependencies between node-rdkafka 2.13 and 2.15 using the Dockerfile and Lambda environment described in the issue. Start with the librdkafka 2.0.2 update and the package-size change; done means explaining the increase and identifying a way to deploy the required SASL_SSL configuration within Lambda limits.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, docker, javascript, nodejs
Domain
build-system, cloud, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.