Blizzard / Blizzard/node-rdkafka
Major difference in size between v2.14.0 and v.15.0
- Dominant language
- JavaScript
- Stars
- 2.2k
- Forks
- 403
- PR merge metrics
- No merged PRs in 30d
Description
**Environment Information**
- OS [Alpine]:
- Node Version [16.15.1]:
- NPM Version [8.11.0]:
- node-rdkafka version [2.14.0]:
**Steps to Reproduce**
when upgrading the library from v2.13 to v2.15 we were unable to deploy to lambda as the size of the package increased from 150MB to 340 MB. The reason we need this update: when using version 2.13 we get the following error on publishing:
Unsupported value "SASL_SSL" for configuration property "security.protocol": OpenSSL not available at build time
with the following dockerfile config (where the npm ci is taking place):
```
FROM public.ecr.aws/lambda/nodejs:16.2022.05.23.09
RUN yum update -y && \
yum install -y \
zip \
bash \
python3 \
g++ \
gcc \
libgcc-5-dev \
libstdc++6 \
linux-headers-$(uname -r) \
gcc-c++ \
make \
openssl-dev \
openssl \
cyrus-sasl-dev \
which \
ca-certificates \
lz4-dev \
musl-dev
ENTRYPOINT []
```
and producer config for auth looks like this:
```
{
'security.protocol': 'sasl_ssl' as const,
'sasl.mechanism': 'SCRAM-SHA-512',
'sasl.username': 'user',
'sasl.password': 'pass'
}
```
I have noticed there was an update of the librdkafka to v2.0.2 in : https://github.com/Blizzard/node-rdkafka/releases/tag/v2.15.0 so i suspect this is what may have fixed this openssl issue, how could we go around this issue while still being able to upload to lambda. The package is so large that i am unable to upload it neither as a lambda layer.
Contributor guide
Research direction
Compare the installed package contents and native dependencies between node-rdkafka 2.13 and 2.15 using the Dockerfile and Lambda environment described in the issue. Start with the librdkafka 2.0.2 update and the package-size change; done means explaining the increase and identifying a way to deploy the required SASL_SSL configuration within Lambda limits.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, docker, javascript, nodejs
- Domain
- build-system, cloud, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100