Zipstack / Zipstack/unstract

fix: [ISSUE] Unable to configure unstract (mock_org/onboard): unable to submit form

Open
#1,189 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

question
Dominant language
Python
Stars
7.2k
Forks
718
Avg merge
3d 1h
Merged PRs (30d)
23

Description

Describe the bug

after loggin in, in "you're 4 steps away from nirvana', test is ok but when i press submit button it displays a popup :

Image

Image

Image

for every configurations

To reproduce

docker installation with docker compose.

VERSION=dev docker compose -f docker-compose.build.yaml build
VERSION=dev docker compose -f docker-compose.yaml up -d

installed on a server X (not localhost)
frontend.unstract.mydomain.intr declared in dns

updates of backend .env:

DJANGO_APP_BACKEND_URL=http://frontend.unstract.mydomain.intr


LOGIN_NEXT_URL="http://frontend.unstract.mydomain.intr/org"
LANDING_URL="http://frontend.unstract.mydomain.intr/landing"
ERROR_URL="http://frontend.unstract.mydomain.intr/error"
WEB_APP_ORIGIN_URL="http://frontend.unstract.mydomain.intr"


# Default user auth credentials
DEFAULT_AUTH_USERNAME="xxxx"
DEFAULT_AUTH_PASSWORD="yyyy"

using proxy_overrides.yaml:

http:
  routers:
    frontend:
      service: frontend
      rule: Host(`frontend.unstract.mydomain.intr`)
    backend:
      service: backend
      rule: Host(`frontend.unstract.mydomain.intr`) && PathPrefix(`/api/v1`, `/deployment`)

  services:
    frontend:
      loadBalancer:
        servers:
        # Point to service running on docker host.
        - url: http://host.docker.internal:3000/
        passHostHeader: true
    backend:
      loadBalancer:
        servers:
        # Point to service running on docker host.
        - url: http://host.docker.internal:8000/
        passHostHeader: true

Expected behavior

configuration should be accepted and written

Environment details

  • Version: dev (15 march 2025)

Additional context

Logs:

unstract-backend                | [2025-03-16 11:27:08 +0000] [19] [DEBUG] GET /api/v1/socket/
unstract-proxy                  | 172.16.99.219 - - [16/Mar/2025:11:27:08 +0000] "GET /api/v1/socket/?EIO=4&transport=websocket HTTP/1.1" 400 25 "-" "-" 8894 "backend@file" "http://host.docker.internal:8000/" 5ms
unstract-backend                | 172.20.0.1 - - [16/Mar/2025:11:27:08 +0000] "GET /api/v1/socket/?EIO=4&transport=websocket HTTP/1.1" 400 25 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36"
unstract-celery-beat            | INFO : [2025-03-16 11:27:11,001]{module:base process:1 thread:135973213555584 request_id:none} :- DB connection (ID: 135972218967680) is established or reused.
unstract-backend                | [2025-03-16 11:27:11 +0000] [19] [DEBUG] POST /api/v1/unstract/mock_org/adapter/
unstract-backend                | INFO : [2025-03-16 11:27:11,499]{module:base process:19 thread:137646667990720 request_id:35a14ba52f5941af84137c0cb9c7c5be} :- DB connection (ID: 137646002629504) is established or reused.
unstract-backend                | ERROR : [2025-03-16 11:27:11,504]{module:exception process:19 thread:137646667990720 request_id:35a14ba52f5941af84137c0cb9c7c5be} :- POST http://frontend.unstract.mydomain.intr/api/v1/unstract/mock_org/adapter/ 500
unstract-backend                | 
unstract-backend                | ValueError('Fernet key must be 32 url-safe base64-encoded bytes.')
unstract-backend                | 
unstract-backend                | ````Traceback (most recent call last):
unstract-backend                |   File "/app/.venv/lib/python3.9/site-packages/cryptography/fernet.py", line 34, in __init__
unstract-backend                |     key = base64.urlsafe_b64decode(key)
unstract-backend                |   File "/usr/local/lib/python3.9/base64.py", line 133, in urlsafe_b64decode
unstract-backend                |     return b64decode(s)
unstract-backend                |   File "/usr/local/lib/python3.9/base64.py", line 87, in b64decode
unstract-backend                |     return binascii.a2b_base64(s)
unstract-backend                | binascii.Error: Incorrect padding
unstract-backend                | 
unstract-backend                | The above exception was the direct cause of the following exception:
unstract-backend                | 
unstract-backend                | Traceback (most recent call last):
unstract-backend                |   File "/app/.venv/lib/python3.9/site-packages/rest_framework/views.py", line 506, in dispatch
unstract-backend                |     response = handler(request, *args, **kwargs)
unstract-backend                |   File "/app/adapter_processor_v2/views.py", line 180, in create
unstract-backend                |     serializer.is_valid(raise_exception=True)
unstract-backend                |   File "/app/.venv/lib/python3.9/site-packages/rest_framework/serializers.py", line 227, in is_valid
unstract-backend                |     self._validated_data = self.run_validation(self.initial_data)
unstract-backend                |   File "/app/.venv/lib/python3.9/site-packages/rest_framework/serializers.py", line 426, in run_validation
unstract-backend                |     value = self.to_internal_value(data)
unstract-backend                |   File "/app/adapter_processor_v2/serializers.py", line 51, in to_internal_value
unstract-backend                |     f: Fernet = Fernet(encryption_secret.encode("utf-8"))
unstract-backend                |   File "/app/.venv/lib/python3.9/site-packages/cryptography/fernet.py", line 36, in __init__
unstract-backend                |     raise ValueError(
unstract-backend                | ValueError: Fernet key must be 32 url-safe base64-encoded bytes.
unstract-backend                | ````
unstract-backend                | Internal Server Error: /api/v1/unstract/mock_org/adapter/
unstract-backend                | Traceback (most recent call last):
unstract-backend                |   File "/app/.venv/lib/python3.9/site-packages/cryptography/fernet.py", line 34, in __init__
unstract-backend                |     key = base64.urlsafe_b64decode(key)
unstract-backend                |   File "/usr/local/lib/python3.9/base64.py", line 133, in urlsafe_b64decode
unstract-backend                |     return b64decode(s)
unstract-backend                |   File "/usr/local/lib/python3.9/base64.py", line 87, in b64decode
unstract-backend                |     return binascii.a2b_base64(s)
unstract-backend                | binascii.Error: Incorrect padding
unstract-backend                | 
unstract-backend                | The above exception was the direct cause of the following exception:
unstract-backend                | 
unstract-backend                | Traceback (most recent call last):
unstract-backend                |   File "/app/.venv/lib/python3.9/site-packages/django/core/handlers/exception.py", line 55, in inner
unstract-backend                |     response = get_response(request)
unstract-backend                |   File "/app/.venv/lib/python3.9/site-packages/django/core/handlers/base.py", line 197, in _get_response
unstract-backend                |     response = wrapped_callback(request, *callback_args, **callback_kwargs)
unstract-backend                |   File "/app/.venv/lib/python3.9/site-packages/django/views/decorators/csrf.py", line 56, in wrapper_view
unstract-backend                |     return view_func(*args, **kwargs)
unstract-backend                |   File "/app/.venv/lib/python3.9/site-packages/rest_framework/viewsets.py", line 125, in view
unstract-backend                |     return self.dispatch(request, *args, **kwargs)
unstract-backend                |   File "/app/.venv/lib/python3.9/site-packages/rest_framework/views.py", line 509, in dispatch
unstract-backend                |     response = self.handle_exception(exc)
unstract-backend                |   File "/app/.venv/lib/python3.9/site-packages/rest_framework/views.py", line 469, in handle_exception
unstract-backend                |     self.raise_uncaught_exception(exc)
unstract-backend                |   File "/app/.venv/lib/python3.9/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
unstract-backend                |     raise exc
unstract-backend                |   File "/app/.venv/lib/python3.9/site-packages/rest_framework/views.py", line 506, in dispatch
unstract-backend                |     response = handler(request, *args, **kwargs)
unstract-backend                |   File "/app/adapter_processor_v2/views.py", line 180, in create
unstract-backend                |     serializer.is_valid(raise_exception=True)
unstract-backend                |   File "/app/.venv/lib/python3.9/site-packages/rest_framework/serializers.py", line 227, in is_valid
unstract-backend                |     self._validated_data = self.run_validation(self.initial_data)
unstract-backend                |   File "/app/.venv/lib/python3.9/site-packages/rest_framework/serializers.py", line 426, in run_validation
unstract-backend                |     value = self.to_internal_value(data)
unstract-backend                |   File "/app/adapter_processor_v2/serializers.py", line 51, in to_internal_value
unstract-backend                |     f: Fernet = Fernet(encryption_secret.encode("utf-8"))
unstract-backend                |   File "/app/.venv/lib/python3.9/site-packages/cryptography/fernet.py", line 36, in __init__
unstract-backend                |     raise ValueError(
unstract-backend                | ValueError: Fernet key must be 32 url-safe base64-encoded bytes.
unstract-backend                | ERROR : [2025-03-16 11:27:11,560]{module:log process:19 thread:137646667990720 request_id:35a14ba52f5941af84137c0cb9c7c5be} :- Internal Server Error: /api/v1/unstract/mock_org/adapter/
unstract-backend                | Traceback (most recent call last):
unstract-backend                |   File "/app/.venv/lib/python3.9/site-packages/cryptography/fernet.py", line 34, in __init__
unstract-backend                |     key = base64.urlsafe_b64decode(key)
unstract-backend                |   File "/usr/local/lib/python3.9/base64.py", line 133, in urlsafe_b64decode
unstract-backend                |     return b64decode(s)
unstract-backend                |   File "/usr/local/lib/python3.9/base64.py", line 87, in b64decode
unstract-backend                |     return binascii.a2b_base64(s)
unstract-backend                | binascii.Error: Incorrect padding
unstract-backend                | 
unstract-backend                | The above exception was the direct cause of the following exception:
unstract-backend                | 
unstract-backend                | Traceback (most recent call last):
unstract-backend                |   File "/app/.venv/lib/python3.9/site-packages/django/core/handlers/exception.py", line 55, in inner
unstract-backend                |     response = get_response(request)
unstract-backend                |   File "/app/.venv/lib/python3.9/site-packages/django/core/handlers/base.py", line 197, in _get_response
unstract-backend                |     response = wrapped_callback(request, *callback_args, **callback_kwargs)
unstract-backend                |   File "/app/.venv/lib/python3.9/site-packages/django/views/decorators/csrf.py", line 56, in wrapper_view
unstract-backend                |     return view_func(*args, **kwargs)
unstract-backend                |   File "/app/.venv/lib/python3.9/site-packages/rest_framework/viewsets.py", line 125, in view
unstract-backend                |     return self.dispatch(request, *args, **kwargs)
unstract-backend                |   File "/app/.venv/lib/python3.9/site-packages/rest_framework/views.py", line 509, in dispatch
unstract-backend                |     response = self.handle_exception(exc)
unstract-backend                |   File "/app/.venv/lib/python3.9/site-packages/rest_framework/views.py", line 469, in handle_exception
unstract-backend                |     self.raise_uncaught_exception(exc)
unstract-backend                |   File "/app/.venv/lib/python3.9/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
unstract-backend                |     raise exc
unstract-backend                |   File "/app/.venv/lib/python3.9/site-packages/rest_framework/views.py", line 506, in dispatch
unstract-backend                |     response = handler(request, *args, **kwargs)
unstract-backend                |   File "/app/adapter_processor_v2/views.py", line 180, in create
unstract-backend                |     serializer.is_valid(raise_exception=True)
unstract-backend                |   File "/app/.venv/lib/python3.9/site-packages/rest_framework/serializers.py", line 227, in is_valid
unstract-backend                |     self._validated_data = self.run_validation(self.initial_data)
unstract-backend                |   File "/app/.venv/lib/python3.9/site-packages/rest_framework/serializers.py", line 426, in run_validation
unstract-backend                |     value = self.to_internal_value(data)
unstract-backend                |   File "/app/adapter_processor_v2/serializers.py", line 51, in to_internal_value
unstract-backend                |     f: Fernet = Fernet(encryption_secret.encode("utf-8"))
unstract-backend                |   File "/app/.venv/lib/python3.9/site-packages/cryptography/fernet.py", line 36, in __init__
unstract-backend                |     raise ValueError(
unstract-backend                | ValueError: Fernet key must be 32 url-safe base64-encoded bytes.
unstract-backend                | 172.20.0.1 - - [16/Mar/2025:11:27:11 +0000] "POST /api/v1/unstract/mock_org/adapter/ HTTP/1.1" 500 145556 "http://frontend.unstract.mydomain.intr/mock_org/onboard" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36"
unstract-proxy                  | 172.16.99.6 - - [16/Mar/2025:11:27:11 +0000] "POST /api/v1/unstract/mock_org/adapter/ HTTP/1.1" 500 145556 "-" "-" 8895 "backend@file" "http://host.docker.internal:8000/" 87ms
unstract-backend                | [2025-03-16 11:27:11 +0000] [19] [DEBUG] POST /api/v1/unstract/mock_org/logs/
unstract-backend                | INFO : [2025-03-16 11:27:11,611]{module:base process:19 thread:137646540064448 request_id:94bec21c5c3f465da0392a5fc3d9147e} :- DB connection (ID: 137646002629504) is established or reused.
unstract-proxy                  | 172.16.99.6 - - [16/Mar/2025:11:27:11 +0000] "POST /api/v1/unstract/mock_org/logs/ HTTP/1.1" 200 54 "-" "-" 8896 "backend@file" "http://host.docker.internal:8000/" 18ms
unstract-backend                | 172.20.0.1 - - [16/Mar/2025:11:27:11 +0000] "POST /api/v1/unstract/mock_org/logs/ HTTP/1.1" 200 54 "http://frontend.unstract.mydomain.intr/mock_org/onboard" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36"
unstract-backend                | [2025-03-16 11:27:11 +0000] [19] [DEBUG] GET /api/v1/socket/
unstract-proxy                  | 172.16.99.6 - - [16/Mar/2025:11:27:11 +0000] "GET /api/v1/socket/?EIO=4&transport=websocket HTTP/1.1" 400 25 "-" "-" 8897 "backend@file" "http://host.docker.internal:8000/" 6ms
unstract-backend                | 172.20.0.1 - - [16/Mar/2025:11:27:11 +0000] "GET /api/v1/socket/?EIO=4&transport=websocket HTTP/1.1" 400 25 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36"
unstract-backend                | [2025-03-16 11:27:13 +0000] [19] [DEBUG] GET /api/v1/socket/
unstract-proxy                  | 172.16.99.6 - - [16/Mar/2025:11:27:13 +0000] "GET /api/v1/socket/?EIO=4&transport=websocket HTTP/1.1" 400 25 "-" "-" 8898 "backend@file" "http://host.docker.internal:8000/" 4ms
unstract-backend                | 172.20.0.1 - - [16/Mar/2025:11:27:13 +0000] "GET /api/v1/socket/?EIO=4&transport=websocket HTTP/1.1" 400 25 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36"
unstract-backend                | [2025-03-16 11:27:13 +0000] [19] [DEBUG] GET /api/v1/socket/
unstract-proxy                  | 172.16.99.219 - - [16/Mar/2025:11:27:13 +0000] "GET /api/v1/socket/?EIO=4&transport=websocket HTTP/1.1" 400 25 "-" "-" 8899 "backend@file" "http://host.docker.internal:8000/" 4ms
unstract-backend                | 172.20.0.1 - - [16/Mar/2025:11:27:13 +0000] "GET /api/v1/socket/?EIO=4&transport=websocket HTTP/1.1" 400 25 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36

Screenshots

If applicable, add screenshots to help explain your problem.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the submission through the Docker Compose setup, then inspect adapter_processor_v2/serializers.py at line 51 and adapter_processor_v2/views.py at line 180, alongside the backend environment configuration. Confirm which configuration value reaches Fernet and trace the request to /api/v1/unstract/mock_org/adapter/. Done means the form submits successfully and the configuration is written without a 500 response.

Written by the indexing model from the issue text.

Assessment

Tech stack
cryptography, django, docker-compose, python
Domain
api, backend, devops, security
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.