MetaCell / MetaCell/cloud-harness

Listener to services should be on a separate thread and resilient by default

Open
#682 2 comments 0 reactions 0 assignees View on GitHub
scope:python-runtime
Dominant language
Python
Stars
19
Forks
5
Avg merge
3d 18h
Merged PRs (30d)
3

Description

Generated applications for django have a default behaviour to sync users which requires kafka and accounts to be fully operative at startup. If they're not fully operative, pods go on a crashloop.
This is not ideal because the user facing part of the application does not necessarily requires user login functionality.

Ideally we should set this kind of listeners on a separate pod/container.

A quicker proposal is the following:

```
if os.environ.get('KUBERNETES_SERVICE_HOST', None):
# init the auth service when running in/for k8s
import threading
from cloudharness_django.services import get_auth_service, init_services
from cloudharness import log
import time
def start_auth_service():
try:
init_services()
except:
log.exception("Error initializing services. Retrying in 5 seconds...")
time.sleep(5)
start_auth_service()

threading.Thread(target=start_auth_service).start()
# start the kafka event listener when running in/for k8s
def start_event_listener():
try:
import cloudharness_django.services.events
log.info("User sync events listener started")
except:
log.exception("Error initializing event queue. Retrying in 5 seconds...")
time.sleep(5)
start_event_listener()
threading.Thread(target=start_event_listener).start()
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the generated Django application's startup path and the auth-service and event-listener initialization mentioned in the issue. Compare the separate pod/container approach with resilient background initialization, then verify that the user-facing application starts without Kafka or accounts being fully available and that failed initialization retries without causing a crash loop.

Written by the indexing model from the issue text.

Assessment

Tech stack
django, kafka, kubernetes, python
Domain
authentication, backend, distributed-systems, infrastructure
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.