qdrant / qdrant/qdrant-helm

Make possible to add ingress object for each pod

Open
#85 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
191
Forks
98
Avg merge
2d 4h
Merged PRs (30d)
6

Description

Hi Qdrant team,

Our team is building automation around Qdrant storage deployed with this helm chart.
We're also using the snapshot APIs and for this we need to communicate with each pod directly.
We found an elegant way to access the cluster API and each pod individually by using existing k8s facilities.

In short we:

  • use cert-manager.io for issuing TLS certificats
  • use Nginx Ingress which allows rewriting target urls
  • we access cluster level API via https://<fqdn>/cluster
  • we access each pod API via https://<fqdn>/pod-<index>

How we did it:

  1. We use nginx ingress controller which allows to rewrite target url,
    we basically followed the answer from here. Our values contains this ingress definition:
ingress:
  annotations:
    cert-manager.io/cluster-issuer: production-cluster-issuer
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/rewrite-target: "/$2"
    nginx.ingress.kubernetes.io/use-regex: 'true'
  enabled: true
  hosts:
    - host: qdrant-service.example.com
      paths:
        - path: "/cluster(/|$)(.*)"
          pathType: Prefix
          servicePort: 6333
  tls:
    - hosts:
        - qdrant-service.example.com
      secretName: qdrant-service.example.com

This ingress definition produces Ingress object pointing to the ClusterIP service which targets the three pods:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: production-cluster-issuer
    kubernetes.io/ingress.class: nginx
    meta.helm.sh/release-name: qdrant
    meta.helm.sh/release-namespace: qdrant-service
    nginx.ingress.kubernetes.io/rewrite-target: /$2
    nginx.ingress.kubernetes.io/use-regex: "true"
....
spec:
  rules:
  - host: qdrant-service.example.com
    http:
      paths:
      - backend:
          service:
            name: qdrant-service
            port:
              number: 6333
        path: /cluster(/|$)(.*)
        pathType: Prefix
  tls:
  - hosts:
    - qdrant-service.example.com
    secretName: qdrant-service.example.com
status:
  loadBalancer:
    ingress:
    - hostname: qdrant-lb.example.com
  1. After we deploy the helm chart, we additionally create ClusterIP service object and Ingress object for each pod.
  • The ClusterIP service looks similarly to the one created by the helm chart, but in the selector section we add the pod-name label:
apiVersion: v1
kind: Service
....
  name: qdrant-service-0
  namespace: qdrant-service
spec:
....
  selector:
    app: qdrant
    app.kubernetes.io/instance: qdrant
    app.kubernetes.io/name: qdrant
    statefulset.kubernetes.io/pod-name: qdrant-service-0
  • Then the backend of the per-pod Ingress object points to corresponding service (quadrant-service-0) and also has rewriting configuration for pod-0 path.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: production-cluster-issuer
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/rewrite-target: "/$2"
    nginx.ingress.kubernetes.io/use-regex: 'true'
  labels:
    app: qdrant
    app.kubernetes.io/instance: qdrant
    app.kubernetes.io/name: qdrant
  name: qdrant-service-0
  namespace: qdrant-service
spec:
  rules:
    - host: qdrant-service.example.com
      http:
        paths:
          - backend:
              service:
                name: qdrant-service-0
                port:
                  number: 6333
            path: "/pod-0(/|$)(.*)"
            pathType: Prefix
  tls:
    - hosts:
        - qdrant-service.example.com
      secretName: qdrant-service.example.com  

Suggestion: It would be great if Qdrant helm chart supports to optionally define per pod Ingress objects.
This would allow users like us to quite easily make each pod accessible.

Thank you!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the Helm chart's existing ingress, service, and StatefulSet configuration, then compare the generated cluster-level resources with the per-pod Service and Ingress examples in the issue. Define what values would represent per-pod resources and verify that rendering the chart produces one pod-targeting Service and Ingress for each replica, with the requested paths and annotations.

Written by the indexing model from the issue text.

Assessment

Tech stack
helm, kubernetes, nginx
Domain
devops, infrastructure
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.