NVIDIA / NVIDIA/GenerativeAIExamples

Frontend app works ok with Port forwarded localhost endpoint but fails with an ingress endpoint

Abierto
#254 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Lenguaje dominante
Jupyter Notebook
Estrellas
4.2k
Forks
1.1k
Merge medio
10 h 15 min
PR fusionados (30 d)
1

Descripción

I am able to access the chatbot front end using port forwarded local host ep.
Now I want to expose this endpoint using Ingress. I have added an ingress to expose the chatbot’s front end using the below manifest
e125830@W76CYQCV44 nim-operator % cat cr8-rag-chatbot-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: rag-chatbot-ep
namespace: rag-sample
spec:
ingressClassName: nginx
rules:

  • host: rag-chatbot-ep.k8s-performance.stage.stl.k8s.int
    http:
    paths:
    • backend:
      service:
      name: rag-playground-multiturn-rag
      port:
      number: 3001
      path: /
      pathType: Prefix
      tls:
  • hosts:
    • rag-chatbot-ep.k8s-performance.stage.stl.k8s.int

I am able to get to the chatbot front end using the new ingress ep https://rag-chatbot-ep.k8s-performance.stage.stl.k8s.int/converse, but when I submit a prompt I get error
Below are the logs and my analysis…

Below are the logs when Chatbot is accessed via nginx ingress ep
[1] Nov 01 17:08:10 [ INFO] - uvicorn.access - 10.42.69.131:37658 - "GET /converse HTTP/1.1" 200
[1] Nov 01 17:08:10 [ INFO] - uvicorn.access - 10.42.69.131:37658 - "GET /content/converse/?__theme=light HTTP/1.1" 200
[1] Nov 01 17:08:10 [ INFO] - uvicorn.access - 10.42.69.131:37658 - "GET /assets/index-CKyT5UbA.css HTTP/1.1" 404
[1] Nov 01 17:08:10 [ INFO] - uvicorn.access - 10.42.69.131:37658 - "GET /theme.css?v=a2458843f5ddb2d155d001fdae495187847b01c05a840ddb350be829a269d776 HTTP/1.1" 404
[1] Nov 01 17:18:22 [ INFO] - uvicorn.access - 10.42.171.194:36704 - "POST /queue/join?__theme=light HTTP/1.1" 405

Below are the logs when Chatbot is accessed via port forwarded local host ep
[1] Nov 01 17:08:29 [ INFO] - uvicorn.access - 127.0.0.1:57842 - "GET /converse HTTP/1.1" 200
[1] Nov 01 17:08:29 [ INFO] - uvicorn.access - 127.0.0.1:57842 - "GET /content/converse/?__theme=light HTTP/1.1" 200
[1] Nov 01 17:08:30 [ INFO] - uvicorn.access - 127.0.0.1:57842 - "GET /assets/index-CKyT5UbA.css HTTP/1.1" 404
[1] Nov 01 17:08:30 [ INFO] - uvicorn.access - 127.0.0.1:57842 - "GET /content/converse/theme.css?v=a2458843f5ddb2d155d001fdae495187847b01c05a840ddb350be829a269d776 HTTP/1.1" 200
[1] Nov 01 17:20:12 [ INFO] - uvicorn.access - 127.0.0.1:52396 - "POST /content/converse/queue/join?__theme=light HTTP/1.1" 200
[1] Nov 01 17:20:12 [ INFO] - frontend.pages.converse - processing inference request - {'prompt': 'Testing', 'use_knowledge_base': False}
[1] Nov 01 17:20:12 [ INFO] - uvicorn.access - 127.0.0.1:52396 - "GET /content/converse/queue/data?session_hash=rdgwzvtp9w8 HTTP/1.1" 200

Main issue is the POST call is missing /content/converse in the ingress ep case

Looks like the the URL posted by the Client itself does not have them for ingress case…

Can one of you please check this and let us know what might be causing this difference in the POST URL’s when using the above endpoints…

Guía de contribución

Abrir la guía de contribución

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Línea de trabajo

Comienza con cr8-rag-chatbot-ingress.yaml y compara las rutas de solicitud de ingress con los logs obtenidos mediante port forwarding. Sigue cómo el frontend construye la URL POST para /queue/join y verifica que el endpoint de ingress conserve el prefijo /content/converse; se considera completado cuando enviar un prompt mediante ingress funciona igual que mediante port forwarding.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
kubernetes, nginx
Área
frontend, infrastructure
Tipo de issue
Error
Dificultad
3/5
Tiempo estimado
1-2 días
Estado de actividad
Estancado
Claridad
Bastante claro
Aptitud para principiantes
35/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.