microsoft / microsoft/pai

Error of job's log-manager and stdout/stderr

Open
#4,816 20 comments 0 reactions 1 assignee View on GitHub

@Binyang2014 is already working on this.

Since Aug 18, 2020.

DRI raised by customer
Dominant language
JavaScript
Stars
2.7k
Forks
554
Avg merge
20h 42m
Merged PRs (30d)
14

Description

Organization Name:

Short summary about the issue/question:
Cannot access log-manager (through the button browser log folder, error 502 Bad Gateway after about 1-5 minutes of page loading) and stdout/stderr (only showing "Loading..." forever). Found "log-manager-xxx" pods are running correctly. The logs are like x.x.x.x - [17/Aug/2020:03:20:18 +0000] "GET /healthz HTTP/1.1" 200 18 "-" "kube-probe/1.15". All other functionalities seem to work correctly (job submission / running / ...).

Brief what process you are following:

How to reproduce it: not sure (but I reinstalled (after kubeadm reset on all machines) the OpenPAI, and this problem was still there)

OpenPAI Environment:

  • OpenPAI version: v1.0
  • Cloud provider or hardware configuration: local cluster
  • OS (e.g. from /etc/os-release): PRETTY_NAME="Ubuntu 18.04.4 LTS"
  • Kernel (e.g. uname -a): 5.4.0-42-generic #46~18.04.1-Ubuntu SMP Fri Jul 10 07:21:24 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
  • Hardware (e.g. core number, memory size, storage size, GPU type etc.):
  • Others:

Anything else we need to know:
The firewall is off on all nodes

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.