submit a job , and the job has not been ran。

openpai/openpai-runtime container on pai-work node error log：

root@pai-worker1:/usr/local# docker logs 6e354aae0a40
+ CHILD_PROCESS=UNKNOWN
+ trap exit_handler EXIT
+ PAI_WORK_DIR=/usr/local/pai
+ PAI_CONFIG_DIR=/usr/local/pai-config
+ PAI_INIT_DIR=/usr/local/pai/init.d
+ PAI_RUNTIME_DIR=/usr/local/pai/runtime.d
+ PAI_LOG_DIR=/usr/local/pai/logs/2e4061a5-244c-45af-8ede-b4b9f6604774
+ PAI_SECRET_DIR=/usr/local/pai/secrets
+ PAI_USER_EXTENSION_SECRET_DIR=/usr/local/pai/user-extension-secrets
+ PAI_TOKEN_SECRET_DIR=/usr/local/pai/token-secrets
+ chmod a+rw /usr/local/pai/logs/2e4061a5-244c-45af-8ede-b4b9f6604774
+ find /usr/local/pai/logs/2e4061a5-244c-45af-8ede-b4b9f6604774 -maxdepth 1 -type f '!' -name init.log
+ LOG_FILES=
+ '[[' '!' -z  ]]
+ find /usr/local/pai -maxdepth 1 -mindepth 1 '!' -name logs -exec rm -rf '{}' ';'
rm: can't remove '/usr/local/pai/user-extension-secrets/..data': Read-only file system
rm: can't remove '/usr/local/pai/user-extension-secrets/userExtensionSecrets.yaml': Read-only file system
rm: can't remove '/usr/local/pai/user-extension-secrets/..2021_11_18_08_54_30.144092891/userExtensionSecrets.yaml': Read-only file system
+ mv ./__init__.py ./common ./init ./init.d ./package_cache ./plugins ./requirements.txt ./runtime ./runtime.d /usr/local/pai
+ cd /usr/local/pai
+ '[[' true '=' true ]]
+ CHILD_PROCESS=FRAMEWORK_BARRIER
+ echo 'frameworkbarrier start'
frameworkbarrier start
+ '[[' -f /var/run/secrets/kubernetes.io/serviceaccount/token ]]
+ unset KUBE_APISERVER_ADDRESS
+ /usr/local/pai/init.d/frameworkbarrier
+ tee /usr/local/pai/logs/2e4061a5-244c-45af-8ede-b4b9f6604774/barrier.log
I1118 08:54:33.933533      14 barrier.go:211] Initializing frameworkbarrier
I1118 08:54:33.933705      14 barrier.go:214] With Config:
kubeApiServerAddress: ""
kubeConfigFilePath: ""
frameworkNamespace: default
frameworkName: 3a3cada7aa94eff914b372e17e17c8e1
barrierCheckIntervalSec: 10
barrierCheckTimeoutSec: 600
W1118 08:54:33.933736      14 client_config.go:549] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I1118 08:54:33.934851      14 barrier.go:227] Running frameworkbarrier
W1118 08:54:33.945092      14 barrier.go:253] Failed to get Framework object from ApiServer: Unauthorized
W1118 08:54:43.947234      14 barrier.go:253] Failed to get Framework object from ApiServer: Unauthorized
W1118 08:54:53.947552      14 barrier.go:253] Failed to get Framework object from ApiServer: Unauthorized
W1118 08:55:03.947109      14 barrier.go:253] Failed to get Framework object from ApiServer: Unauthorized
W1118 08:55:13.947432      14 barrier.go:253] Failed to get Framework object from ApiServer: Unauthorized

webport log

Start with the openpai-runtime container logs and the init.d/frameworkbarrier entry point, focusing on the repeated Kubernetes ApiServer “Unauthorized” response and the service-account token configuration. Reproduce the job submission and verify that the frameworkbarrier can authenticate and the job proceeds past the barrier.