手工模式分布式训练TDM，scheduler报错产生core,麻烦帮忙看下问题,ps_synchronize_leave_op操作引起

启动scheduler和ps都正常，一旦启动worker进行tdm模型训练，scheduler会异常退出。
### 定位到是由worker的xdl.execute(xdl.ps_synchronize_leave_op(np.array(xdl.get_task_index(), dtype=np.int32)))代码引起的。
### 我看ps_synchronize_leave_op在SyncRunHook中都是和ps_synchronize_enter_op成对出现的.
如果在代码里面把ps_synchronize_enter_op也加上，运行是正常的。
### 看ps scheduler和synchook的代码，发现这两个op应该是用于控制同步的，并按照xdl使用文档中同步训练的方式中把ps_synchronize_leave_op换成了worker_report_finish_op，目前来看scheduler一切正常，难道这个train.py中的这句代码是写错了???求解答。

scheduler报错信息如下:
2019-10-21 09:33:54.043822: W /home/yue.song/XDL-OpenSource-master/xdl/ps-plus/ps-plus/scheduler/scheduler_impl.cc:443] Schedule a new op to save checkpoint ckpt-...................4
2019-10-21 09:33:54.044416: W /home/yue.song/XDL-OpenSource-master/xdl/ps-plus/ps-plus/scheduler/scheduler_impl.cc:357] Saving checkpoint ckpt-...................4
2019-10-21 09:33:54.147183: W /home/yue.song/XDL-OpenSource-master/xdl/ps-plus/ps-plus/scheduler/scheduler_impl.cc:661] server 1 finish save, status OK, waiting 1 more.
2019-10-21 09:33:54.147941: W /home/yue.song/XDL-OpenSource-master/xdl/ps-plus/ps-plus/scheduler/scheduler_impl.cc:661] server 0 finish save, status OK, waiting 0 more.
2019-10-21 09:33:54.148189: W /home/yue.song/XDL-OpenSource-master/xdl/ps-plus/ps-plus/scheduler/scheduler_impl.cc:359] Saving checkpoint ckpt-...................4, Get Status OK
### 2019-10-21 09:33:54.150978: E /home/yue.song/XDL-OpenSource-master/xdl/ps-plus/ps-plus/scheduler/synchronizer.cc:92] Receive token 2 from 0 while current_token_ is 3
### 2019-10-21 09:33:54.151007: /home/yue.song/XDL-OpenSource-master/xdl/ps-plus/ps-plus/scheduler/synchronizer.cc:96] Worker 0 not granted token, but it call leave with token 2, current token is 3
seastar binding cores: false
WARN 2019-10-21 09:32:13,033 seastar - Seastar compiled with default allocator, heap profiler not supported
WARN 2019-10-21 09:32:13,033 seastar - Seastar compiled with default allocator, heap profiler not supported
WARN 2019-10-21 09:32:13,034 seastar - Seastar compiled with default allocator, heap profiler not supported
WARN 2019-10-21 09:32:13,034 seastar - Seastar compiled with default allocator, heap profiler not supported
WARN 2019-10-21 09:32:13,034 seastar - Seastar compiled with default allocator, heap profiler not supported
WARN 2019-10-21 09:32:13,034 seastar - Seastar compiled with default allocator, heap profiler not supported
WARN 2019-10-21 09:32:13,035 seastar - Seastar compiled with default allocator, heap profiler not supported
WARN 2019-10-21 09:32:13,036 seastar - Seastar compiled with default allocator, heap profiler not supported
WARN 2019-10-21 09:32:13,038 [shard 0] seastar - Unable to set SCHED_FIFO scheduling policy for timer thread; latency impact possible. Try adding CAP_SYS_NICE
Established connection: server_id = 0 on cpu 0, server addr: 127.0.1.1:34132.
Established connection: server_id = 1 on cpu 1, server addr: 127.0.1.1:42854.
Aborting.
Backtrace:
0x00000000008efecc
0x00000000008efff0
0x00000000008f00c5
/lib/x86_64-linux-gnu/libpthread.so.0+0x000000000001138f
/lib/x86_64-linux-gnu/libc.so.6+0x0000000000035427
/lib/x86_64-linux-gnu/libc.so.6+0x0000000000037029
0x00000000006b74e3
0x00000000007ef0e2
0x00000000008167dc
0x000000000081696d
0x00000000006b7f54
/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0x00000000000b8c7f
/lib/x86_64-linux-gnu/libpthread.so.0+0x00000000000076b9
/lib/x86_64-linux-gnu/libc.so.6+0x000000000010741c
Aborted (core dumped)
cmd: /usr/local/lib/python2.7/dist-packages/xdl/python/utils/../../bin/ps -smem "333" -bc "False" -sp "zfs://localhost:2181" -sqps "15625" -snet "62" -r "scheduler" -sn "2" -cp "/home/mi/workspace/tdm/dist/data/checkpoint"
Traceback (most recent call last):
File "train.py", line 17, in
import xdl
File "/usr/local/lib/python2.7/dist-packages/xdl/__init__.py", line 16, in
from xdl.python.init import *
File "/usr/local/lib/python2.7/dist-packages/xdl/python/__init__.py", line 16, in
from xdl.python.init import *
File "/usr/local/lib/python2.7/dist-packages/xdl/python/init.py", line 44, in
from xdl.python.training.train_session import TrainSession, LoggerHook, SyncRunHook, SemiSyncRunHook, BarrierHook, WorkerFinishHook, QpsMetricsHook, MetricsHook, MetricsPrinterHook
File "/usr/local/lib/python2.7/dist-packages/xdl/python/training/train_session.py", line 21, in
from xdl.python.training.env import current_env, is_local_mode
File "/usr/local/lib/python2.7/dist-packages/xdl/python/training/env.py", line 235, in
_ENV = init_env()
File "/usr/local/lib/python2.7/dist-packages/xdl/python/training/env.py", line 233, in init_env
return DistributedEnv()
File "/usr/local/lib/python2.7/dist-packages/xdl/python/training/env.py", line 149, in __init__
self._start()
File "/usr/local/lib/python2.7/dist-packages/xdl/python/training/env.py", line 58, in _start
self._scheduler_do()
File "/usr/local/lib/python2.7/dist-packages/xdl/python/training/env.py", line 175, in _scheduler_do
bind_cores = self._bind_core)
File "/usr/local/lib/python2.7/dist-packages/xdl/python/utils/ps_utils.py", line 95, in run_ps_scheduler
bc=bind_cores)
File "/usr/local/lib/python2.7/dist-packages/xdl/python/utils/ps_utils.py", line 67, in run_ps_cmd
raise ValueError("Run cmd Error, cmd=[%s] exit_code=[%s]" % (cmd, ret))
ValueError: Run cmd Error, cmd=[/usr/local/lib/python2.7/dist-packages/xdl/python/utils/../../bin/ps -smem "333" -bc "False" -sp "zfs://localhost:2181" -sqps "15625" -snet "62" -r "scheduler" -sn "2" -cp "/home/mi/workspace/tdm/dist/data/checkpoint"] exit_code=[34304]

请帮忙定位下，万分感谢。

@songyue1104 @lovickie @yiling-dc

Start with train.py and xdl/python/training/train_session.py, especially SyncRunHook and the reported ps_synchronize_leave_op call. Then inspect ps-plus/ps-plus/scheduler/synchronizer.cc around the logged token checks and compare the worker_report_finish_op path. Done means the distributed TDM run no longer aborts the scheduler and the corrected synchronization behavior is validated.

Written by the indexing model from the issue text.