LINE算法模型训练相关问题咨询

样本数据量：大概9千万个节点，node_id（0~90000000），机器内存251G，64核，1ps + 5worker

bash dist_tf_euler.sh --data_dir hdfs://path/data --euler_zk_addr IP:2181 --euler_zk_path /euler --max_id 90000000 --model line --mode train --model_dir ./model --learning_rate 0.01 --dim 512

1、运行line算法模型时，设置训练参数： --max_id 90000000，--dim 512，抛出如下错误：
2019-04-28 09:53:19.518423: W tensorflow/core/framework/allocator.cc:124] Allocation of 184231307264 exceeds 10% of system memory.
2019-04-28 09:53:19.518480: W tensorflow/core/framework/op_kernel.cc:1401] OP_REQUIRES failed at random_op.cc:202 : Resource exhausted: OOM when allocating tensor with shape[89956693,512] and type float on /job:ps/replica:0/task:0/device:CPU:0 by allocator cpu
请问这个问题也没有办法解决？

2、将dim设置为--dim 12时，某个worker打印日志如下：
INFO:tensorflow:loss = 2129.7446, mrr = 0.40621746, step = 0
INFO:tensorflow:loss = 14377.218, mrr = 0.44117844, step = 84 (232.371 sec)
INFO:tensorflow:loss = 127506900.0, mrr = 0.4401042, step = 187 (256.154 sec)
......
通过观察发现，ps进程占用的cpu达到4000%左右，而对应的worker占用cpu不到1%，即所有压力落在的ps上，请问如何提升worker的计算能力及降低ps的压力？
另外，我们将ps数增大为3个时，即3ps + 5worker，主ps占用的cpu依旧很大（4000%左右），而其他两个ps几乎不占用cpu，并且worker占用cpu也不到1%，请问这如何解释？

3、在问题2的条件下（2ps + 5worker），运行时会出现如下问题：
INFO:tensorflow:loss = 2128.9492, mrr = 0.4132162, step = 0
INFO:tensorflow:loss = 144028.48, mrr = 0.41656902, step = 88 (227.890 sec)
INFO:tensorflow:loss = 21924712.0, mrr = 0.43326825, step = 195 (236.491 sec)
INFO:tensorflow:An error was raised. This may be due to a preemption in a connected worker or parameter server. The current session will be closed and a new session will be created. This error may also occur due to a gRPC failure caused by high memory or network bandwidth usage in the parameter servers. If this error occurs repeatedly, try increasing the number of parameter servers assigned to the job. Error: Socket closed
请问这是什么原因引起的？

@yangsiran 谢谢解答~

Start with the dist_tf_euler.sh invocation and the reported TensorFlow logs for the LINE training run. Investigate the OOM at shape [89956693,512], the uneven ps and worker CPU usage, and the Socket closed error under the stated ps/worker configurations. Done means identifying the causes and documenting or implementing a verified resolution for these three conditions.

Written by the indexing model from the issue text.