Angel-ML / Angel-ML/angel

ERROR transport.MatrixTransportClient: hanlder rpc response failed

Ouverte
#1,044 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
Java
Étoiles
6.8k
Forks
1.6k
Merge moyen
44 min
PR mergées (30 j)
1

Description

Environment:

- Java version: 1.8

- Scala version: 2.11.12

- Spark version: 2.4.7

- Angel version: 3.1.0

- OS and version: Linux version 3.10.0-514.el7.x86_64

Problem description:
When I call the PSMatrix pull operation in map operation of RDD, I will cause the card master to be calculated

Yarn application Logs:
20/11/23 16:00:22 ERROR transport.MatrixTransportClient: hanlder rpc response failed
java.lang.ArrayIndexOutOfBoundsException: 4096
at it.unimi.dsi.fastutil.ints.Int2IntOpenHashMap.rehash(Int2IntOpenHashMap.java:1073)
at it.unimi.dsi.fastutil.ints.Int2IntOpenHashMap.insert(Int2IntOpenHashMap.java:321)
at it.unimi.dsi.fastutil.ints.Int2IntOpenHashMap.put(Int2IntOpenHashMap.java:327)
at com.tencent.angel.psagent.matrix.transport.adapter.GetRowsFlowCache.getRowPartSize(GetRowsFlowCache.java:85)
at com.tencent.angel.psagent.matrix.transport.adapter.GetRowsFlowCache.addSubResponse(GetRowsFlowCache.java:107)
at com.tencent.angel.psagent.matrix.transport.adapter.GetRowsFlowCache.addSubResponse(GetRowsFlowCache.java:35)
at com.tencent.angel.psagent.matrix.transport.adapter.UserRequestAdapter.notifyResponse(UserRequestAdapter.java:692)
at com.tencent.angel.psagent.matrix.transport.MatrixTransportClient$Responser.handleGetRowsSplitResponse(MatrixTransportClient.java:2496)
at com.tencent.angel.psagent.matrix.transport.MatrixTransportClient$Responser.run(MatrixTransportClient.java:2271)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/11/23 16:00:22 ERROR transport.MatrixTransportClient: hanlder rpc response failed
java.lang.ArrayIndexOutOfBoundsException: 4096
at it.unimi.dsi.fastutil.ints.Int2IntOpenHashMap.rehash(Int2IntOpenHashMap.java:1073)
at it.unimi.dsi.fastutil.ints.Int2IntOpenHashMap.insert(Int2IntOpenHashMap.java:321)
at it.unimi.dsi.fastutil.ints.Int2IntOpenHashMap.put(Int2IntOpenHashMap.java:327)
at com.tencent.angel.psagent.matrix.transport.adapter.GetRowsFlowCache.getRowPartSize(GetRowsFlowCache.java:85)
at com.tencent.angel.psagent.matrix.transport.adapter.GetRowsFlowCache.addSubResponse(GetRowsFlowCache.java:107)
at com.tencent.angel.psagent.matrix.transport.adapter.GetRowsFlowCache.addSubResponse(GetRowsFlowCache.java:35)
at com.tencent.angel.psagent.matrix.transport.adapter.UserRequestAdapter.notifyResponse(UserRequestAdapter.java:692)
at com.tencent.angel.psagent.matrix.transport.MatrixTransportClient$Responser.handleGetRowsSplitResponse(MatrixTransportClient.java:2496)
at com.tencent.angel.psagent.matrix.transport.MatrixTransportClient$Responser.run(MatrixTransportClient.java:2271)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/11/23 16:00:22 ERROR transport.MatrixTransportClient: hanlder rpc response failed
java.lang.ArrayIndexOutOfBoundsException: 4096
at it.unimi.dsi.fastutil.ints.Int2IntOpenHashMap.rehash(Int2IntOpenHashMap.java:1073)
at it.unimi.dsi.fastutil.ints.Int2IntOpenHashMap.insert(Int2IntOpenHashMap.java:321)
at it.unimi.dsi.fastutil.ints.Int2IntOpenHashMap.put(Int2IntOpenHashMap.java:327)
at com.tencent.angel.psagent.matrix.transport.adapter.GetRowsFlowCache.getRowPartSize(GetRowsFlowCache.java:85)
at com.tencent.angel.psagent.matrix.transport.adapter.GetRowsFlowCache.addSubResponse(GetRowsFlowCache.java:107)
at com.tencent.angel.psagent.matrix.transport.adapter.GetRowsFlowCache.addSubResponse(GetRowsFlowCache.java:35)
at com.tencent.angel.psagent.matrix.transport.adapter.UserRequestAdapter.notifyResponse(UserRequestAdapter.java:692)
at com.tencent.angel.psagent.matrix.transport.MatrixTransportClient$Responser.handleGetRowsSplitResponse(MatrixTransportClient.java:2496)
at com.tencent.angel.psagent.matrix.transport.MatrixTransportClient$Responser.run(MatrixTransportClient.java:2271)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/11/23 16:00:22 INFO memory.MemoryStore: Block rdd_123_29 stored as values in memory (estimated size 86.0 KB, free 905.1 MB)
20/11/23 16:00:22 INFO executor.Executor: Finished task 29.0 in stage 30.0 (TID 3334). 875 bytes result sent to driver
20/11/23 16:00:22 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 3335
20/11/23 16:00:22 INFO executor.Executor: Running task 124.0 in stage 30.0 (TID 3335)
20/11/23 16:00:22 INFO storage.BlockManager: Found block rdd_121_124 locally
20/11/23 16:00:22 ERROR transport.MatrixTransportClient: hanlder rpc response failed
java.lang.ArrayIndexOutOfBoundsException: -1
at it.unimi.dsi.fastutil.ints.Int2IntOpenHashMap.rehash(Int2IntOpenHashMap.java:1066)
at it.unimi.dsi.fastutil.ints.Int2IntOpenHashMap.insert(Int2IntOpenHashMap.java:321)
at it.unimi.dsi.fastutil.ints.Int2IntOpenHashMap.put(Int2IntOpenHashMap.java:327)
at com.tencent.angel.psagent.matrix.transport.adapter.GetRowsFlowCache.getRowPartSize(GetRowsFlowCache.java:85)
at com.tencent.angel.psagent.matrix.transport.adapter.GetRowsFlowCache.addSubResponse(GetRowsFlowCache.java:107)
at com.tencent.angel.psagent.matrix.transport.adapter.GetRowsFlowCache.addSubResponse(GetRowsFlowCache.java:35)
at com.tencent.angel.psagent.matrix.transport.adapter.UserRequestAdapter.notifyResponse(UserRequestAdapter.java:692)
at com.tencent.angel.psagent.matrix.transport.MatrixTransportClient$Responser.handleGetRowsSplitResponse(MatrixTransportClient.java:2496)
at com.tencent.angel.psagent.matrix.transport.MatrixTransportClient$Responser.run(MatrixTransportClient.java:2271)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) ...
20/11/23 16:00:22 ERROR transport.MatrixTransportClient: hanlder rpc response failed
java.lang.ArrayIndexOutOfBoundsException
20/11/23 16:00:22 ERROR transport.MatrixTransportClient: hanlder rpc response failed
java.lang.ArrayIndexOutOfBoundsException
20/11/23 16:00:22 ERROR transport.MatrixTransportClient: hanlder rpc response failed
java.lang.ArrayIndexOutOfBoundsException
20/11/23 16:00:22 ERROR transport.MatrixTransportClient: hanlder rpc response failed
java.lang.ArrayIndexOutOfBoundsException
20/11/23 16:00:22 ERROR transport.MatrixTransportClient: hanlder rpc response failed
java.lang.ArrayIndexOutOfBoundsException
20/11/23 16:00:22 ERROR transport.MatrixTransportClient: hanlder rpc response failed
java.lang.ArrayIndexOutOfBoundsException
20/11/23 16:00:22 ERROR transport.MatrixTransportClient: hanlder rpc response failed
java.lang.ArrayIndexOutOfBoundsException
20/11/23 16:00:22 ERROR transport.MatrixTransportClient: hanlder rpc response failed
java.lang.ArrayIndexOutOfBoundsException
20/11/23 16:00:22 ERROR transport.MatrixTransportClient: hanlder rpc response failed
java.lang.ArrayIndexOutOfBoundsException
20/11/23 16:00:22 ERROR transport.MatrixTransportClient: hanlder rpc response failed
java.lang.ArrayIndexOutOfBoundsException
20/11/23 16:00:22 ERROR transport.MatrixTransportClient: hanlder rpc response failed
java.lang.ArrayIndexOutOfBoundsException
20/11/23 16:00:22 ERROR transport.MatrixTransportClient: hanlder rpc response failed
java.lang.ArrayIndexOutOfBoundsException
20/11/23 16:00:22 ERROR transport.MatrixTransportClient: hanlder rpc response failed
java.lang.ArrayIndexOutOfBoundsException
20/11/23 16:00:22 ERROR transport.MatrixTransportClient: hanlder rpc response failed
java.lang.ArrayIndexOutOfBoundsException
20/11/23 16:00:22 ERROR transport.MatrixTransportClient: hanlder rpc response failed
java.lang.ArrayIndexOutOfBoundsException
20/11/23 16:00:22 ERROR transport.MatrixTransportClient: hanlder rpc response failed
java.lang.ArrayIndexOutOfBoundsException
20/11/23 16:00:22 ERROR transport.MatrixTransportClient: hanlder rpc response failed
java.lang.ArrayIndexOutOfBoundsException
20/11/23 16:00:22 ERROR transport.MatrixTransportClient: hanlder rpc response failed
java.lang.ArrayIndexOutOfBoundsException

I don't know why such a mistake occurred

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.