derrickoswald / derrickoswald/CIMSpark
Joining two VertexPartitions with different indexes is slow.
- Dominant language
- Scala
- Stars
- 19
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
In performing the network topology processing to create the TopologyNode and TopologyIsland RDD, numerous messages about _joining VertexPartitions with different indexes is slow_ are logged:
```
WARN impl.ShippableVertexPartitionOps: Joining two VertexPartitions with different indexes is slow.
```
It is unclear which operations are causing these messages or if they can be avoided. Google searches indicate that the VertexRDD that are used should be cached, but preliminary attempts to cache generated VertexRDD had no effect. Other processing which may be at fault is to update the ACDCTerminal, and ConnectivityNode RDD including their superclasses.
An attempt should be made to understand the origin of these messages and ameliorate them if possible.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.