lyhue1991 / lyhue1991/PythonAiRoad
有一处不是不是很明白?
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 415
- Forks
- 99
- PR merge metrics
- No merged PRs in 30d
Description
您好,我在看 [DBSCAN算法的Spark实现.ipynb](https://github.com/lyhue1991/PythonAiRoad/blob/master/DBSCAN%E7%AE%97%E6%B3%95%E7%9A%84Spark%E5%AE%9E%E7%8E%B0.ipynb) 代码时候,有一处不是很明白,希望可以解答一下。
文中在第八步骤
```python
/*================================================================================*/
// 八,求每一个簇的代表核心和簇元素数量
/*================================================================================*/
....
val rdd_result = rdd_cluster.reduceByKey((a,b)=>{
val id_set = a._3 | b._3 # 不明白
val result = if(a._2>=b._2) (a._1,a._2,id_set)
else (b._1,b._2,id_set)
result
})
...
```
在这里,`rdd_cluster`进行了按照键的归并操作,但是rdd_cluster的已经是聚类的结果了,就说明`rdd_cluster`的键都是唯一的,没有重复的,所以我认为这一步骤`reduceByKey`是无用操作,是吗?
第二,如果我的第一的疑问有问题,那么成功运行了`reduceByKey`的逻辑,但是在`reduceByKey`的代码中,对相同类的邻居集合进行了并集`|`操作,为什么邻居数量`result._2`取的不是`id_set`的长度,而是`Max(a._2,b._2)`?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read DBSCAN算法的Spark实现.ipynb, starting at step eight and the construction of rdd_cluster before reduceByKey. Trace whether keys can repeat and how the tuple fields are defined. Done means the notebook clearly explains the aggregation and the relationship between the merged id_set and the reported count.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- spark
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100