lyhue1991 / lyhue1991/PythonAiRoad

有一处不是不是很明白?

Open
#1 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Jupyter Notebook
Stars
415
Forks
99
PR merge metrics
No merged PRs in 30d

Description

您好,我在看 [DBSCAN算法的Spark实现.ipynb](https://github.com/lyhue1991/PythonAiRoad/blob/master/DBSCAN%E7%AE%97%E6%B3%95%E7%9A%84Spark%E5%AE%9E%E7%8E%B0.ipynb) 代码时候,有一处不是很明白,希望可以解答一下。
文中在第八步骤
```python
/*================================================================================*/
// 八,求每一个簇的代表核心和簇元素数量
/*================================================================================*/
....
val rdd_result = rdd_cluster.reduceByKey((a,b)=>{
val id_set = a._3 | b._3 # 不明白
val result = if(a._2>=b._2) (a._1,a._2,id_set)
else (b._1,b._2,id_set)
result
})
...
```
在这里,`rdd_cluster`进行了按照键的归并操作,但是rdd_cluster的已经是聚类的结果了,就说明`rdd_cluster`的键都是唯一的,没有重复的,所以我认为这一步骤`reduceByKey`是无用操作,是吗?
第二,如果我的第一的疑问有问题,那么成功运行了`reduceByKey`的逻辑,但是在`reduceByKey`的代码中,对相同类的邻居集合进行了并集`|`操作,为什么邻居数量`result._2`取的不是`id_set`的长度,而是`Max(a._2,b._2)`?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read DBSCAN算法的Spark实现.ipynb, starting at step eight and the construction of rdd_cluster before reduceByKey. Trace whether keys can repeat and how the tuple fields are defined. Done means the notebook clearly explains the aggregation and the relationship between the merged id_set and the reported count.

Written by the indexing model from the issue text.

Assessment

Tech stack
spark
Domain
machine-learning
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.