baidu / baidu/lac

一个困扰了10天的问题! python3+lac竟然会遇到编码问题

Open
#259 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
4k
Forks
592
PR merge metrics
No merged PRs in 30d

Description

在本地用python3.7.1_gcc82_pd2.0rc1_cpu测试是成功的,通过lac对文本进行分词,也输出正常的字符串
image

但是在spark集群上,使用相同版本的python3.7.1_gcc82_pd2.0rc1_cpu通过spark集群调用失败,debug输出的结果是乱码
image
单独输出原始字符就是正常的,这就代表是lac.run(text)输出结果为乱码,希望能够有相应解答~

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the difference between local execution and the Spark-cluster execution using the stated Python and LAC versions. Start at the lac.run(text) call and inspect how its returned text is decoded or displayed in the cluster; done means the segmentation output remains correctly encoded under Spark.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, spark
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.