explosion / explosion/spacy-stanza

Can't use Spacy-Stanza in a databricks/spark UDF

Open
#86 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
748
Forks
63
PR merge metrics
No merged PRs in 30d

Description

I am using Spacy in a pyspark dataframe in a UDF. Regular Spacy works fine, but with spacy-stanza I get a 'pickling error', which suggests to me that the library is not serializable.
Here's the error: [PicklingError: Could not serialize object: TypeError: can't pickle _thread.RLock objects

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.