JuliaPy / JuliaPy/PyCall.jl

enhance jlfun2pyfun to use explicit (non-varargs) arguments when possible

Đang mở
#795 3 bình luận 0 reaction 0 người được giao Xem trên GitHub
enhancement
Ngôn ngữ chính
Julia
Star
1.5k
Fork
186
Chỉ số merge pull request
Không có pull request nào được merge trong 30 ngày

Mô tả

Hi everyone,
I am currently designing Scikit Learn estimators using Julia functions with the help of the @PyDef macro.
The purpose of it is being able to integrate my Julia code into the ScikitLearn's interface for pipelines, gird search, etc...

I followed the tutorial from this homepage and everything worked fine up to a point.
I was able to instantiate my custom estimator, integrate it in a pipeline and call fit, transform, predict, etc... on it.

More information on [Scikit Learn's estimator and how to design a custom one](https://scikit-learn.org/stable/developers/develop.html).

Here is a toy example of a custom estimator using a Julia function:

```
using ScikitLearn, PyCall, Statistics
using ScikitLearn.Pipelines: Pipeline
@sk_import base: BaseEstimator
@sk_import base: TransformerMixin

# Define a function in Julia to put into a ScikitLearn estimator
f(X, a, b) = @. a * X + b

"""
A Dummy transformer that applies the function f to the data
"""
@pydef mutable struct FunctionTransformer <: (TransformerMixin, BaseEstimator)
function __init__(self, b)
self.b = b
end

function fit(self, X, y=nothing)
self.a_ = mean(X)
return self
end

function transform(self, X)
return f(X, self.a_, self.b)
end

function fit_transform(self, X, y)
self.fit(X, y)
return self.transform(X)
end
end
```

Here is the part which is working:
```
# Try the transformer
X = zeros(Int, 3, 2)
transformer = FunctionTransformer(-1)
fit!(transformer, X)
transform(transformer, X)

# Try it in a pipeline
pipe = Pipeline([("Dummy", transformer)])
fit!(transformer, X)
transform(transformer, X)
```

The problem occured while I tried to use the get_params() (necessary to use the estimator in a grid search) method on my estimator. It is supposed to be inherited from the BaseEstimator class and the developper shouldn't worry about it. However, I encountered the following error:

```
# Try geting params
transformer.get_params()
```

> ERROR: PyError ($(Expr(:escape, :(ccall(#= C:\Users\user\.julia\packages\PyCall\zqDXB\src\pyfncall.jl:43 =# @pysym(:PyObject_Call), PyPtr, (PyPtr, PyPtr, PyPtr), o, pyargsptr, kw)))))
RuntimeError("scikit-learn estimators should always specify their parameters in the signature of their __init__ (no varargs). with constructor (*args, **kwargs) doesn't follow this convention.")
File "C:\Users\user\AppData\Local\Programs\Python\Python37\lib\site-packages\sklearn\base.py", line 205, in get_params
for key in self._get_param_names():
File "C:\Users\user\AppData\Local\Programs\Python\Python37\lib\site-packages\sklearn\base.py", line 185, in _get_param_names
% (cls, init_signature))
Stacktrace:
[1] pyerr_check at C:\Users\user\.julia\packages\PyCall\zqDXB\src\exception.jl:60 [inlined]
[2] pyerr_check at C:\Users\user\.julia\packages\PyCall\zqDXB\src\exception.jl:64 [inlined]
[3] _handle_error(::String) at C:\Users\user\.julia\packages\PyCall\zqDXB\src\exception.jl:81
[4] macro expansion at C:\Users\user\.julia\packages\PyCall\zqDXB\src\exception.jl:95 [inlined]
[5] #110 at C:\Users\user\.julia\packages\PyCall\zqDXB\src\pyfncall.jl:43 [inlined]
[6] disable_sigint at .\c.jl:446 [inlined]
[7] __pycall! at C:\Users\user\.julia\packages\PyCall\zqDXB\src\pyfncall.jl:42 [inlined]
[8] _pycall!(::PyObject, ::PyObject, ::Tuple{}, ::Int64, ::Ptr{Nothing}) at C:\Users\user\.julia\packages\PyCall\zqDXB\src\pyfncall.jl:29
[9] #call#117 at C:\Users\user\.julia\packages\PyCall\zqDXB\src\pyfncall.jl:11 [inlined]
[10] (::PyObject)() at C:\Users\user\.julia\packages\PyCall\zqDXB\src\pyfncall.jl:86
[11] top-level scope at REPL[20]:1
[12] eval(::Module, ::Any) at .\boot.jl:330
[13] eval_user_input(::Any, ::REPL.REPLBackend) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.2\REPL\src\REPL.jl:86
[14] run_backend(::REPL.REPLBackend) at C:\Users\user\.julia\packages\Revise\Pcs5V\src\Revise.jl:1073
[15] top-level scope at REPL[1]:0

Reading the Python's error message, it seemed to me that the problem came from the Python class' constructor's, which had been automatically created thanks to PyDef. The get_params() function expect a class whose constructor params are all specified, no *args and no **kwargs.
In my julian definition of the constructor, there are no positional nor optional arguments.
I went for a little check to get the Python class' constructor params:
```
py"""
import inspect
cls = $FunctionTransformer
print(inspect.signature(getattr(cls.__init__, 'deprecated_original', cls.__init__)))
"""
```
> (*args, **kwargs)

But it seems that the produced Python class' constructor has, causing this issue!

I could, off course, skip this problem by re-defining my own get_params() method, but this would on one hand be very tedious since I have several estimators each having multiple parameters to get, and on the other hand, this slight detail migth cause non-solvable problems for someone else.

Thanks by advance for the help :)

Hướng dẫn đóng góp

Chưa lập chỉ mục được hướng dẫn đóng góp cho kho mã nguồn này

Hướng nghiên cứu

Bắt đầu từ entry point jlfun2pyfun được @PyDef sử dụng và tái hiện vấn đề với ví dụ FunctionTransformer. Kiểm tra signature __init__ được tạo bằng inspect.signature và so sánh với yêu cầu get_params của scikit-learn; hoàn thành khi các constructor có các đối số Julia tường minh không còn để lộ *args và **kwargs.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Đánh giá

Công nghệ
julia, python
Lĩnh vực
tooling
Loại issue
Tính năng
Độ khó
4/5
Thời gian dự kiến
3-5 ngày
Mức độ hoạt động
Đình trệ
Độ rõ ràng
Khá rõ ràng
Mức phù hợp với người mới
45/100

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.