[BUG] User defined graph analysis algorithm error
- Dominant language
- C++
- Stars
- 3.6k
- Forks
- 468
- Avg merge
- 29m
- Merged PRs (30d)
- 1
Description
**Describe the bug**
Using the official custom algorithm, an error occurred during operation
**To Reproduce**
Steps to reproduce the behavior:
code:
```python
from graphscope.analytical.udf.decorators import pregel
from graphscope.framework.app import AppAssets
# 装饰器, 定义顶点数据和消息数据的类型
@pregel(vd_type="double", md_type="double")
class SSSP_Pregel(AppAssets):
@staticmethod
def Init(v, context):
v.set_value(1000000000.0)
@staticmethod
def Compute(messages, v, context):
src_id = context.get_config(b"src")
cur_dist = v.value()
new_dist = 1000000000.0
if v.id() == src_id:
new_dist = 0
for message in messages:
new_dist = min(message, new_dist)
if new_dist < cur_dist:
v.set_value(new_dist)
for e_label_id in range(context.edge_label_num()):
edges = v.outgoing_edges(e_label_id)
for e in edges:
v.send(e.vertex(), new_dist + e.get_int(2))
v.vote_to_halt()
@staticmethod
def Combine(messages):
ret = 1000000000.0
for m in messages:
ret = min(ret, m)
return ret
```
error:
```
2023-04-17 17:36:24,719 [ERROR][utils:276]: Failed to run command: cmake . -DNETWORKX=ON -DCMAKE_PREFIX_PATH='/usr/local/lib/python3.8/dist-packages/graphscope.runtime;/usr/local/lib/python3.8/dist-packages/graphscope.runtime/openmpi' -DCYTHON_PREGEL_APP=ON -DENABLE_PREGEL_COMBINE=ON, error message is: CMake Error at /usr/local/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find Python (missing: Python_INCLUDE_DIRS Python_LIBRARIES
Development Development.Module Development.Embed) (found version "2.7.17")
Call Stack (most recent call first):
/usr/local/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
/usr/local/share/cmake-3.23/Modules/FindPython.cmake:561 (find_package_handle_standard_args)
CMakeLists.txt:156 (find_package)
2023-04-17 17:36:24,751 [ERROR][rpc:188]: Runstep failed with code: ANALYTICAL_ENGINE_INTERNAL_ERROR, message: Error occurred during RunStep, The traceback is: Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/gscoordinator/utils.py", line 361, in _compile_on_local
logger.debug(run_command(command, cwd=workdir))
File "/usr/local/lib/python3.8/dist-packages/gscoordinator/utils.py", line 281, in run_command
raise RuntimeError(f"Failed to run command: {args}, err: {err}")
RuntimeError: Failed to run command: cmake . -DNETWORKX=ON -DCMAKE_PREFIX_PATH='/usr/local/lib/python3.8/dist-packages/graphscope.runtime;/usr/local/lib/python3.8/dist-packages/graphscope.runtime/openmpi' -DCYTHON_PREGEL_APP=ON -DENABLE_PREGEL_COMBINE=ON, err: CMake Error at /usr/local/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find Python (missing: Python_INCLUDE_DIRS Python_LIBRARIES
Development Development.Module Development.Embed) (found version "2.7.17")
Call Stack (most recent call first):
/usr/local/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
/usr/local/share/cmake-3.23/Modules/FindPython.cmake:561 (find_package_handle_standard_args)
CMakeLists.txt:156 (find_package)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/gscoordinator/coordinator.py", line 309, in _RunStep
head, bodies = self._operation_executor.run_on_analytical_engine(
File "/usr/local/lib/python3.8/dist-packages/gscoordinator/monitor.py", line 193, in runOnAnalyticalEngineWarp
res = func(instance, dag_def, dag_bodies, loader_op_bodies)
File "/usr/local/lib/python3.8/dist-packages/gscoordinator/op_executor.py", line 167, in run_on_analytical_engine
dag_def, dag_bodies = self.pre_process(dag_def, dag_bodies, loader_op_bodies)
File "/usr/local/lib/python3.8/dist-packages/gscoordinator/op_executor.py", line 144, in pre_process
op, _, _ = self._maybe_compile_app(op)
File "/usr/local/lib/python3.8/dist-packages/gscoordinator/op_executor.py", line 257, in _maybe_compile_app
compiled_path = self._compile_lib_and_distribute(
File "/usr/local/lib/python3.8/dist-packages/gscoordinator/op_executor.py", line 395, in _compile_lib_and_distribute
lib_path, java_jar_path, java_ffi_path, app_type = compile_func(
File "/usr/local/lib/python3.8/dist-packages/gscoordinator/utils.py", line 540, in compile_app
lib_path = compile_library(commands, library_dir, library_name, launcher)
File "/usr/local/lib/python3.8/dist-packages/gscoordinator/utils.py", line 311, in compile_library
return _compile_on_local(commands, workdir, output_name)
File "/usr/local/lib/python3.8/dist-packages/gscoordinator/utils.py", line 368, in _compile_on_local
raise CompilationError(
graphscope.framework.errors.CompilationError: Failed to compile f50bd1a6e93c0147cc21f356cd7595dbe1530149785c69a5b2b72efc7146b635 on platform system: Linux
machine: x86_64
platform: Linux-4.15.0-200-generic-x86_64-with-glibc2.27
uname: uname_result(system='Linux', node='c9c8c977cf55', release='4.15.0-200-generic', version='#211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022', machine='x86_64', processor='x86_64')
kernel_ver: #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022
mac_ver: ('', ('', '', ''), '')
gcc_ver: b'gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0\nCopyright (C) 2017 Free Software Foundation, Inc.\nThis is free software; see the source for copying conditions. There is NO\nwarranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.\n\n'
python_ver: 3.8.14
Traceback (most recent call last):
File "/root/demo/code/graph_analysis.py", line 29, in
sssp_context = my_alg_pie(simple_graph, src=6)
File "/usr/local/lib/python3.8/dist-packages/graphscope/analytical/udf/wrapper.py", line 63, in call
return app_assets(graph, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/graphscope/framework/app.py", line 321, in __call__
app_ = graph.session._wrapper(AppDAGNode(graph, self))
File "/usr/local/lib/python3.8/dist-packages/graphscope/client/session.py", line 941, in _wrapper
return self.run(dag_node)
File "/usr/local/lib/python3.8/dist-packages/graphscope/client/session.py", line 977, in run
return self.run_fetches(fetches)
File "/usr/local/lib/python3.8/dist-packages/graphscope/client/session.py", line 987, in run_fetches
response = self._grpc_client.run(fetch_handler.targets)
File "/usr/local/lib/python3.8/dist-packages/graphscope/client/rpc.py", line 98, in run
return self._run_step_impl(dag_def)
File "/usr/local/lib/python3.8/dist-packages/graphscope/client/utils.py", line 156, in with_grpc_catch
return fn(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/graphscope/client/rpc.py", line 198, in _run_step_impl
raise exc
graphscope.framework.errors.CompilationError: Failed to compile f50bd1a6e93c0147cc21f356cd7595dbe1530149785c69a5b2b72efc7146b635 on platform system: Linux
machine: x86_64
platform: Linux-4.15.0-200-generic-x86_64-with-glibc2.27
uname: uname_result(system='Linux', node='c9c8c977cf55', release='4.15.0-200-generic', version='#211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022', machine='x86_64', processor='x86_64')
kernel_ver: #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022
mac_ver: ('', ('', '', ''), '')
gcc_ver: b'gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0\nCopyright (C) 2017 Free Software Foundation, Inc.\nThis is free software; see the source for copying conditions. There is NO\nwarranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.\n\n'
python_ver: 3.8.14
```
**Environment (please complete the following information):**
- GraphScope version: [0.19.0]
- OS: [ Linux]
Contributor guide
Assessment
This issue has not been assessed yet.