lablup / lablup/backend.ai

ERROR ai.backend.manager.api.admin ADMIN.GQL Exception

Open
#1,816 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

### What Operating System(s) are you seeing this problem on?

Linux (x86-64) Ubuntu-22.04

### Backend.AI version

23.9.6

### Describe the bug

When the manager service start, it produce the bug log. It seems that the GQL query param `compute_session_list ` is None.

```sh
2024-01-02 14:23:55.583 INFO aiohttp.access [18159] 10.10.60.2 [02/Jan/2024:06:23:55 +0000] "POST /admin/graphql HTTP/1.1" 200 593 "-" "Backend.AI Web Server 23.09.6"
2024-01-02 14:23:55.597 DEBUG ai.backend.manager.api.utils [18159] stripped raw params: {'query': 'query($limit:Int!, $offset:Int!, $ak:String, $group_id:String, $status:String) {\n compute_session_list(limit:$limit, offset:$offset, access_key:$ak, group_id:$group_id, status:$status) {\n total_count\n }\n }', 'variables': {'limit': 1, 'offset': 0, 'status': 'RUNNING'}}
2024-01-02 14:23:55.617 ERROR ai.backend.manager.api.admin [18159] ADMIN.GQL Exception: {'message': "'NoneType' object has no attribute '_apply_params_to_element'", 'locations': [{'line': 2, 'column': 7}], 'path': ['compute_session_list']}
2024-01-02 14:23:55.627 DEBUG ai.backend.manager.plugin.error_monitor [18159] collected an error log [ERROR] "ai.backend.manager.api.exceptions.GraphQLError: GraphQL-generated error.
-> extra_data: [{'message': "'NoneType' object has no attribute '_apply_params_to_element'", 'locations': [{'line': 2, 'column': 7}], 'path': ['compute_session_list']}]" from manager
```

the traceback info :

```sh
2024-01-05 10:39:23.244 ERROR graphql.execution.utils [15066] Traceback (most recent call last):
File "/home/bai/manager/.venv/lib/python3.11/site-packages/promise/promise.py", line 844, in handle_future_result
resolve(future.result())
^^^^^^^^^^^^^^^
File "/home/bai/manager/.venv/lib/python3.11/site-packages/aiomonitor/monitor.py", line 487, in _coro_wrapper
return await coro
^^^^^^^^^^
File "/home/bai/manager/.venv/lib/python3.11/site-packages/ai/backend/manager/models/base.py", line 844, in wrapped
return await resolve_func(executor, info, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/bai/manager/.venv/lib/python3.11/site-packages/ai/backend/manager/models/gql.py", line 1647, in resolve_compute_session_list
items = await ComputeSession.load_slice(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/bai/manager/.venv/lib/python3.11/site-packages/ai/backend/manager/models/session.py", line 1519, in load_slice
return [cls.from_row(ctx, r) async for r in (await db_sess.stream(query))]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/bai/manager/.venv/lib/python3.11/site-packages/ai/backend/manager/models/session.py", line 1519, in
return [cls.from_row(ctx, r) async for r in (await db_sess.stream(query))]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/bai/manager/.venv/lib/python3.11/site-packages/sqlalchemy/ext/asyncio/result.py", line 174, in __anext__
row = await greenlet_spawn(self._onerow_getter, self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/bai/manager/.venv/lib/python3.11/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 128, in greenlet_spawn
result = context.switch(value)
^^^^^^^^^^^^^^^^^^^^^
File "/home/bai/manager/.venv/lib/python3.11/site-packages/sqlalchemy/engine/result.py", line 457, in onerow
row = self._fetchone_impl()
^^^^^^^^^^^^^^^^^^^^^
File "/home/bai/manager/.venv/lib/python3.11/site-packages/sqlalchemy/engine/result.py", line 1340, in _fetchone_impl
return self._real_result._fetchone_impl(hard_close=hard_close)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/bai/manager/.venv/lib/python3.11/site-packages/sqlalchemy/engine/result.py", line 1743, in _fetchone_impl
row = next(self.iterator, _NO_ROW)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/bai/manager/.venv/lib/python3.11/site-packages/sqlalchemy/orm/loading.py", line 158, in chunks
post_load.invoke(context, path)
File "/home/bai/manager/.venv/lib/python3.11/site-packages/sqlalchemy/orm/loading.py", line 1335, in invoke
loader(context, path, states, self.load_keys, *arg, **kw)
File "/home/bai/manager/.venv/lib/python3.11/site-packages/sqlalchemy/orm/strategies.py", line 2985, in _load_for_path
loadopt._generate_extra_criteria(context),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/bai/manager/.venv/lib/python3.11/site-packages/sqlalchemy/orm/strategy_options.py", line 154, in _generate_extra_criteria
return k2._apply_params_to_element(k1, and_(*self._extra_criteria))
^^^^^^^^^^^^^^^^^^^^^^^^^^^
graphql.error.located_error.GraphQLLocatedError: 'NoneType' object has no attribute '_apply_params_to_element'

2024-01-05 10:39:23.245 ERROR ai.backend.manager.api.admin [15066] ADMIN.GQL Exception: {'message': "'NoneType' object has no attribute '_apply_params_to_element'", 'locations': [{'line': 2, 'column': 7}], 'path': ['compute_session_list']}

```

### To Reproduce

sudo systemctl restart backend.ai-manager.service

### Expected Behavior

### Anything else?

_No response_

JIRA Issue: BA-163

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.