ERROR ai.backend.manager.api.admin ADMIN.GQL Exception
- Dominant language
- Python
- Stars
- 670
- Forks
- 183
- Avg merge
- 17h 7m
- Merged PRs (30d)
- 358
Description
### What Operating System(s) are you seeing this problem on?
Linux (x86-64) Ubuntu-22.04
### Backend.AI version
23.9.6
### Describe the bug
When the manager service start, it produce the bug log. It seems that the GQL query param `compute_session_list ` is None.
```sh
2024-01-02 14:23:55.583 INFO aiohttp.access [18159] 10.10.60.2 [02/Jan/2024:06:23:55 +0000] "POST /admin/graphql HTTP/1.1" 200 593 "-" "Backend.AI Web Server 23.09.6"
2024-01-02 14:23:55.597 DEBUG ai.backend.manager.api.utils [18159] stripped raw params: {'query': 'query($limit:Int!, $offset:Int!, $ak:String, $group_id:String, $status:String) {\n compute_session_list(limit:$limit, offset:$offset, access_key:$ak, group_id:$group_id, status:$status) {\n total_count\n }\n }', 'variables': {'limit': 1, 'offset': 0, 'status': 'RUNNING'}}
2024-01-02 14:23:55.617 ERROR ai.backend.manager.api.admin [18159] ADMIN.GQL Exception: {'message': "'NoneType' object has no attribute '_apply_params_to_element'", 'locations': [{'line': 2, 'column': 7}], 'path': ['compute_session_list']}
2024-01-02 14:23:55.627 DEBUG ai.backend.manager.plugin.error_monitor [18159] collected an error log [ERROR] "ai.backend.manager.api.exceptions.GraphQLError: GraphQL-generated error.
-> extra_data: [{'message': "'NoneType' object has no attribute '_apply_params_to_element'", 'locations': [{'line': 2, 'column': 7}], 'path': ['compute_session_list']}]" from manager
```
the traceback info :
```sh
2024-01-05 10:39:23.244 ERROR graphql.execution.utils [15066] Traceback (most recent call last):
File "/home/bai/manager/.venv/lib/python3.11/site-packages/promise/promise.py", line 844, in handle_future_result
resolve(future.result())
^^^^^^^^^^^^^^^
File "/home/bai/manager/.venv/lib/python3.11/site-packages/aiomonitor/monitor.py", line 487, in _coro_wrapper
return await coro
^^^^^^^^^^
File "/home/bai/manager/.venv/lib/python3.11/site-packages/ai/backend/manager/models/base.py", line 844, in wrapped
return await resolve_func(executor, info, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/bai/manager/.venv/lib/python3.11/site-packages/ai/backend/manager/models/gql.py", line 1647, in resolve_compute_session_list
items = await ComputeSession.load_slice(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/bai/manager/.venv/lib/python3.11/site-packages/ai/backend/manager/models/session.py", line 1519, in load_slice
return [cls.from_row(ctx, r) async for r in (await db_sess.stream(query))]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/bai/manager/.venv/lib/python3.11/site-packages/ai/backend/manager/models/session.py", line 1519, in
return [cls.from_row(ctx, r) async for r in (await db_sess.stream(query))]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/bai/manager/.venv/lib/python3.11/site-packages/sqlalchemy/ext/asyncio/result.py", line 174, in __anext__
row = await greenlet_spawn(self._onerow_getter, self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/bai/manager/.venv/lib/python3.11/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 128, in greenlet_spawn
result = context.switch(value)
^^^^^^^^^^^^^^^^^^^^^
File "/home/bai/manager/.venv/lib/python3.11/site-packages/sqlalchemy/engine/result.py", line 457, in onerow
row = self._fetchone_impl()
^^^^^^^^^^^^^^^^^^^^^
File "/home/bai/manager/.venv/lib/python3.11/site-packages/sqlalchemy/engine/result.py", line 1340, in _fetchone_impl
return self._real_result._fetchone_impl(hard_close=hard_close)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/bai/manager/.venv/lib/python3.11/site-packages/sqlalchemy/engine/result.py", line 1743, in _fetchone_impl
row = next(self.iterator, _NO_ROW)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/bai/manager/.venv/lib/python3.11/site-packages/sqlalchemy/orm/loading.py", line 158, in chunks
post_load.invoke(context, path)
File "/home/bai/manager/.venv/lib/python3.11/site-packages/sqlalchemy/orm/loading.py", line 1335, in invoke
loader(context, path, states, self.load_keys, *arg, **kw)
File "/home/bai/manager/.venv/lib/python3.11/site-packages/sqlalchemy/orm/strategies.py", line 2985, in _load_for_path
loadopt._generate_extra_criteria(context),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/bai/manager/.venv/lib/python3.11/site-packages/sqlalchemy/orm/strategy_options.py", line 154, in _generate_extra_criteria
return k2._apply_params_to_element(k1, and_(*self._extra_criteria))
^^^^^^^^^^^^^^^^^^^^^^^^^^^
graphql.error.located_error.GraphQLLocatedError: 'NoneType' object has no attribute '_apply_params_to_element'
2024-01-05 10:39:23.245 ERROR ai.backend.manager.api.admin [15066] ADMIN.GQL Exception: {'message': "'NoneType' object has no attribute '_apply_params_to_element'", 'locations': [{'line': 2, 'column': 7}], 'path': ['compute_session_list']}
```
### To Reproduce
sudo systemctl restart backend.ai-manager.service
### Expected Behavior
### Anything else?
_No response_
JIRA Issue: BA-163
Contributor guide
Assessment
This issue has not been assessed yet.