DIRACGrid / DIRACGrid/DIRAC

[Monitoring] search_phase_execution_exception

Open
#8,286 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
126
Forks
191
Avg merge
1d 13h
Merged PRs (30d)
33

Description

When trying to plot waiting jobs grouped by jobgroup, I get the following.
I suspect there are just too many groups....

```python
2025-08-16T21:17:06,369966Z Monitoring/Monitoring ERROR: Exception while generating plot TransportError(503, 'search_phase_execution_exception')
Traceback (most recent call last):
File "/opt/dirac/versions/v12.0.0a30-1755084796/Linux-x86_64/lib/python3.11/site-packages/DIRAC/MonitoringSystem/Service/MonitoringHandler.py", line 111, in transfer_toClient
result = self._generatePlotFromFileId(fileId)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/dirac/versions/v12.0.0a30-1755084796/Linux-x86_64/lib/python3.11/site-packages/DIRAC/MonitoringSystem/Service/MonitoringHandler.py", line 144, in _generatePlotFromFileId
result = self.export_generatePlot(plotRequest)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/dirac/versions/v12.0.0a30-1755084796/Linux-x86_64/lib/python3.11/site-packages/DIRAC/MonitoringSystem/Service/MonitoringHandler.py", line 221, in export_generatePlot
return reporter.generate(reportRequest)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/dirac/versions/v12.0.0a30-1755084796/Linux-x86_64/lib/python3.11/site-packages/DIRAC/MonitoringSystem/private/MainReporter.py", line 96, in generate
return plotter.generate(reportRequest)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/dirac/versions/v12.0.0a30-1755084796/Linux-x86_64/lib/python3.11/site-packages/DIRAC/MonitoringSystem/private/Plotters/BasePlotter.py", line 116, in generate
retVal = self.__retrieveReportData(reportRequest, reportHash)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/dirac/versions/v12.0.0a30-1755084796/Linux-x86_64/lib/python3.11/site-packages/DIRAC/MonitoringSystem/private/Plotters/BasePlotter.py", line 164, in __retrieveReportData
return gDataCache.getReportData(reportRequest=reportRequest, reportHash=reportHash, dataFunc=funcObj)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/dirac/versions/v12.0.0a30-1755084796/Linux-x86_64/lib/python3.11/site-packages/DIRAC/Core/Utilities/Plotting/DataCache.py", line 44, in getReportData
retVal = dataFunc(reportRequest)
^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/dirac/versions/v12.0.0a30-1755084796/Linux-x86_64/lib/python3.11/site-packages/DIRAC/MonitoringSystem/private/Plotters/WMSHistoryPlotter.py", line 31, in _reportNumberOfJobs
retVal = self._getTimedData(
^^^^^^^^^^^^^^^^^^^
File "/opt/dirac/versions/v12.0.0a30-1755084796/Linux-x86_64/lib/python3.11/site-packages/DIRAC/MonitoringSystem/private/Plotters/BasePlotter.py", line 215, in _getTimedData
retVal = self._retrieveBucketedData(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/dirac/versions/v12.0.0a30-1755084796/Linux-x86_64/lib/python3.11/site-packages/DIRAC/MonitoringSystem/private/DBUtils.py", line 81, in _retrieveBucketedData
return self.__db.retrieveBucketedData(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/dirac/versions/v12.0.0a30-1755084796/Linux-x86_64/lib/python3.11/site-packages/DIRAC/MonitoringSystem/DB/MonitoringDB.py", line 212, in retrieveBucketedData
retVal = s.execute()
^^^^^^^^^^^
File "/opt/dirac/versions/v12.0.0a30-1755084796/Linux-x86_64/lib/python3.11/site-packages/opensearchpy/helpers/search.py", line 760, in execute
opensearch.search(
File "/opt/dirac/versions/v12.0.0a30-1755084796/Linux-x86_64/lib/python3.11/site-packages/opensearchpy/client/utils.py", line 180, in _wrapped
return func(*args, params=params, headers=headers, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/dirac/versions/v12.0.0a30-1755084796/Linux-x86_64/lib/python3.11/site-packages/opensearchpy/client/__init__.py", line 1748, in search
return self.transport.perform_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/dirac/versions/v12.0.0a30-1755084796/Linux-x86_64/lib/python3.11/site-packages/opensearchpy/transport.py", line 445, in perform_request
raise e
File "/opt/dirac/versions/v12.0.0a30-1755084796/Linux-x86_64/lib/python3.11/site-packages/opensearchpy/transport.py", line 408, in perform_request
status, headers_response, data = connection.perform_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/dirac/versions/v12.0.0a30-1755084796/Linux-x86_64/lib/python3.11/site-packages/opensearchpy/connection/http_urllib3.py", line 297, in perform_request
self._raise_error(
File "/opt/dirac/versions/v12.0.0a30-1755084796/Linux-x86_64/lib/python3.11/site-packages/opensearchpy/connection/base.py", line 315, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(
opensearchpy.exceptions.TransportError: TransportError(503, 'search_phase_execution_exception')
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.