action_service.list_values no limit or offset support?

SUMMARY

action_service.list_values not working properly?

STACKSTORM VERSION

3.7.0

OS, environment, install method

OS install on RHEL

Steps to reproduce the problem

As per the contrib/runners/python_runner/python_runner/python_action_wrapper.py the action service allows for listing of datastore values. The relevant method I think is this:

def list_values(self, local=True, prefix=None):
    return self.datastore_service.list_values(local=local, prefix=prefix)

However, the above is calling below method without the "limit" parameter:
st2common/st2common/services/datastore.py line 71

def list_values(self, local=True, prefix=None, limit=None, offset=0):
    """
    Retrieve all the datastores items.

    :param local: List values from a namespace local to this pack/class. Defaults to True.
    :type: local: ``bool``

    :param prefix: Optional key name prefix / startswith filter.
    :type prefix: ``str``

    :param limit: Number of keys to get. Defaults to the configuration set at 'api.max_page_size'.
    :type limit: ``integer``

    :param offset: Number of keys to offset. Defaults to 0.
    :type offset: ``integer``

    :rtype: ``list`` of :class:`KeyValuePair`
    """
    client = self.get_api_client()
    self._logger.debug("Retrieving all the values from the datastore")

    limit = limit or cfg.CONF.api.max_page_size
    key_prefix = self._get_full_key_prefix(local=local, prefix=prefix)
    kvps = client.keys.get_all(prefix=key_prefix, limit=limit, offset=offset)
    return kvps

Using action_service.list_values I'm not able to provide the limit parameter and thus the action will fail with:
oslo_config.cfg.NoSuchOptError: no such option in group api: max_page_size

even though I did put max_page_size = 100 into st2.conf and restarted st2.

Is there a reason why I can't provide the limit to the action_service.list_values method?

(another missing param is "offset", which would also be useful in certain situations)

Expected Results

a list of keyvalue pairs

Actual Results

st2.actions.python.SetAccountsObject: DEBUG Creating new Client object.
st2.actions.python.SetAccountsObject: DEBUG Retrieving all the values from the datastore
Traceback (most recent call last):
File "/opt/stackstorm/st2/lib/python3.8/site-packages/oslo_config/cfg.py", line 2262, in _get
return self.__cache[key]
KeyError: ('api', 'max_page_size')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/stackstorm/st2/lib/python3.8/site-packages/python_runner/python_action_wrapper.py", line 395, in
obj.run()
File "/opt/stackstorm/st2/lib/python3.8/site-packages/python_runner/python_action_wrapper.py", line 214, in run
output = action.run(**self._parameters)
File "/opt/stackstorm/internal_packs/preprocessing_shared_scripts/actions/set_accounts_obj.py", line 8, in run
accts_objects = self.get_overrides(capability_name)
File "/opt/stackstorm/internal_packs/preprocessing_shared_scripts/actions/set_accounts_obj.py", line 12, in get_overrides
return self.action_service.list_values(local=False, prefix=f'{capability}accounts')
File "/opt/stackstorm/st2/lib/python3.8/site-packages/python_runner/python_action_wrapper.py", line 136, in list_values
return self.datastore_service.list_values(local=local, prefix=prefix)
File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2common/services/datastore.py", line 92, in list_values
limit = limit or cfg.CONF.api.max_page_size
File "/opt/stackstorm/st2/lib/python3.8/site-packages/oslo_config/cfg.py", line 2547, in getattr
return self._conf._get(name, self._group)
File "/opt/stackstorm/st2/lib/python3.8/site-packages/oslo_config/cfg.py", line 2264, in _get
value = self._do_get(name, group, namespace)
File "/opt/stackstorm/st2/lib/python3.8/site-packages/oslo_config/cfg.py", line 2282, in _do_get
info = self._get_opt_info(name, group)
File "/opt/stackstorm/st2/lib/python3.8/site-packages/oslo_config/cfg.py", line 2415, in _get_opt_info
raise NoSuchOptError(opt_name, group)
oslo_config.cfg.NoSuchOptError: no such option in group api: max_page_size

Making sure to follow these steps will guarantee the quickest resolution possible.

Thanks!

Start with list_values in contrib/runners/python_runner/python_runner/python_action_wrapper.py and st2common/st2common/services/datastore.py. Trace how the action service delegates to the datastore service, then expose limit and offset while preserving the existing defaults; done means actions can request bounded, offset datastore results without the shown configuration error.