Keypair API response inconsistency admin vs. non-admin when calling system scoped keys

SUMMARY

Keypair API returns inconsistent results when called by admin vs. non-admin user when applying name / prefix filters on system scoped keys the user has been granted access to via RBAC.

This leads to the st2client throwing exceptions when using get_by_name method, because for some reason get_by_name client method calls /keys/?name=<key_name>&scope=system instead of /keys/<key_name>?scope=system, as instances > 1 and leads to the client returning ALL keys when doing get_all(prefix=prefix), because the API response is always ALL the keys the user has access to.

This is an issue on API level and stems from this piece of code that does the proper action when called by an admin, but applies no filter when called by a non-admin:

https://github.com/StackStorm/st2/blob/master/st2api/st2api/controllers/v1/keyvalue.py#L232

        if scope in [ALL_SCOPE, SYSTEM_SCOPE, FULL_SYSTEM_SCOPE]:
            decrypted_keys = []
            # If user has system role, then retrieve all system scoped items
            if has_system_role:
                raw_filters["scope"] = FULL_SYSTEM_SCOPE
                raw_filters["prefix"] = prefix

                items = self._get_all(
                    from_model_kwargs=from_model_kwargs,
                    sort=sort,
                    offset=offset,
                    limit=limit,
                    raw_filters=raw_filters,
                    requester_user=requester_user,
                )

                kvp_apis_system.extend(items.json or [])
                if decrypt and items.json:
                    decrypted_keys.extend(
                        kv_api["name"] for kv_api in items.json if kv_api["secret"]
                    )
            else:
                # Otherwise if user is not an admin, then get the list of
                # system scoped items that user is granted permission to.
                for key in get_all_system_kvp_names_for_user(current_user):
                    try:
                        item = self._get_one_by_scope_and_name(
                            from_model_kwargs=from_model_kwargs,
                            scope=FULL_SYSTEM_SCOPE,
                            name=key,
                        )

                        kvp_apis_system.append(item)
                    except Exception as e:
                        LOG.error("Unable to get key %s: %s", key, str(e))
                        continue
                    if decrypt and item.secret:
                        decrypted_keys.append(key)

STACKSTORM VERSION

Paste the output of st2 --version:
st2 3.8.1, on Python 3.8.10

OS, environment, install method

K8S HA deployment in AWS

Steps to reproduce the problem

call /api/v1/keys?name=<key_name>&scope=system

Expected Results

Expected is an array of 1 keys with a specific name or a set of ? keys matching a prefix.

Actual Results

Output is always ALL the keys that the user has been granted access to via RBAC, regardless of name or prefix.

Making sure to follow these steps will guarantee the quickest resolution possible.

Thanks!

Start in st2api/st2api/controllers/v1/keyvalue.py around line 232 and reproduce /api/v1/keys?name=<key_name>&scope=system for both admin and non-admin users with RBAC access. Compare name and prefix filtering, and consider the related st2client get_by_name and get_all behavior; done means non-admin responses contain only matching system-scoped keys.