hasura / hasura/graphql-engine
high cpu usage when calling add_inherited_role api
- Dominant language
- TypeScript
- Stars
- 32.1k
- Forks
- 3k
- PR merge metrics
- PR metrics pending
Description
### Is your proposal related to a problem?
when calling [add_inherited_role api](https://hasura.io/docs/latest/api-reference/metadata-api/inherited-roles/#metadata-add-inherited-role), the cpu usage becomes high, especially when the size of role_set becomes very large, like twenty or thirty or larger. More seriously, sometimes it looks like the service would simply restart, making the service unreachable.
cpu usage before add_inherited_role called

cpu usage after add_inherited_role called

service log when calling add_inherited_role
```
2022-10-03T03:01:50.456960675Z {"type":"http-log","timestamp":"2022-10-03T03:01:50.456+0000","level":"info","detail":{"operation":{"query_execution_time":15.057786914,"user_vars":{"x-hasura-role":"admin"},"request_id":"badb8152-17b2-477d-becb-8d9eb2081b50","response_size":21,"query":{"args":{"role_name":"1-57d4c86577927d04196da2e4af48c98e","role_set":["TaskQuery","HotelSystemQuery","TagQuery","TaskDownload","TaskNotify","HotelUserQuery","HotelAgencyQuery","TaskAdd","TaskUpdate","TaskUpdateNeedNum","TaskUpdateDeadline","TaskUpdateAgencyDeadline","TaskUpdateHotelUserState","TaskUpdateLaborerState","TaskCancel","HotelAgencyAdd","HotelAgencyUpdate","ImageQuery","TagAdd","HotelUserAdd","GenImgUrl","HotelUserUpdate","HotelUserTagAdd","HotelUserTagRemove","MaterialQuery","MaterialCategoryQuery","MaterialCategoryUpdate","MaterialCategoryAdd","MaterialCategoryRemove","MaterialAdd","ExcelGetUploadUrl","MaterialImportExcel","UnitOfMeasurementAdd","MaterialUpdate","MaterialRemove","UnitOfMeasurementQuery","UnitOfMeasurementRemove","DishQuery","TypeOfDishQuery","DishAdd","TypeOfDishAdd","DishUpdate","DishRemove","TypeOfDishUpdate","TypeOfDishRemove","MenuQuery","MenuAdd","MenuUpdate","MenuRemove","EoTemplateQuery","EoTemplateAdd","EoTemplateUpdate","EoTemplateRemove","EventTypeQuery","EventTypeAdd","EventTypeUpdate","EventTypeRemove","CustomerQuery","CustomerAdd","CustomerUpdate","CustomerRemove","EoQuery","EoPurchaseAdviseQuery","EoAdd","EoUpdate","EoMenuUpdate","EoMenuAdd","EoCancel","HotelRoleQuery","HotelRoleAdd","HotelRoleUpdate","HotelRoleRemove","DepartmentUpdate","DepartmentAdd","DepartmentRemove","DepartmentQuery","HotelQuery","HotelUserImport","TagUpdate","TagRemove","MiniAppLogin","HotelSupplierQuery","HotelSupplierCategoryManage","HotelSupplierAdd","HotelSupplierUpdate","HotelSupplierRemove","InquiryQuery","InquiryAdd","InquiryCancel","QuotationImport","HotelSupplierCategoryQuery","HotelSupplierCategoryUpdate","PlatformSupplierQuery","MaterialQueryFromExcel","MaterialPriceQuery","FixedPriceAdd","FixedPriceWithoutQuotationImport","QuotationExport","FixedPriceWithQuotationImport","MaterialPriceInsert","CostOverrunQuery","PurchaseRequisitionQuery","PurchaseRequisitionMaterialQuery","PurchaseOrderMaterialQuery","PurchaseOrderQuery","PurchaseOrderViewQuery","WechatQyUserQuery","WechatQyApprovalQuery","EoPresetNotifiedUserAdd","EoPresetNotifiedUserUpdate","EoPresetNotifiedUserQuery","CustomerTagQuery","WarehouseManagerQuery","WarehouseMaterialQuery","InventoryExpiringReportQuery","InventoryLowConsumptionReportQuery","InventoryRealtimeReportQuery","InventoryTransactionReportQuery","BatchQuery","InventoryCheckQuery","InventoryTransactionQuery","InventoryRequisitionQuery","InventoryRequisitionReturnQuery","PurchaseReceiptQuery","InventoryInitQuery"]},"type":"add_inherited_role"},"request_mode":"non-graphql","request_read_time":5.207e-6},"request_id":"badb8152-17b2-477d-becb-8d9eb2081b50","http_info":{"status":200,"http_version":"HTTP/1.1","url":"/v1/metadata","ip":"172.18.0.2","method":"POST","content_encoding":null}}}
2022-10-03T03:01:55.215398266Z {"type":"startup","timestamp":"2022-10-03T03:01:55.213+0000","level":"info","detail":{"kind":"server_configuration","info":{"live_query_options":{"batch_size":100,"refetch_delay":1},"transaction_isolation":"ISOLATION LEVEL READ COMMITTED","enable_maintenance_mode":false,"enabled_log_types":["http-log","websocket-log","startup","webhook-log","query-log"],"server_host":"HostAny","websocket_connection_init_timeout":"WSConnectionInitTimeout {unWSConnectionInitTimeout = Seconds {seconds = 3s}}","enable_allowlist":false,"remote_schema_permissions":false,"log_level":"info","auth_hook_mode":"GET","use_prepared_statements":true,"unauth_role":null,"stringify_numeric_types":false,"v1-boolean-null-collapse":false,"graceful_shutdown_timeout":60,"enabled_apis":["metadata","graphql","config","pgdump"],"enable_telemetry":true,"enable_console":false,"auth_hook":"http://hotelsop-backend:8081/hs/auth","infer_function_permissions":true,"experimental_features":["inherited_roles"],"events_fetch_batch_size":100,"jwt_secret":null,"cors_config":{"allowed_origins":"*","disabled":false,"ws_read_cookie":null},"websocket_compression_options":"NoCompression","console_assets_dir":null,"admin_secret_set":true,"port":8080,"websocket_keep_alive":"KeepAliveDelay {unKeepAliveDelay = Seconds {seconds = 5s}}"}}}
2022-10-03T03:01:56.271049142Z {"type":"startup","timestamp":"2022-10-03T03:01:55.213+0000","level":"info","detail":{"kind":"postgres_connection","info":{"retries":1,"database_url":"postgres://postgres:...@postgres:5432/postgres"}}}
2022-10-03T03:02:14.285422059Z {"type":"startup","timestamp":"2022-10-03T03:02:14.285+0000","level":"info","detail":{"kind":"catalog_migrate","info":"Already at the latest catalog version (47); nothing to do."}}
2022-10-03T03:02:14.285456300Z {"type":"startup","timestamp":"2022-10-03T03:02:14.285+0000","level":"info","detail":{"kind":"schema-sync","info":"Schema sync enabled. Polling at Milliseconds {milliseconds = 1s}"}}
2022-10-03T03:02:15.293127379Z {"type":"unstructured","timestamp":"2022-10-03T03:02:14.285+0000","level":"info","detail":"Thread SchemeUpdate.listener (re)started"}
2022-10-03T03:02:15.293160057Z {"type":"startup","timestamp":"2022-10-03T03:02:14.285+0000","level":"info","detail":{"kind":"schema-sync","info":{"thread_id":"ThreadId 41","instance_id":"6304b7d7-e885-4bea-9003-de43ad3f06fb","message":"listener thread started"}}}
2022-10-03T03:02:15.293165216Z {"type":"startup","timestamp":"2022-10-03T03:02:14.285+0000","level":"info","detail":{"kind":"schema-sync","info":{"thread_id":"ThreadId 54","instance_id":"6304b7d7-e885-4bea-9003-de43ad3f06fb","message":"processor thread started"}}}
2022-10-03T03:02:15.293169008Z {"type":"startup","timestamp":"2022-10-03T03:02:14.285+0000","level":"info","detail":{"kind":"event_triggers","info":"starting workers"}}
2022-10-03T03:02:15.293172674Z {"type":"unstructured","timestamp":"2022-10-03T03:02:14.285+0000","level":"info","detail":"Thread processEventQueue (re)started"}
2022-10-03T03:02:15.293175751Z {"type":"unstructured","timestamp":"2022-10-03T03:02:14.285+0000","level":"info","detail":"Thread asyncActionsProcessor (re)started"}
2022-10-03T03:02:15.293179026Z {"type":"startup","timestamp":"2022-10-03T03:02:14.285+0000","level":"info","detail":{"kind":"scheduled_triggers","info":"preparing data"}}
2022-10-03T03:02:15.293182197Z {"type":"unstructured","timestamp":"2022-10-03T03:02:14.285+0000","level":"info","detail":"Thread runCronEventsGenerator (re)started"}
2022-10-03T03:02:15.293185276Z {"type":"unstructured","timestamp":"2022-10-03T03:02:14.285+0000","level":"info","detail":"Thread processScheduledTriggers (re)started"}
2022-10-03T03:02:15.293188258Z {"type":"unstructured","timestamp":"2022-10-03T03:02:14.285+0000","level":"info","detail":"Thread checkForUpdates (re)started"}
2022-10-03T03:02:15.293191531Z {"type":"startup","timestamp":"2022-10-03T03:02:14.285+0000","level":"info","detail":{"kind":"telemetry","info":"Help us improve Hasura! The graphql-engine server collects anonymized usage stats which allows us to keep improving Hasura at warp speed. To read more or opt-out, visit https://hasura.io/docs/latest/graphql/core/guides/telemetry.html"}}
2022-10-03T03:02:15.293195779Z {"type":"startup","timestamp":"2022-10-03T03:02:14.285+0000","level":"info","detail":{"kind":"server","info":{"time_taken":19.192937574,"message":"starting API server"}}}
2022-10-03T03:02:15.293199025Z {"type":"unstructured","timestamp":"2022-10-03T03:02:14.285+0000","level":"info","detail":"Thread runTelemetry (re)started"}
2022-10-03T03:02:15.293478479Z {"type":"unstructured","timestamp":"2022-10-03T03:02:14.285+0000","level":"info","detail":"Thread ourIdleGC (re)started"}
2022-10-03T03:02:15.293525337Z {"type":"unstructured","timestamp":"2022-10-03T03:02:14.285+0000","level":"info","detail":"Thread SchemeUpdate.processor (re)started"}
2022-10-03T03:02:15.293564655Z {"type":"unstructured","timestamp":"2022-10-03T03:02:14.285+0000","level":"info","detail":"Thread asyncActionSubscriptionsProcessor (re)started"}
2022-10-03T03:02:15.293575853Z {"type":"schema-sync-thread","timestamp":"2022-10-03T03:02:14.285+0000","level":"info","detail":{"thread_type":"processor","info":{"currentVersion":0,"latestResourceVersion":2436}}}
2022-10-03T03:02:30.376100867Z {"type":"schema-sync-thread","timestamp":"2022-10-03T03:02:30.375+0000","level":"info","detail":{"thread_type":"processor","info":{"message":"Schema Version changed with notifications"}}}
```
### Describe the solution you'd like
the service keep reachable when adding inherited role
### Describe alternatives you've considered
no suggestion
### If the feature is approved, would you be willing to submit a PR?
no
Contributor guide
Research direction
Start by reproducing the metadata add_inherited_role API call at /v1/metadata with a large role_set, using the request and service logs in this issue as a baseline. Trace the inherited-roles metadata handling and verify that adding the role no longer causes excessive CPU usage or makes the service unreachable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- graphql, postgres
- Domain
- api, authorization, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100