chef / chef/chef-server

estatsd impact the chef server performance and estatsd can not be disabled by estatsd['enable'] = false

Open
#1,504 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Component: opscode-erchef Status: To be prioritized Triage: Try Reproducing Type: Bug Type: Enhancement
Dominant language
Erlang
Stars
303
Forks
211
Avg merge
1d 8h
Merged PRs (30d)
5

Description

I encounter 2 issues, anyone have any idea?

  1. estatsd can not be disabled by estatsd['enable'] = false
  2. estatsd impact the chef server performance badly when setting to emit metric to server other than localhost, got error like this:
    2018-05-22 13:48:37 =CRASH REPORT====
    crasher:
    initial call: stats_hero:init/1
    pid: <0.1748.0>
    registered_name: []
    exception exit: {{timeout,{gen_server,call,[<0.1030.0>,{send,[[[["chefAPI",".application.allRequests"],58,"1",124,"c","\n"],[["chefAPI",".","",".allRequests"],58,"1",124,"c","\n"],[["chefAPI",".application.byRequestType.","clients",".","GET"],58,"1",124,"c","\n"]]]}]}},[{gen_server,init_it,6,[{file,"gen_server.erl"},{line,352}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}
    ancestors: [stats_hero_worker_sup,stats_hero_sup,<0.1020.0>]
    messages: []
    links: [<0.1044.0>]
    dictionary: []
    trap_exit: false
    status: running
    heap_size: 987
    stack_size: 27
    reductions: 2626
    neighbours:
    2018-05-22 13:48:37 =SUPERVISOR REPORT====
    Supervisor: {local,stats_hero_worker_sup}
    Context: child_terminated
    Reason: {timeout,{gen_server,call,[<0.1030.0>,{send,[[[["chefAPI",".application.allRequests"],58,"1",124,"c","\n"],[["chefAPI",".","",".allRequests"],58,"1",124,"c","\n"],[["chefAPI",".application.byRequestType.","clients",".","GET"],58,"1",124,"c","\n"]]]}]}}
    Offender: [{pid,<0.1748.0>},{id,stats_hero},{mfargs,{stats_hero,start_link,undefined}},{restart_type,temporary},{shutdown,brutal_kill},{child_type,worker}]

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the estatsd configuration, especially estatsd['enable'], and the stats_hero worker shown in the crash report. Reproduce the disabled setting and the timeout when emitting metrics to a non-localhost server. Done means estatsd can be disabled and metric delivery failures no longer badly affect Chef Server performance.

Written by the indexing model from the issue text.

Assessment

Tech stack
erlang
Domain
observability-sre
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.