canonical / canonical/microceph

prometheus mds metrics no longer available

Open
#485 4 comments 0 reactions 0 assignees View on GitHub
bug workaround-available
Dominant language
Go
Stars
396
Forks
74
Avg merge
2d 20h
Merged PRs (30d)
7

Description

# Issue report

Somewhere between revision 1100 and 1139, some prometheus metrics went missing. The one I've noticed so far is the MDS metrics. I found this issue when one of my clusters stopped reporting those metrics. Seeing it was on a newer version, I upgraded the working clusters to the latest revision and they are now all exhibiting the same issue.

## What version of MicroCeph are you using ?

microceph (reef/stable) 18.2.4+snapc9f2b08f92

## What are the steps to reproduce this issue ?

1. Install latest reef/stable release.
2. Enable prometheus mgr module: `ceph mgr module enable prometheus`
3. Curl endpoint to get metrics and check for mds metrics: `curl localhost:9283/metrics| grep mds`

## What happens (observed behaviour) ?

Only the metadata of the MDS daemon is there:
```
# HELP ceph_mds_metadata MDS Metadata
# TYPE ceph_mds_metadata untyped
ceph_mds_metadata{ceph_daemon="mds.scooby",fs_id="1",hostname="scooby",public_addr="10.0.0.7:6809/1914232058",rank="0",ceph_version="ceph version 18.2.4 (e7ad5345525c7aa95470c26863873b581076945d) reef (stable)"} 1.0
```

## What were you expecting to happen ?

Before 1100 -> 1139 upgrade, all MDS metrics were available:
```
# HELP ceph_mds_metadata MDS Metadata
# TYPE ceph_mds_metadata untyped
ceph_mds_metadata{ceph_daemon="mds.scooby",fs_id="1",hostname="scooby",public_addr="10.0.0.7:6809/972755473",rank="0",ceph_version="ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)"} 1.0
# HELP ceph_mds_caps Capabilities
# TYPE ceph_mds_caps gauge
ceph_mds_caps{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_ceph_cap_op_flush_ack caps truncate notify
# TYPE ceph_mds_ceph_cap_op_flush_ack counter
ceph_mds_ceph_cap_op_flush_ack{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_ceph_cap_op_flushsnap_ack caps truncate notify
# TYPE ceph_mds_ceph_cap_op_flushsnap_ack counter
ceph_mds_ceph_cap_op_flushsnap_ack{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_ceph_cap_op_grant Grant caps
# TYPE ceph_mds_ceph_cap_op_grant counter
ceph_mds_ceph_cap_op_grant{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_ceph_cap_op_revoke Revoke caps
# TYPE ceph_mds_ceph_cap_op_revoke counter
ceph_mds_ceph_cap_op_revoke{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_ceph_cap_op_trunc caps truncate notify
# TYPE ceph_mds_ceph_cap_op_trunc counter
ceph_mds_ceph_cap_op_trunc{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_dir_commit Directory commit
# TYPE ceph_mds_dir_commit counter
ceph_mds_dir_commit{ceph_daemon="mds.scooby"} 3.0
# HELP ceph_mds_dir_fetch_complete Fetch complete dirfrag
# TYPE ceph_mds_dir_fetch_complete counter
ceph_mds_dir_fetch_complete{ceph_daemon="mds.scooby"} 43.0
# HELP ceph_mds_dir_fetch_keys Fetch keys from dirfrag
# TYPE ceph_mds_dir_fetch_keys counter
ceph_mds_dir_fetch_keys{ceph_daemon="mds.scooby"} 6.0
# HELP ceph_mds_dir_merge Directory merge
# TYPE ceph_mds_dir_merge counter
ceph_mds_dir_merge{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_dir_split Directory split
# TYPE ceph_mds_dir_split counter
ceph_mds_dir_split{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_exported_inodes Exported inodes
# TYPE ceph_mds_exported_inodes counter
ceph_mds_exported_inodes{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_forward Forwarding request
# TYPE ceph_mds_forward counter
ceph_mds_forward{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_handle_client_cap_release Client cap release msg
# TYPE ceph_mds_handle_client_cap_release counter
ceph_mds_handle_client_cap_release{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_handle_client_caps Client caps msg
# TYPE ceph_mds_handle_client_caps counter
ceph_mds_handle_client_caps{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_handle_client_caps_dirty Client dirty caps msg
# TYPE ceph_mds_handle_client_caps_dirty counter
ceph_mds_handle_client_caps_dirty{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_handle_inode_file_caps Inter mds caps msg
# TYPE ceph_mds_handle_inode_file_caps counter
ceph_mds_handle_inode_file_caps{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_imported_inodes Imported inodes
# TYPE ceph_mds_imported_inodes counter
ceph_mds_imported_inodes{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_inodes Inodes
# TYPE ceph_mds_inodes gauge
ceph_mds_inodes{ceph_daemon="mds.scooby"} 140992.0
# HELP ceph_mds_inodes_expired Inodes expired
# TYPE ceph_mds_inodes_expired gauge
ceph_mds_inodes_expired{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_inodes_pinned Inodes pinned
# TYPE ceph_mds_inodes_pinned gauge
ceph_mds_inodes_pinned{ceph_daemon="mds.scooby"} 105904.0
# HELP ceph_mds_inodes_with_caps Inodes with capabilities
# TYPE ceph_mds_inodes_with_caps gauge
ceph_mds_inodes_with_caps{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_load_cent Load per cent
# TYPE ceph_mds_load_cent gauge
ceph_mds_load_cent{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_openino_dir_fetch OpenIno incomplete directory fetchings
# TYPE ceph_mds_openino_dir_fetch counter
ceph_mds_openino_dir_fetch{ceph_daemon="mds.scooby"} 33.0
# HELP ceph_mds_process_request_cap_release Process request cap release
# TYPE ceph_mds_process_request_cap_release counter
ceph_mds_process_request_cap_release{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_reply_latency_sum Reply latency Total
# TYPE ceph_mds_reply_latency_sum counter
ceph_mds_reply_latency_sum{ceph_daemon="mds.scooby"} 0.70521454
# HELP ceph_mds_reply_latency_count Reply latency Count
# TYPE ceph_mds_reply_latency_count counter
ceph_mds_reply_latency_count{ceph_daemon="mds.scooby"} 73.0
# HELP ceph_mds_request Requests
# TYPE ceph_mds_request counter
ceph_mds_request{ceph_daemon="mds.scooby"} 75.0
# HELP ceph_mds_root_rbytes root inode rbytes
# TYPE ceph_mds_root_rbytes gauge
ceph_mds_root_rbytes{ceph_daemon="mds.scooby"} 597387365897.0
# HELP ceph_mds_root_rfiles root inode rfiles
# TYPE ceph_mds_root_rfiles gauge
ceph_mds_root_rfiles{ceph_daemon="mds.scooby"} 234472.0
# HELP ceph_mds_root_rsnaps root inode rsnaps
# TYPE ceph_mds_root_rsnaps gauge
ceph_mds_root_rsnaps{ceph_daemon="mds.scooby"} 55.0
# HELP ceph_mds_slow_reply Slow replies
# TYPE ceph_mds_slow_reply counter
ceph_mds_slow_reply{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_subtrees Subtrees
# TYPE ceph_mds_subtrees gauge
ceph_mds_subtrees{ceph_daemon="mds.scooby"} 3.0
# HELP ceph_mds_cache_ireq_enqueue_scrub Internal Request type enqueue scrub
# TYPE ceph_mds_cache_ireq_enqueue_scrub counter
ceph_mds_cache_ireq_enqueue_scrub{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_cache_ireq_exportdir Internal Request type export dir
# TYPE ceph_mds_cache_ireq_exportdir counter
ceph_mds_cache_ireq_exportdir{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_cache_ireq_flush Internal Request type flush
# TYPE ceph_mds_cache_ireq_flush counter
ceph_mds_cache_ireq_flush{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_cache_ireq_fragmentdir Internal Request type fragmentdir
# TYPE ceph_mds_cache_ireq_fragmentdir counter
ceph_mds_cache_ireq_fragmentdir{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_cache_ireq_fragstats Internal Request type frag stats
# TYPE ceph_mds_cache_ireq_fragstats counter
ceph_mds_cache_ireq_fragstats{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_cache_ireq_inodestats Internal Request type inode stats
# TYPE ceph_mds_cache_ireq_inodestats counter
ceph_mds_cache_ireq_inodestats{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_cache_num_recovering_enqueued Files waiting for recovery
# TYPE ceph_mds_cache_num_recovering_enqueued gauge
ceph_mds_cache_num_recovering_enqueued{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_cache_num_recovering_prioritized Files waiting for recovery with elevated priority
# TYPE ceph_mds_cache_num_recovering_prioritized gauge
ceph_mds_cache_num_recovering_prioritized{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_cache_num_recovering_processing Files currently being recovered
# TYPE ceph_mds_cache_num_recovering_processing gauge
ceph_mds_cache_num_recovering_processing{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_cache_num_strays Stray dentries
# TYPE ceph_mds_cache_num_strays gauge
ceph_mds_cache_num_strays{ceph_daemon="mds.scooby"} 6119.0
# HELP ceph_mds_cache_num_strays_delayed Stray dentries delayed
# TYPE ceph_mds_cache_num_strays_delayed gauge
ceph_mds_cache_num_strays_delayed{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_cache_num_strays_enqueuing Stray dentries enqueuing for purge
# TYPE ceph_mds_cache_num_strays_enqueuing gauge
ceph_mds_cache_num_strays_enqueuing{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_cache_recovery_completed File recoveries completed
# TYPE ceph_mds_cache_recovery_completed counter
ceph_mds_cache_recovery_completed{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_cache_recovery_started File recoveries started
# TYPE ceph_mds_cache_recovery_started counter
ceph_mds_cache_recovery_started{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_cache_strays_created Stray dentries created
# TYPE ceph_mds_cache_strays_created counter
ceph_mds_cache_strays_created{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_cache_strays_enqueued Stray dentries enqueued for purge
# TYPE ceph_mds_cache_strays_enqueued counter
ceph_mds_cache_strays_enqueued{ceph_daemon="mds.scooby"} 243.0
# HELP ceph_mds_cache_strays_migrated Stray dentries migrated
# TYPE ceph_mds_cache_strays_migrated counter
ceph_mds_cache_strays_migrated{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_cache_strays_reintegrated Stray dentries reintegrated
# TYPE ceph_mds_cache_strays_reintegrated counter
ceph_mds_cache_strays_reintegrated{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_log_ev Events
# TYPE ceph_mds_log_ev gauge
ceph_mds_log_ev{ceph_daemon="mds.scooby"} 116900.0
# HELP ceph_mds_log_evadd Events submitted
# TYPE ceph_mds_log_evadd counter
ceph_mds_log_evadd{ceph_daemon="mds.scooby"} 325.0
# HELP ceph_mds_log_evex Total expired events
# TYPE ceph_mds_log_evex counter
ceph_mds_log_evex{ceph_daemon="mds.scooby"} 953.0
# HELP ceph_mds_log_evexd Current expired events
# TYPE ceph_mds_log_evexd gauge
ceph_mds_log_evexd{ceph_daemon="mds.scooby"} 953.0
# HELP ceph_mds_log_evexg Expiring events
# TYPE ceph_mds_log_evexg gauge
ceph_mds_log_evexg{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_log_evtrm Trimmed events
# TYPE ceph_mds_log_evtrm counter
ceph_mds_log_evtrm{ceph_daemon="mds.scooby"} 953.0
# HELP ceph_mds_log_jlat_sum Journaler flush latency Total
# TYPE ceph_mds_log_jlat_sum counter
ceph_mds_log_jlat_sum{ceph_daemon="mds.scooby"} 58.250196456
# HELP ceph_mds_log_jlat_count Journaler flush latency Count
# TYPE ceph_mds_log_jlat_count counter
ceph_mds_log_jlat_count{ceph_daemon="mds.scooby"} 80.0
# HELP ceph_mds_log_replayed Events replayed
# TYPE ceph_mds_log_replayed counter
ceph_mds_log_replayed{ceph_daemon="mds.scooby"} 117528.0
# HELP ceph_mds_log_seg Segments
# TYPE ceph_mds_log_seg gauge
ceph_mds_log_seg{ceph_daemon="mds.scooby"} 129.0
# HELP ceph_mds_log_segadd Segments added
# TYPE ceph_mds_log_segadd counter
ceph_mds_log_segadd{ceph_daemon="mds.scooby"} 1.0
# HELP ceph_mds_log_segex Total expired segments
# TYPE ceph_mds_log_segex counter
ceph_mds_log_segex{ceph_daemon="mds.scooby"} 1.0
# HELP ceph_mds_log_segexd Current expired segments
# TYPE ceph_mds_log_segexd gauge
ceph_mds_log_segexd{ceph_daemon="mds.scooby"} 1.0
# HELP ceph_mds_log_segexg Expiring segments
# TYPE ceph_mds_log_segexg gauge
ceph_mds_log_segexg{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_log_segtrm Trimmed segments
# TYPE ceph_mds_log_segtrm counter
ceph_mds_log_segtrm{ceph_daemon="mds.scooby"} 1.0
# HELP ceph_mds_mem_cap Capabilities
# TYPE ceph_mds_mem_cap gauge
ceph_mds_mem_cap{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_mem_cap_plus Capabilities added
# TYPE ceph_mds_mem_cap_plus counter
ceph_mds_mem_cap_plus{ceph_daemon="mds.scooby"} 6.0
# HELP ceph_mds_mem_cap_minus Capabilities removed
# TYPE ceph_mds_mem_cap_minus counter
ceph_mds_mem_cap_minus{ceph_daemon="mds.scooby"} 6.0
# HELP ceph_mds_mem_dir Directories
# TYPE ceph_mds_mem_dir gauge
ceph_mds_mem_dir{ceph_daemon="mds.scooby"} 105299.0
# HELP ceph_mds_mem_dir_plus Directories opened
# TYPE ceph_mds_mem_dir_plus counter
ceph_mds_mem_dir_plus{ceph_daemon="mds.scooby"} 105299.0
# HELP ceph_mds_mem_dir_minus Directories closed
# TYPE ceph_mds_mem_dir_minus counter
ceph_mds_mem_dir_minus{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_mem_dn Dentries
# TYPE ceph_mds_mem_dn gauge
ceph_mds_mem_dn{ceph_daemon="mds.scooby"} 140992.0
# HELP ceph_mds_mem_dn_plus Dentries opened
# TYPE ceph_mds_mem_dn_plus counter
ceph_mds_mem_dn_plus{ceph_daemon="mds.scooby"} 140992.0
# HELP ceph_mds_mem_dn_minus Dentries closed
# TYPE ceph_mds_mem_dn_minus counter
ceph_mds_mem_dn_minus{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_mem_heap Heap size
# TYPE ceph_mds_mem_heap gauge
ceph_mds_mem_heap{ceph_daemon="mds.scooby"} 223424.0
# HELP ceph_mds_mem_ino Inodes
# TYPE ceph_mds_mem_ino gauge
ceph_mds_mem_ino{ceph_daemon="mds.scooby"} 140425.0
# HELP ceph_mds_mem_ino_plus Inodes opened
# TYPE ceph_mds_mem_ino_plus counter
ceph_mds_mem_ino_plus{ceph_daemon="mds.scooby"} 140750.0
# HELP ceph_mds_mem_ino_minus Inodes closed
# TYPE ceph_mds_mem_ino_minus counter
ceph_mds_mem_ino_minus{ceph_daemon="mds.scooby"} 325.0
# HELP ceph_mds_server_cap_acquisition_throttle Cap acquisition throttle counter
# TYPE ceph_mds_server_cap_acquisition_throttle counter
ceph_mds_server_cap_acquisition_throttle{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_cap_revoke_eviction Cap Revoke Client Eviction
# TYPE ceph_mds_server_cap_revoke_eviction counter
ceph_mds_server_cap_revoke_eviction{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_handle_client_request Client requests
# TYPE ceph_mds_server_handle_client_request counter
ceph_mds_server_handle_client_request{ceph_daemon="mds.scooby"} 75.0
# HELP ceph_mds_server_handle_client_session Client session messages
# TYPE ceph_mds_server_handle_client_session counter
ceph_mds_server_handle_client_session{ceph_daemon="mds.scooby"} 21.0
# HELP ceph_mds_server_handle_peer_request Peer requests
# TYPE ceph_mds_server_handle_peer_request counter
ceph_mds_server_handle_peer_request{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_create_latency_sum Request type create latency Total
# TYPE ceph_mds_server_req_create_latency_sum counter
ceph_mds_server_req_create_latency_sum{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_create_latency_count Request type create latency Count
# TYPE ceph_mds_server_req_create_latency_count counter
ceph_mds_server_req_create_latency_count{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_getattr_latency_sum Request type get attribute latency Total
# TYPE ceph_mds_server_req_getattr_latency_sum counter
ceph_mds_server_req_getattr_latency_sum{ceph_daemon="mds.scooby"} 0.069502791
# HELP ceph_mds_server_req_getattr_latency_count Request type get attribute latency Count
# TYPE ceph_mds_server_req_getattr_latency_count counter
ceph_mds_server_req_getattr_latency_count{ceph_daemon="mds.scooby"} 66.0
# HELP ceph_mds_server_req_getfilelock_latency_sum Request type get file lock latency Total
# TYPE ceph_mds_server_req_getfilelock_latency_sum counter
ceph_mds_server_req_getfilelock_latency_sum{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_getfilelock_latency_count Request type get file lock latency Count
# TYPE ceph_mds_server_req_getfilelock_latency_count counter
ceph_mds_server_req_getfilelock_latency_count{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_getvxattr_latency_sum Request type get virtual extended attribute latency Total
# TYPE ceph_mds_server_req_getvxattr_latency_sum counter
ceph_mds_server_req_getvxattr_latency_sum{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_getvxattr_latency_count Request type get virtual extended attribute latency Count
# TYPE ceph_mds_server_req_getvxattr_latency_count counter
ceph_mds_server_req_getvxattr_latency_count{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_link_latency_sum Request type link latency Total
# TYPE ceph_mds_server_req_link_latency_sum counter
ceph_mds_server_req_link_latency_sum{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_link_latency_count Request type link latency Count
# TYPE ceph_mds_server_req_link_latency_count counter
ceph_mds_server_req_link_latency_count{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_lookup_latency_sum Request type lookup latency Total
# TYPE ceph_mds_server_req_lookup_latency_sum counter
ceph_mds_server_req_lookup_latency_sum{ceph_daemon="mds.scooby"} 0.113742706
# HELP ceph_mds_server_req_lookup_latency_count Request type lookup latency Count
# TYPE ceph_mds_server_req_lookup_latency_count counter
ceph_mds_server_req_lookup_latency_count{ceph_daemon="mds.scooby"} 4.0
# HELP ceph_mds_server_req_lookuphash_latency_sum Request type lookup hash of inode latency Total
# TYPE ceph_mds_server_req_lookuphash_latency_sum counter
ceph_mds_server_req_lookuphash_latency_sum{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_lookuphash_latency_count Request type lookup hash of inode latency Count
# TYPE ceph_mds_server_req_lookuphash_latency_count counter
ceph_mds_server_req_lookuphash_latency_count{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_lookupino_latency_sum Request type lookup inode latency Total
# TYPE ceph_mds_server_req_lookupino_latency_sum counter
ceph_mds_server_req_lookupino_latency_sum{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_lookupino_latency_count Request type lookup inode latency Count
# TYPE ceph_mds_server_req_lookupino_latency_count counter
ceph_mds_server_req_lookupino_latency_count{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_lookupname_latency_sum Request type lookup name latency Total
# TYPE ceph_mds_server_req_lookupname_latency_sum counter
ceph_mds_server_req_lookupname_latency_sum{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_lookupname_latency_count Request type lookup name latency Count
# TYPE ceph_mds_server_req_lookupname_latency_count counter
ceph_mds_server_req_lookupname_latency_count{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_lookupparent_latency_sum Request type lookup parent latency Total
# TYPE ceph_mds_server_req_lookupparent_latency_sum counter
ceph_mds_server_req_lookupparent_latency_sum{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_lookupparent_latency_count Request type lookup parent latency Count
# TYPE ceph_mds_server_req_lookupparent_latency_count counter
ceph_mds_server_req_lookupparent_latency_count{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_lookupsnap_latency_sum Request type lookup snapshot latency Total
# TYPE ceph_mds_server_req_lookupsnap_latency_sum counter
ceph_mds_server_req_lookupsnap_latency_sum{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_lookupsnap_latency_count Request type lookup snapshot latency Count
# TYPE ceph_mds_server_req_lookupsnap_latency_count counter
ceph_mds_server_req_lookupsnap_latency_count{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_lssnap_latency_sum Request type list snapshot latency Total
# TYPE ceph_mds_server_req_lssnap_latency_sum counter
ceph_mds_server_req_lssnap_latency_sum{ceph_daemon="mds.scooby"} 0.000190177
# HELP ceph_mds_server_req_lssnap_latency_count Request type list snapshot latency Count
# TYPE ceph_mds_server_req_lssnap_latency_count counter
ceph_mds_server_req_lssnap_latency_count{ceph_daemon="mds.scooby"} 1.0
# HELP ceph_mds_server_req_mkdir_latency_sum Request type make directory latency Total
# TYPE ceph_mds_server_req_mkdir_latency_sum counter
ceph_mds_server_req_mkdir_latency_sum{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_mkdir_latency_count Request type make directory latency Count
# TYPE ceph_mds_server_req_mkdir_latency_count counter
ceph_mds_server_req_mkdir_latency_count{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_mknod_latency_sum Request type make node latency Total
# TYPE ceph_mds_server_req_mknod_latency_sum counter
ceph_mds_server_req_mknod_latency_sum{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_mknod_latency_count Request type make node latency Count
# TYPE ceph_mds_server_req_mknod_latency_count counter
ceph_mds_server_req_mknod_latency_count{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_mksnap_latency_sum Request type make snapshot latency Total
# TYPE ceph_mds_server_req_mksnap_latency_sum counter
ceph_mds_server_req_mksnap_latency_sum{ceph_daemon="mds.scooby"} 0.19603525
# HELP ceph_mds_server_req_mksnap_latency_count Request type make snapshot latency Count
# TYPE ceph_mds_server_req_mksnap_latency_count counter
ceph_mds_server_req_mksnap_latency_count{ceph_daemon="mds.scooby"} 1.0
# HELP ceph_mds_server_req_open_latency_sum Request type open latency Total
# TYPE ceph_mds_server_req_open_latency_sum counter
ceph_mds_server_req_open_latency_sum{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_open_latency_count Request type open latency Count
# TYPE ceph_mds_server_req_open_latency_count counter
ceph_mds_server_req_open_latency_count{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_readdir_latency_sum Request type read directory latency Total
# TYPE ceph_mds_server_req_readdir_latency_sum counter
ceph_mds_server_req_readdir_latency_sum{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_readdir_latency_count Request type read directory latency Count
# TYPE ceph_mds_server_req_readdir_latency_count counter
ceph_mds_server_req_readdir_latency_count{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_rename_latency_sum Request type rename latency Total
# TYPE ceph_mds_server_req_rename_latency_sum counter
ceph_mds_server_req_rename_latency_sum{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_rename_latency_count Request type rename latency Count
# TYPE ceph_mds_server_req_rename_latency_count counter
ceph_mds_server_req_rename_latency_count{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_renamesnap_latency_sum Request type rename snapshot latency Total
# TYPE ceph_mds_server_req_renamesnap_latency_sum counter
ceph_mds_server_req_renamesnap_latency_sum{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_renamesnap_latency_count Request type rename snapshot latency Count
# TYPE ceph_mds_server_req_renamesnap_latency_count counter
ceph_mds_server_req_renamesnap_latency_count{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_rmdir_latency_sum Request type remove directory latency Total
# TYPE ceph_mds_server_req_rmdir_latency_sum counter
ceph_mds_server_req_rmdir_latency_sum{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_rmdir_latency_count Request type remove directory latency Count
# TYPE ceph_mds_server_req_rmdir_latency_count counter
ceph_mds_server_req_rmdir_latency_count{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_rmsnap_latency_sum Request type remove snapshot latency Total
# TYPE ceph_mds_server_req_rmsnap_latency_sum counter
ceph_mds_server_req_rmsnap_latency_sum{ceph_daemon="mds.scooby"} 0.325743616
# HELP ceph_mds_server_req_rmsnap_latency_count Request type remove snapshot latency Count
# TYPE ceph_mds_server_req_rmsnap_latency_count counter
ceph_mds_server_req_rmsnap_latency_count{ceph_daemon="mds.scooby"} 1.0
# HELP ceph_mds_server_req_rmxattr_latency_sum Request type remove extended attribute latency Total
# TYPE ceph_mds_server_req_rmxattr_latency_sum counter
ceph_mds_server_req_rmxattr_latency_sum{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_rmxattr_latency_count Request type remove extended attribute latency Count
# TYPE ceph_mds_server_req_rmxattr_latency_count counter
ceph_mds_server_req_rmxattr_latency_count{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_setattr_latency_sum Request type set attribute latency Total
# TYPE ceph_mds_server_req_setattr_latency_sum counter
ceph_mds_server_req_setattr_latency_sum{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_setattr_latency_count Request type set attribute latency Count
# TYPE ceph_mds_server_req_setattr_latency_count counter
ceph_mds_server_req_setattr_latency_count{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_setdirlayout_latency_sum Request type set directory layout latency Total
# TYPE ceph_mds_server_req_setdirlayout_latency_sum counter
ceph_mds_server_req_setdirlayout_latency_sum{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_setdirlayout_latency_count Request type set directory layout latency Count
# TYPE ceph_mds_server_req_setdirlayout_latency_count counter
ceph_mds_server_req_setdirlayout_latency_count{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_setfilelock_latency_sum Request type set file lock latency Total
# TYPE ceph_mds_server_req_setfilelock_latency_sum counter
ceph_mds_server_req_setfilelock_latency_sum{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_setfilelock_latency_count Request type set file lock latency Count
# TYPE ceph_mds_server_req_setfilelock_latency_count counter
ceph_mds_server_req_setfilelock_latency_count{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_setlayout_latency_sum Request type set file layout latency Total
# TYPE ceph_mds_server_req_setlayout_latency_sum counter
ceph_mds_server_req_setlayout_latency_sum{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_setlayout_latency_count Request type set file layout latency Count
# TYPE ceph_mds_server_req_setlayout_latency_count counter
ceph_mds_server_req_setlayout_latency_count{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_setxattr_latency_sum Request type set extended attribute latency Total
# TYPE ceph_mds_server_req_setxattr_latency_sum counter
ceph_mds_server_req_setxattr_latency_sum{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_setxattr_latency_count Request type set extended attribute latency Count
# TYPE ceph_mds_server_req_setxattr_latency_count counter
ceph_mds_server_req_setxattr_latency_count{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_symlink_latency_sum Request type symbolic link latency Total
# TYPE ceph_mds_server_req_symlink_latency_sum counter
ceph_mds_server_req_symlink_latency_sum{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_symlink_latency_count Request type symbolic link latency Count
# TYPE ceph_mds_server_req_symlink_latency_count counter
ceph_mds_server_req_symlink_latency_count{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_unlink_latency_sum Request type unlink latency Total
# TYPE ceph_mds_server_req_unlink_latency_sum counter
ceph_mds_server_req_unlink_latency_sum{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_server_req_unlink_latency_count Request type unlink latency Count
# TYPE ceph_mds_server_req_unlink_latency_count counter
ceph_mds_server_req_unlink_latency_count{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_sessions_average_load Average Load
# TYPE ceph_mds_sessions_average_load gauge
ceph_mds_sessions_average_load{ceph_daemon="mds.scooby"} 58.0
# HELP ceph_mds_sessions_avg_session_uptime Average session uptime
# TYPE ceph_mds_sessions_avg_session_uptime gauge
ceph_mds_sessions_avg_session_uptime{ceph_daemon="mds.scooby"} 62.0
# HELP ceph_mds_sessions_session_add Sessions added
# TYPE ceph_mds_sessions_session_add counter
ceph_mds_sessions_session_add{ceph_daemon="mds.scooby"} 13.0
# HELP ceph_mds_sessions_session_count Session count
# TYPE ceph_mds_sessions_session_count gauge
ceph_mds_sessions_session_count{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_sessions_session_remove Sessions removed
# TYPE ceph_mds_sessions_session_remove counter
ceph_mds_sessions_session_remove{ceph_daemon="mds.scooby"} 13.0
# HELP ceph_mds_sessions_sessions_open Sessions currently open
# TYPE ceph_mds_sessions_sessions_open gauge
ceph_mds_sessions_sessions_open{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_sessions_sessions_stale Sessions currently stale
# TYPE ceph_mds_sessions_sessions_stale gauge
ceph_mds_sessions_sessions_stale{ceph_daemon="mds.scooby"} 0.0
# HELP ceph_mds_sessions_total_load Total Load
# TYPE ceph_mds_sessions_total_load gauge
ceph_mds_sessions_total_load{ceph_daemon="mds.scooby"} 58.0
ceph_objecter_op_active{ceph_daemon="mds.scooby"} 0.0
ceph_objecter_op_r{ceph_daemon="mds.scooby"} 196.0
ceph_objecter_op_rmw{ceph_daemon="mds.scooby"} 0.0
ceph_objecter_op_w{ceph_daemon="mds.scooby"} 1421.0
ceph_purge_queue_pq_executed{ceph_daemon="mds.scooby"} 243.0
ceph_purge_queue_pq_executing{ceph_daemon="mds.scooby"} 0.0
ceph_purge_queue_pq_executing_high_water{ceph_daemon="mds.scooby"} 16.0
ceph_purge_queue_pq_executing_ops{ceph_daemon="mds.scooby"} 0.0
ceph_purge_queue_pq_executing_ops_high_water{ceph_daemon="mds.scooby"} 16.0
ceph_purge_queue_pq_item_in_journal{ceph_daemon="mds.scooby"} 0.0
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.