Azure / Azure/azure-linux-extensions

Filesystem: TransfersPerSecond staying at 0/s

Open
#634 0 comments 6 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
333
Forks
278
Avg merge
2d 9h
Merged PRs (30d)
4

Description

I'm trying to set up LAD on a MongoDB cluster in Azure. We want to monitor IOPS on attached disks mounted at various points in the filesystem, specifically these directories (snipped from `mount`):

```
/dev/sdc1 on /media/mongodata type xfs (rw,relatime,attr2,inode64,noquota)
/dev/sdf1 on /media/mongodata/qascaling.45loadTest/index type xfs (rw,relatime,attr2,inode64,noquota)
/dev/sde1 on /media/mongodata/local type xfs (rw,relatime,attr2,inode64,noquota)
/dev/sdd1 on /media/mongodata/qascaling.45loadTest/collection type xfs (rw,relatime,attr2,inode64,noquota)
```

A sample bit of the configuration would look like this:

```json
{
"annotation": [
{
"displayName": "collection IOPS",
"locale": "en-us"
}
],
"class": "filesystem",
"condition": "Name=\"/media/mongodata/qascaling.45loadTest/collection\"",
"counter": "TransfersPerSecond",
"counterSpecifier": "/builtin/Filesystem/TransfersPerSecond/collection",
"type": "builtin",
"unit": "CountPerSecond"
}
```

Despite generating lots of load using `dd`, I can't seem to get the counter for this or any other custom metric to show any traffic. The only exception is when I set up one for the root:

```json
{
"annotation": [
{
"displayName": "root IOPS",
"locale": "en-us"
}
],
"class": "filesystem",
"condition": "Name=\"/\"",
"counter": "TransfersPerSecond",
"counterSpecifier": "/builtin/Filesystem/TransfersPerSecond/root",
"type": "builtin",
"unit": "CountPerSecond"
}
```

I know I can use the `Disk` builtin counters but we've had issues before where the SCSI disk identifiers are different between instances so we can't be sure if `sdb` on one machine is the same mount point as `sdb` on another. (We use UUIDs to mount the data disks to ensure we're attaching disks to the right place on the file system).

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the filesystem metric configuration for TransfersPerSecond, comparing the entries conditioned on "/" and the mounted paths under /media/mongodata. Reproduce the report by generating load with dd and check whether mounted-path counters emit traffic while the root counter does. Done means the intended custom filesystem counters report activity reliably for these mounts.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, mongodb, python
Domain
cloud, observability
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.