matplotlib / matplotlib/matplotlib

[Bug]: Misalignment of Labels and Incorrect Density Values in `ax.hist()` for Categorical Variables

Open
#28,029 10 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

topic: categorical topic: hist
Dominant language
Python
Stars
23.2k
Forks
8.5k
Avg merge
1d 6h
Merged PRs (30d)
66

Description

### Bug summary

I have stumbled upon an issue while using `ax.hist()` with categorical variables and setting `density = True`. The issue arises from the following behaviors:

1. **Incorrect density computation:** When a categorical variable is being used, the function internally converts the variable to integers (seen [here](https://github.com/matplotlib/matplotlib/blob/380bba1d0b4f3665afdb164ad784dd234ccb8de3/lib/matplotlib/axes/_axes.py#L6922)). Consequently, the bin width isn't necessarily 1, leading to miscalculated density values. Underlying function `np.histogram`, which is used internally, takes into account the bar width while ensuring the entire area equates to 1. However, when dealing with categoricals, we'd expect the sum of the column heights to be 1.
2. **Misalignment of labels:** The function doesn't position labels precisely in the center of the bars. To align the x-ticks at the centers, one could ostensibly utilize the `bins` that are produced by the function. However, it's challenging to unveil how the 'category to int' conversion (mentioned above) was performed. From the code [here](https://github.com/matplotlib/matplotlib/blob/380bba1d0b4f3665afdb164ad784dd234ccb8de3/lib/matplotlib/axes/_axes.py#L6922), it seems the elements in the first array are converted (in case there are multiple arrays) in the order they emerge, with the resultant map used to convert elements of the second array. If new categories are introduced, they are assigned the subsequent free integer. Unfortunately, there are no documented details about this process.

### Code for reproduction

```Python
fig, ax = plt.subplots()
ax.hist(["a","b","c","d"], bins=4, density=True)
```

### Actual outcome

`(array([0.33333333, 0.33333333, 0.33333333, 0.33333333]), array([0. , 0.75, 1.5 , 2.25, 3. ]), )`

### Expected outcome

In the case the density parameter is passed I would expect that the sum of the columns would be 1 as we are dealing with categorical variables and the width of the bar should not be taken into account. Also it would be nice to be able to know how they are converted to integer to be able to plot the labels correctly in the plot.

### Additional information

Proposed improvements to the above behaviors:

- It may be beneficial to reconsider how the function calculates densities when managing categorical data, explicitly setting the bin width to 1.
- The function could be enhanced to bijectively handle the 'category to int' conversion and provide clear documentation to make it more trustworthy.
- A solution needs to be implemented to ensure labels are correctly positioned at the center of the bars.

### Operating system

OS/X

### Matplotlib Version

3.8.3

### Matplotlib Backend

MacOSX

### Python version

3.10.14

### Jupyter version

_No response_

### Installation

conda

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in lib/matplotlib/axes/_axes.py around the categorical conversion referenced in the issue, then run the provided ax.hist() reproduction. Compare the density values and bar-label positions with the expected categorical behavior; done means the density and category-to-position behavior are corrected and documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
data-visualization
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.