Devographics / Devographics/surveys

Lack of Diversity in State of JS 2024 Video Creators List

Open
#254 24 comments 0 reactions 0 assignees View on GitHub
diversity State of JS 2024
Dominant language
No language data
Stars
69
Forks
7
PR merge metrics
No merged PRs in 30d

Description

*Note: this post centers women in the discussion because this is what the feedback I've received was about, but the same issue applies to any minoritized segment of the developer community, whether BIPOC, LGBTQ, people experiencing disabilities, and more*

## The Issue

The "Which video creators do you follow?" question in the [State of JS 2024 survey](https://survey.devographics.com/en-US/survey/state-of-js/2024) originally featured a list that included no women creators:

![bafkreigwplbusr5xudcpg5fggnpibrjfowtvta42pkfft6lnrlozo77axa](https://github.com/user-attachments/assets/dd39c97b-0e08-4b11-94c1-35abdc3e2fbe)

## The Root Cause

There are two basic ways to collect data in a survey: through a freeform text field, or through a list of predefined options (which in the State of JS case also supports an optional "other answers…" freeform field). Each of them have their pros and cons:

#### Predefined Options

Pros:
- More answers

Cons:
- Biases results by favoring items included among the predefined options.

#### Freeform Textfield

Pros:
- Unbiased:

Cons:
- Requires more effort and recall on the respondent's part, which translates into fewer answers.

## Old Solution

The solution I originally settled on was to alternate between both formats each year. In 2023, this question used a freeform text field:

Screenshot 2024-11-14 at 9 56 08

I then used the resulting data to populate the set of predefined options for the 2024 edition. This strategy aimed to balance the pros and cons of both formats, by ensuring maximum question participation with predefined options one year, while hopefully "resetting" any bias by switching back to freeform input the following year.

In other words, because respondents did not mention women video creators in 2023, they were then absent from the list I compiled for the 2024 edition.

## New Solution

The downside is that without this background knowledge, it can see like this year's list of predefined options is the result of an arbitrary process that doesn't include any consideration for diversity and inclusion.

For that reason I've decided to just **permanently keep the "Video Creators" and "People" question as freeform inputs**, to avoid giving off this impression and thus alienating the very audience the survey lacks the most.

## Survey Goals

I do feel the need to add one more thing: especially lately, being "anti-DEI" in the tech community is becoming worryingly trendy. So I want to be absolutely clear that this is not what's going on here. I firmly believe that inclusivity and representation efforts are important for both moral and practical reasons.

Which is exactly why I believe the survey's goal should be to *measure* the success of those efforts, and not to try and put a thumb on the scale one way or another. So even though artificially drafting a more diverse list would quiet a lot of the criticisms around this issue, I feel like it would not be conducive to **actually measuring** what the survey's audience thinks.

(The fact that the survey audience itself might be biased or unrepresentative is a related – and quite valid – concern. But again, masking this issue by tweaking the list would not actually be fixing it either.)

## What You Can Do

If you are concerned about this issue, the two best things you can to do are:

1. Follow more women developers and video creators (see [this Bluesky list for example](https://bsky.app/profile/charlesvillard.co/post/3latxqymogc2j))
2. Enter their name in the survey

And if you happen to be a woman video creator yourself, I would suggest encouraging your audience to take the survey. I realize this comes across as self-serving coming from the person who runs the survey, but it's the surest way to make a difference in the stats.

## What Do You Think?

I'm well aware that this comes at a cost for women/BIPOC/LGBTQ/disabled/etc. content creators, since they end up drowned out by the stats.

For that reason, I am not saying this is a perfect approach. I've gone back and forth over this issue many times in the past, and I probably will again in the future.

For example, maybe you think that using the surveys' platform to promote women creators should take priority over its other goals, and that's certainly understandable as well.

So if you have a suggestion on how to tackle this issue, I would love to hear more!

## Recommended Reading

- [A look at gender dynamics in the developer community](https://dev.to/sachagreif/a-look-at-gender-demographics-in-the-developer-community-part-1-4d76)

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue concerns the State of JS 2024 survey's “Video Creators” and “People” questions; start by locating their YAML survey definitions and inspecting how answer formats are configured. Done means both questions permanently use freeform input while retaining the optional “other answers…” behavior. The discussion is 668 days quiet and contains no linked implementation.

Written by the indexing model from the issue text.

Assessment

Domain
content
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.