astropy / astropy/astropy-project

Run a user survey

Open
#11 23 comments 0 reactions 0 assignees View on GitHub
community :handshake:
Dominant language
TeX
Stars
48
Forks
48
PR merge metrics
No merged PRs in 30d

Description

At the NumFOCUS summit, we discussed the merits of running a short user survey to gather general information about user experience. For example, both [dask](https://github.com/dask/dask/issues/4748) and [pandas](https://github.com/pandas-dev/pandas/issues/27477) have recently run such surveys.

We have run surveys (either via google form or by mailing list post) in the past, but recently these have been for very specific goals (assessing performance, python 2 to 3, etc.). Others have also run more general surveys of the astronomical community about, e.g., [software use](https://arxiv.org/abs/1507.03989).

This issue is meant to start a discussion about whether we think it would be useful to run a similar survey. However, I think we should mainly discuss this face-to-face at the coordination meeting, and this issue should just serve as a place to remind us and collect broader feedback.

This is all open for discussion, but I was imagining a short, information-gathering survey to assess how users use and perceive the astropy core package. For example, what operating systems they use, how they learn to use astropy, experience contributing to astropy, etc.

## Goals of a survey

I've included here a few goals for running such a survey, but please suggest modifications or additions.

* Gather statistics about the systems, computers, and environments in which people are using astropy
* Gather some very basic demographic statistics, e.g., at least country of residence/work, position (academic, industry, etc.),
* Gather statistics about what subpackages our users use the most, and what other packages are being used with astropy most frequently
* Gather information about points of entry to astropy
* Gather information about what educational materials people use, attendance to conferences (e.g., PyAstro, AstroHackWeek, AAS hack day, etc.)
* Gather information about perceptions of the developer community
* Understand whether our users know how to (if they want/need to): report a bug, get help/support, contribute code

## Open threads

* We should discuss or coordinate with NumFOCUS on how to review the questions. For example, is there a social scientist we can pay to consult on this?
* If we want to publish results (in a journal, e.g., BAAS), do we need IRB approval?
* Do we want to borrow questions from the [Python developers survey](https://www.jetbrains.com/research/python-developers-survey-2018/) to compare / benchmark against the broader Python community? Is that useful?

Discuss!

Contributor guide

No contributing guide indexed for this repository

Research direction

No files, tests, or implementation entry points are identified. Start by reviewing the survey goals and open threads, then determine whether the project wants a survey and resolve the NumFOCUS, IRB, comparison, and publication questions; done would be an agreed survey plan.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
content
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.