TWE: IS25: RP026: Preference Test for New Design Colors Survey: Analyze Data

### Dependencies
- [x] #768
- [x] #779

### Overview
We need to conduct data analysis so we can inform design team on the color preference of the design system.

### Action Items
#### Get familiar with this research plan and its status
- [x] Check out each link in the section of `Resources for creating this issue` to gather what this research plan is for and why we are doing the research
- [x] RP026 Research Plan Google Doc
- [x] RP026 Research Roadmap
- [x] RP026 Wiki Page
- [x] RP026 Survey - Preview Mode
- [x] RP026 Survey - Editing Mode
- [x] RP026: TWE Color Preference Test (Responses)
- [x] RP026 Edited Survey Responses

#### Conduct the Quantitative Data Analysis
- [x] Locate the data analysis plan for this research plan
- [x] Refer to the data analysis section RP026's research plan to see if specific data analysis plan was in place
- [ ] If yes, read through the methods or techniques mentioned in the analysis plan to see if they make sense to you. A specific data analysis plan will denote what type of analyses will be used.
- [ ] Check the data we have collected and the research goals to see if all the specific analysis techniques or methods can be applied to address the research goals.
- [ ] If yes, make a comment in this issue to confirm that.
- [ ] If no, follow the instructions in the following section to create a new issue to develop a feasible data analysis plan.
- [x] If there is not a specific data analysis plan, create a new issue to come up with the data analysis plan. Please go create a new issue for creating a data analysis plan/checklist now, and assign yourself to that issue.
- [x] Finish setting up and customize the new data analysis plan/checklist issue so it will be ready to be worked on.
- [x] Add the `dependency` label to this issue.
- [x] Add the new issue number from the issue you just created in the `Dependencies` section above.
- [x] Move this issue to `Icebox` column.
- [x] Remove yourself from the issue.
- [x] Leave a comment in this issue to notify team of the current status.
- [x] Start working on the `data analysis plan/checklist` issue.
- [ ] If you don't have experience or are not comfortable with quantitative or statistical analysis, please let the UXR Lead(s) know, so they could coordinate with you on the next steps.
- [x] Once the `data analysis plan/checklist` issue gets finished, move this issue into `Prioritized backlog` or `In progress` column depending on availability.
- If it is in `In progress` column, please assign yourself to this issue.
- [ ] If you are not sure if there is a specific data analysis plan in place, reach out to UXR Lead(s) to get more clarification.
- [x] List out the specific data analysis plan/checklist in Resources 2.01
- [ ] If there is a specific data analysis plan in the research plan document, make a bookmark of the data analysis section, copy the bookmark, and paste the URL in Resources 2.01 in `()` so it becomes a hyperlink
- [ ] If the specific data analysis plan/checklist is generated in another issue comment, choose copy link to copy the issue comment and paste it in `()` in Resources 2.01
- [x] Read through the Data Analysis Plan/Checklist in this section
- [x] Data analysis plan:
- [x] Preference questions in pairs, i.e. A vs B, B vs C, A vs C:
- Win rate
- McNemar Exact test with CI: https://en.wikipedia.org/wiki/McNemar%27s_test; https://measuringu.com/mcnemar-exact-test/
- Do 2x2 table for each design, so 3 times; then calculate Confidence Interval, and to adjust the p value.
- [x] Scale/rating questions: In this case, it is a within-subjects design and ordinal variables
- [x] Descriptive analysis: i.e. average and/or median and/or mode ratings from 3 questions from all respondents for each color scheme (9 values), average and/or median and/or mode ratings from all 9 questions from each respondent (36 values), average and/or median and/or mode ratings for each color scheme from each respondent (36x3 values), average and/or median and/or mode ratings for each 1-7 rating question in each color scheme from each respondent (36x9 values), frequency distribution for each color scheme, etc.
- [x] Inferential analysis: Are the ratings statistically different?
- [x] Sample size: Use G*Power to calculate if we have a big enough sample size for conducting inferential stats.
- [x] If we do not, then skip the inferential stats. If we do, then follow the steps below:
- [x] Need to calculate Cronbach’s alpha to determine the internal consistency among the three rating questions, meaning if it is about the same topic. Then we can conduct more inferential stats if appropriate.
- https://stats.oarc.ucla.edu/spss/faq/what-does-cronbachs-alpha-mean/#:~:text=Cronbach's%20alpha%20is%20a%20measure,that%20the%20measure%20is%20unidimensional.
- https://www.jstor.org/stable/3648121?seq=1#metadata_info_tab_contents
- https://www.pbsciences.org/?mno=18637
- [x] If treating the variables as ordinal variables:
- Kruskal-Wallis H test: if treated as ordinal data (differences among the medians among the three ratings if we assume they are independent groups of ratings)
- A Mann-Whitney test (a non-parametric test assuming that the variable is ordinal)
- [x] If treating the variables as interval variables:
- One-way repeated measures ANOVA (compare the means of the three ratings if we assume they are independent groups of ratings)
- Post hoc analysis might be needed
- [x] Multi-select questions: Select as many out of the 19 motional responses within each design, i.e. design A, design B, design C:
- First treat each word as a binary variable, so if selected, then 1, not selected, then 0, to code the data from the survey
- Descriptive analysis:
- counting selected emotional responses (in frequencies) and calculate the percentages out of the number of participants and out of the number of answers (for each color and overall);
- count the # of positive and negative words selected by each respondent for each color, assign a positive and negative word a value, then give each respondent's emotional response a score kinda like a c score
- Inferential analysis:
- Sample size: Use G*Power to calculate if we have a big enough sample size for conducting inferential stats.
- If we do not, then skip the inferential stats. If we do, then follow the steps below:
- One-way repeated measures ANOVA (compare the means of the three ratings if we assume they are independent groups of ratings)
- [x] Review analysis method(s) or technique(s) based on research goals, data (to be) collected, and resources
- [ ] Suggest any change(s) needed
- [ ] Perform the data analyses
- [x] Make a copy of the spreadsheet in Resources 1.07
- [x] Copy and paste the link of the copy of the spreadsheet for data analysis in Resources 2.02
- [ ] Make sure to conduct analyses in new tabs in the spreadsheet, and rename each tab with the analysis name
- [ ] Please make some notes in the spreadsheet as you conduct the analyses when needed
- [ ] If other software or tools are used, please list out the software or tools in a new tab and copy or screenshot the syntax and/or results to show the process
- [ ] Check if any qualitative data has been collected from open-ended questions
- [ ] If yes, collect any quotes from open-ended questions to corroborate the findings
- [ ] If no, skip this step
- [ ] Once all the analyses are done, leave a comment and @ the UXR lead(s) to inform of your progress.
- [ ] Move the issue to `Question/Review` column

#### Review
- [ ] Peer review
- [ ] Research lead review

#### Share the Results - this maybe for next phase, Data Synthesis
- [ ] Slide deck or report
- [ ] List out the new spreadsheet as Resources 2.02 with name and URL

#### Update the Wiki Page

### Resources/Instructions
#### Resources for creating this issue
- 1.01 [RP026: Research Plan Google Doc](https://docs.google.com/document/d/19PZ6gy3BB8tE9j5bjpHkOM4shdZLM7zQVyXBPuccO-s/edit?usp=sharing)
- 1.02 [TWE: Research Roadmap: Other](https://docs.google.com/document/d/1j0FOrYd6eyoSb3u8Ek0QDZeFq9_9Qecz8JtUG021ZoA/edit?usp=drive_link)
- 1.03 [RP026 Wiki Page](https://github.com/hackforla/internship/wiki/Research-Plan-26:-Usability-Study-for-New-Design-Colors)
- 1.04 [RP026: Color Preference Test - Preview Mode](https://docs.google.com/forms/d/19uXm_HduYvPxZTUXTCWY6RFrYADOiPYLvUYbnmUFZz0/preview)
- 1.05 [RP026: Color Preference Test - Editing Mode](https://docs.google.com/forms/d/19uXm_HduYvPxZTUXTCWY6RFrYADOiPYLvUYbnmUFZz0/edit)
- 1.06 [RP026: TWE Color Preference Test (Responses)](https://docs.google.com/spreadsheets/d/1jGTNZu5cgO2zU_S7Oxudpv3x0A4eoBoDJvyH-x_xXFY/edit?usp=sharing)
- 1.07 [RP026 Edited Survey Responses](https://docs.google.com/spreadsheets/d/1fIieSNmfYvjUFtdYsVpFMqgN6fPNxGaztde9Rag1-Ts/edit?usp=sharing)

#### Resources gathered during the completion of this issue
- 2.01 [Data Analysis Plan/Checklist](https://github.com/hackforla/internship/issues/779#issuecomment-3335224612)
- 2.02 [RP026 Data Analysis Spreadsheet](https://docs.google.com/spreadsheets/d/1FU5ngEGcOl-VkNu6hjk_zD1Azkf6Pwz0PsJwQY6Z9UI/edit?usp=sharing)
- 2.03