IQSS / IQSS/dataverse

Unclear how to find more granularity of files beyond "File Type" (application, tabulardata, data, etc.)

Open
#3,597 3 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Feature: Metadata Feature: Search/Browse Type: Bug User Role: Curator UX & UI: Design
Dominant language
Java
Stars
1.1k
Forks
564
Avg merge
2d 2h
Merged PRs (30d)
29

Description

If you got to https://dataverse.harvard.edu/dataverse/polbehavior and click "Files" you see "File Type" as in the screenshot below, but the detail is not very granular. You see files flagged as application, tabulardata, data, etc.:

political_behavior_dataverse_-_2017-01-26_12 20 34

What if you want a specific type of file such as an R file? It's possible but you have to know which undiscoverable field to search on, which in this case is fileContentType. Specifically, a search for fileContentType:type/x-r-syntax ( https://dataverse.harvard.edu/dataverse/harvard?q=fileContentType%3Atype%2Fx-r-syntax ) will show 1,553 as of this writing in the Harvard Dataverse:

harvard_dataverse_-_2017-01-26_12 30 24

fileContentType (and related fields, possibly) should be exposed on the Advanced Search page, which currently looks like this:

advanced_search_-harvard_dataverse-_2017-01-26_12 25 10

The way I figured out to look for type/x-r-syntax is by looking at a list of files for a dataset in JSON format with this: https://dataverse.harvard.edu/api/datasets/:persistentId?persistentId=doi:10.7910/DVN/SCN9LA

The background on this issue is that this morning @christophergandrud were discussing how one would identify all the R files in an installation of Dataverse. As a starting point, we looked at http://dx.doi.org/10.7910/DVN/SCN9LA (screenshot below) which I happened to know has an R file called "Thal PB Replication.R" that can be used to reproduce plots in a paper. (Once this platform for reproducibility has been publicly announced, I'll mention it. 😄 )

Anyway, with fileContentType:type/x-r-syntax as a starting point, one should be able to iterate through all these R files via Search API: http://guides.dataverse.org/en/4.6/api/search.html#iteration

(This issue is related: What are the allowed search fields for the Search API q parameter? #2558).

replication_data_for_class_isolation_and_affluent_americans perception_of_social_conditions-political_behavior_dataverse-_2017-01-26_12 11 27

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Advanced Search page and the Search API field discussion in issue #2558. Trace how searchable file metadata is represented, especially fileContentType and related fields, then expose the appropriate fields for advanced searches and verify that users can find specific file types such as R files.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.