Ready to embrace VCF 4.3 UTF-8 and percent encoding?

Open

#333 opened on Feb 7, 2016 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty: 5/5
Estimated time: Over a week
Newbie friendliness: 25/100

Issue type: Feature
Clarity: Needs clarification
Activity status: Stale
Tech stack: c
Domain: bioinformatics

Research direction

Start with htslib/vcf.c and htslib/vcf.h, then inspect the existing tests for VCF string parsing and writing. Establish the expected VCF 4.3 UTF-8 and percent-encoding behavior, especially for string vectors containing commas; done means the behavior is tested and the approach is ready for maintainer review.

Written by the indexing model from the issue text.

Description

I am working on an htslib branch that adds support for UTF-8 and percent encoded string values to htslib/vcf.{c,h}. The UTF-8 support is mainly transparent, since virtually all keys, which have fixed length encoding assumptions, are limited to ASCII values by the spec. Percent decoding is performed only when reading from files and encoding is done when writing to files. The major special case is when handling string vectors, since the current code uses commas as delimiters without checking strings do not contain commas. My provisional solution is to use one of the disallowed non-printable characters as a string delimiter, to avoid having to apply percent encoding to in-memory representations.

More testing is needed before I propose a PR, but I wanted to sanity check with the main htslib maintainers that:

this functionality is desirable
my basic approach is feasible
and someone has already done this work in a branch and I'm re-inventing the wheel

Dominant language: C
Stars: 950
Forks: 475
Avg merge: 3d 13h
Merged PRs (30d): 11

Contributor guide

Open the contributing guide

First steps

Read the whole issue, then the project's contributing guide.
Comment on the issue to say you are picking it up — it saves two people doing the same work.
Fork the repository and make your change on a branch.
Open a pull request that references the issue number.