Introduce dataframe file type to replace / supplement beddb and bed2ddb
- Dominant language
- Python
- Stars
- 39
- Forks
- 22
- PR merge metrics
- No merged PRs in 30d
Description
The goal of this PR is to discuss the introduction of a new file type that replaces the `beddb` and `bed2ddb` formats. This file type will be able to store any type of data and be used as backing for gene annotations, bed-like regions, arbitrary points, etc...
## Questions to address:
1. Tile API: the current API takes a zoom level, start and end position. It works right now because any genomic data is converted to a linearized representation where chromosomes are concatenated using a given chromosome order.
Dataframe-backed files will not have this limitation. The tile API will have to have a chromosome order associated with it to indicate which data should be retrieved between coordinates x0 and x1.
Example API:
```
def get_1D_tile_data(
filename='my_file.tsv',
tile_position=[1,0],
group_column=['chr'],
position_columns=['start', 'end'],
group_order=[('chr1', 1000), ('chr2', 5000), ('chrX', 4000), ('chrM', 3000)]
)
```
2. Column to use as the index: A dataframe may have the start and end positions at arbitrary positions. The request should include an indicator of which columns to use for the positions of the data.
## Use cases
1. Replacing the current beddb and tile bed2db formats.
## Perfomance
Filtering a 970K line file takes about 200ms. It may be possible to improve this through parallelization, sorting, indexing or subdividing the file into sections (e.g. chromosomes)

Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.