daphne-project / daphne-project/daphne

Efficient File I/O Plug-ins for Widely-used File Formats

Open
#985 1 comment 0 reactions 0 assignees View on GitHub
AMLS summer 2026 IO LDE summer 2026 LDE winter 2025/26 student project
Dominant language
C++
Stars
81
Forks
83
PR merge metrics
No merged PRs in 30d

Description

**Motivation:** The input to integrated data analysis pipelines, that combine query processing, machine learning, and high-performance computing, could be provided in various general-purpose and domain-specific file formats. To embrace a large variety of such formats, DAPHNE is extensible w.r.t. to file readers/writers (pull request coming soon), i.e., expert users can add their own file I/O plug-ins without touching the source code of DAPHNE. However, so far there is only a limited number of plug-ins available.

**Task:** (in C++) Implement additional file I/O plug-ins for a range of widely used file formats for different data modalities, such as tabular data, (sparse) matrices/graphs, audio, images, and time series. These plug-ins may be based on existing open-source libraries (with compatible license). Apply format-specific and format-agnostic tricks to make the file I/O efficient, e.g., by exploiting parallelism or pushing down certain operations into the readers.

**Hints:**
*coming soon*

Contributor guide

Open the contributing guide

Research direction

Start by reviewing DAPHNE's file reader/writer plug-in extension described in the issue and identify the supported plug-in entry points. Choose a specific widely used format and data modality, then verify that the added plug-in reads or writes it efficiently using compatible open-source libraries; completion should include the plug-in and evidence of efficient file I/O.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.