CGI-FR / CGI-FR/LINO

Support Multiple Export Formats for lino pull Command

Open
#309 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
23
Forks
5
PR merge metrics
No merged PRs in 30d

Description

Currently, the `lino pull` command supports exporting data in JSONL format. While JSONL is a flexible and widely-used format, it may not always be suitable for all use cases. To address the needs of a broader range of users and applications, we propose enhancing the `lino pull` command to support multiple export formats.

**New Formats to Support:**
1. **SQL:** Generate SQL scripts that can be executed or saved as files to initialize a test database. This is particularly useful for testing applications that do not support JSONL, such as Spring Boot applications.
2. **Avro/Parquet:** Optimize CPU consumption and file compression by using binary serialization formats like Avro or Parquet.

**Proposed Changes:**
- Add a `--format` flag to the `lino pull` command, allowing users to specify the export format.
- Implement export options for SQL and Avro/Parquet formats.
- Ensure compatibility with existing flags and options in the `lino pull` command.
- Update the documentation to reflect the new export formats and usage examples.

**Impact:**
- Users will have more flexibility in choosing the format that best suits their application's needs.
- Improve performance and storage efficiency for users who require compressed binary formats.

**Acceptance Criteria:**
- The `lino pull` command successfully exports data in the specified format (SQL, Avro, Parquet).
- Documentation and usage examples are updated.
- Comprehensive tests are added for each export format.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.