Import/Export: ✨ Quick CSV Validation for dotCMS Import Endpoint

Problem Statement

Developers and administrators face issues when uploading CSV files that do not meet certain structural and formatting standards, which can lead to errors in the system. Quick validations are required to ensure that these files are appropriate for processing without needing to parse each line, thus ensuring efficiency and integrity.

Objectives

• Implement quick, surface-level validations for CSV files on import to ensure they meet predefined standards.
• Validate CSV files against a defined set of criteria including format, size, and encoding without fully parsing the file.

User Statement

As a developer using dotCMS, I want quick preliminary checks performed on my uploaded CSV files to ensure they meet necessary standards before being processed, so I can avoid integration and formatting issues.

External Links

[No external links available.]

Assumptions & Initiation Needs

• Assumes that there is a framework in place for accessing and reading file metadata and contents quickly.
• Requires setting up parameters for file size limits and valid encoding formats that are configurable.
• Quality Assurance Notes & Workarounds
• QA should prepare various test cases covering different file sizes, formats, with and without the proper extensions, and files with various forms of corruption to ensure robustness.

Technical Details

No specific details about dotCMS versions or technologies other than general handling of CSV files and character encodings.

```[tasklist]

Acceptance Criteria

• The system rejects files that do not have a '.csv' extension.
• The system checks that the file is in a standard CSV format.
• The system checks that the files have expected headers (and columns)
• The system ensures that the file can be opened and is not corrupt.
• Make sure the file has an encoding compatible with the system, such as UTF-8.
• Make sure the file uses the correct delimiter, usually a comma.
  \`

Start by locating the dotCMS CSV import endpoint and the existing path for accessing uploaded file metadata and contents. Clarify the configurable size and encoding rules, expected headers and delimiter, and how validation failures should be reported; done means the acceptance criteria are covered by tests for valid, malformed, oversized, incorrectly encoded, and incorrectly structured CSV files.