ci: separate hermetic unit tests from online E2E tests

Problem

The PR workflow runs make unittest, but the target executes only:

pytest tests/test_dataset.py::TestDataSet::test_download_small

This test downloads data from S3 and Aliyun OSS, so it is an online E2E/integration test—not a unit test.

As a result:

• PR CI does not run the pure local unit-test suite.
• CI depends on external network and storage availability.
• New local tests can pass manually but remain unprotected by CI.
• Failures do not clearly distinguish code regressions from external-service failures.

The repository already defines an integration pytest marker, but classification is incomplete. Running:

pytest --collect-only -m "not integration" tests

still collects service-dependent tests and fails on missing optional SDKs such as Chroma, Pinecone, Turbopuffer, and MySQL.

Some files also mix both test types, such as test_milvus.py, test_pgvector.py, and test_dataset.py.

Proposed approach

Separate tests into two explicit categories:

1. Pure local unit tests

   • No network access.
   • No live database or external service.
   • No credentials or large dataset downloads.
   • Deterministic and fast.
   • Run on every pull request.

2. Online E2E tests

   • Require dataset downloads, a running database, cloud credentials, containers, or optional provider SDKs.
   • Run in dedicated jobs or workflows with their prerequisites documented.
   • Can be scheduled, manually triggered, or enabled for selected PRs.

Prefer separate directories so unit-test collection does not import E2E-only dependencies:

tests/
├── unit/
└── e2e/

Add explicit Make targets:

unit-test:
	PYTHONPATH=`pwd` python3 -m pytest tests/unit

e2e-test:
	PYTHONPATH=`pwd` python3 -m pytest tests/e2e -svv

The existing unittest target can temporarily alias unit-test for compatibility.

Acceptance criteria

• make unit-test passes with only pip install -e ".[test]".
• Every PR runs the complete local unit-test suite.
• Unit tests require no network, credentials, external services, or downloaded datasets.
• Online E2E tests have a separate command and CI job/workflow.
• test_download_small is moved to the E2E category.
• Mixed test modules are split or clearly reorganized.
• Optional E2E dependencies and service prerequisites are documented.
• CI job names clearly distinguish unit tests from online E2E tests.

Start with the existing make unittest target and pytest integration-marker configuration, then inspect tests/test_dataset.py, test_milvus.py, and test_pgvector.py for mixed test types. Verify collection and execution using the proposed unit and E2E commands. Done means local unit tests run without external dependencies, test_download_small is categorized as E2E, prerequisites are documented, and PR jobs distinguish both suites.