ORC Test Plan: Unit and Component Testing

This document outlines a comprehensive plan to add unit and component-level testing to the ORC project using Google Test (gtest). The project currently has a robust integration test suite (`orc_test`) that validates end-to-end functionality by compiling source files, processing object files, and comparing detected ODR violations against expected results. This plan focuses on filling the gap with lower-level tests that verify individual components and modules in isolation.

## Current Testing State

### Existing Integration Tests
- **Location**: `test/battery/` directory
- **Framework**: Google Test (gtest)
- **Test Runner**: `test/src/main.cpp` and `test/src/fixed_vector_tests.cpp`
- **Coverage**: End-to-end ODR violation detection scenarios
- **Test Types**:
- Byte size mismatches
- Data member location conflicts
- Vtable element location issues
- Calling convention mismatches
- Static/free function conflicts
- Class/struct differences
- Typedef issues
- Inline attribute handling
- Accessibility conflicts

### Testing Infrastructure Already in Place
- Google Test is already a project dependency (v1.14.0)
- `CMakeLists.txt` includes gtest setup (lines 71-81)
- `orc_test` executable exists with proper linking (lines 187-213)
- CI/CD pipeline exists (`.github/workflows/build-and-test.yml`)

## Testing Strategy

### Unit Tests
**Purpose**: Test individual functions, classes, and methods in isolation

**Scope**:
- Utility functions (string manipulation, demangle operations)
- Data structures (fixed_vector, hash tables, memory pools)
- DWARF parsing primitives
- File format parsers (Mach-O, COFF, AR)
- Attribute processing
- Settings/configuration handling

### Component Tests
**Purpose**: Test larger subsystems and their interactions

**Scope**:
- Object file registry operations
- DWARF structure processing
- Symbol registration and lookup
- ODR violation detection logic
- Report generation
- Async/parallel processing coordination

## Implementation Plan

### Phase 1: Infrastructure Setup (Weeks 1-2)

#### 1.1 Directory Structure
Create a new test directory structure:
```
test/
├── battery/ # Existing integration tests
├── src/
│ ├── main.cpp # Existing integration test runner
│ └── fixed_vector_tests.cpp # Existing unit tests
├── unit/ # New: Unit tests
│ ├── CMakeLists.txt
│ ├── main.cpp
│ ├── str_tests.cpp
│ ├── hash_tests.cpp
│ ├── memory_tests.cpp
│ ├── dwarf_constants_tests.cpp
│ ├── settings_tests.cpp
│ └── demangle_tests.cpp
└── component/ # New: Component tests
├── CMakeLists.txt
├── main.cpp
├── object_file_registry_tests.cpp
├── dwarf_parser_tests.cpp
├── macho_parser_tests.cpp
├── coff_parser_tests.cpp
├── ar_parser_tests.cpp
├── odrv_detection_tests.cpp
└── report_generation_tests.cpp
```

#### 1.2 CMake Configuration Updates

**Update `CMakeLists.txt`** to add new test targets:

```cmake
# After the existing orc_test target (around line 217)

# Unit Tests
file(GLOB UNIT_TEST_FILES CONFIGURE_DEPENDS ${PROJECT_SOURCE_DIR}/test/unit/*.cpp)
list(REMOVE_ITEM UNIT_TEST_FILES ${PROJECT_SOURCE_DIR}/test/unit/main.cpp)
add_executable(orc_unit_tests
${SRC_FILES}
${HEADER_FILES}
${PROJECT_SOURCE_DIR}/test/unit/main.cpp
${UNIT_TEST_FILES}
)
target_include_directories(orc_unit_tests
PRIVATE
${PROJECT_SOURCE_DIR}/include
${PROJECT_SOURCE_DIR}/third_party/adobe-contract-checks/include
)
target_link_libraries(orc_unit_tests
PRIVATE
stlab::stlab
TBB::tbb
tomlplusplus::tomlplusplus
Tracy::TracyClient
nlohmann_json::nlohmann_json
GTest::gtest_main
)
if (PROJECT_IS_TOP_LEVEL)
target_compile_options(orc_unit_tests PRIVATE -Wall -Werror)
set_target_properties(orc_unit_tests PROPERTIES XCODE_GENERATE_SCHEME ON)
endif()

# Component Tests
file(GLOB COMPONENT_TEST_FILES CONFIGURE_DEPENDS ${PROJECT_SOURCE_DIR}/test/component/*.cpp)
list(REMOVE_ITEM COMPONENT_TEST_FILES ${PROJECT_SOURCE_DIR}/test/component/main.cpp)
add_executable(orc_component_tests
${SRC_FILES}
${HEADER_FILES}
${PROJECT_SOURCE_DIR}/test/component/main.cpp
${COMPONENT_TEST_FILES}
)
target_include_directories(orc_component_tests
PRIVATE
${PROJECT_SOURCE_DIR}/include
${PROJECT_SOURCE_DIR}/third_party/adobe-contract-checks/include
)
target_link_libraries(orc_component_tests
PRIVATE
stlab::stlab
TBB::tbb
tomlplusplus::tomlplusplus
Tracy::TracyClient
nlohmann_json::nlohmann_json
GTest::gtest_main
)
if (PROJECT_IS_TOP_LEVEL)
target_compile_options(orc_component_tests PRIVATE -Wall -Werror)
set_target_properties(orc_component_tests PROPERTIES XCODE_GENERATE_SCHEME ON)
endif()
```

#### 1.3 Update Justfile

Add recipes to the `justfile` for running the new test suites:

```just
# Run all tests (integration, unit, and component)
test: test-unit test-component test-integration

# Run unit tests
test-unit:
cmake --build build --target orc_unit_tests
./build/Debug/orc_unit_tests

# Run component tests
test-component:
cmake --build build --target orc_component_tests
./build/Debug/orc_component_tests

# Run integration tests
test-integration:
cmake --build build --target orc_test
./build/Debug/orc_test test/battery/

# Run all tests with verbose output
test-verbose:
cmake --build build --target orc_unit_tests
./build/Debug/orc_unit_tests --gtest_verbose
cmake --build build --target orc_component_tests
./build/Debug/orc_component_tests --gtest_verbose
cmake --build build --target orc_test
./build/Debug/orc_test test/battery/

# Run tests with coverage (requires additional setup)
test-coverage:
cmake -B build -GXcode -DTRACY_ENABLE=OFF -DCMAKE_BUILD_TYPE=Coverage
cmake --build build --target orc_unit_tests
cmake --build build --target orc_component_tests
./build/Debug/orc_unit_tests
./build/Debug/orc_component_tests
# Process coverage data here
```

### Phase 2: Unit Test Implementation (Weeks 3-6)

#### 2.1 String Utilities Tests (`test/unit/str_tests.cpp`)
**Module**: `src/str.cpp` / `include/orc/str.hpp`

**Test Cases**:
- String conversion utilities
- String parsing and manipulation
- Edge cases (empty strings, special characters)
- Performance benchmarks for critical paths

**Dependencies**: Minimal (standard library only)

#### 2.2 Hash Function Tests (`test/unit/hash_tests.cpp`)
**Module**: `src/hash.cpp` / `include/orc/hash.hpp`

**Test Cases**:
- Hash function correctness
- Collision resistance (basic verification)
- Consistency (same input → same output)
- Performance characteristics
- Edge cases (empty data, large data)

**Dependencies**: Minimal

#### 2.3 Memory Management Tests (`test/unit/memory_tests.cpp`)
**Module**: `include/orc/memory.hpp`

**Test Cases**:
- Memory pool allocation/deallocation
- Alignment requirements
- Thread safety (if applicable)
- Memory leak detection (using ASAN or similar)
- Performance under load

**Dependencies**: Threading library (if testing concurrency)

#### 2.4 DWARF Constants Tests (`test/unit/dwarf_constants_tests.cpp`)
**Module**: `src/dwarf_constants.cpp` / `include/orc/dwarf_constants.hpp`

**Test Cases**:
- Constant value correctness
- Enum-to-string conversions
- String-to-enum conversions
- Coverage of all DWARF tags, attributes, forms
- Invalid value handling

**Dependencies**: None

#### 2.5 Settings Tests (`test/unit/settings_tests.cpp`)
**Module**: `src/settings.cpp` / `include/orc/settings.hpp`

**Test Cases**:
- Configuration file parsing
- Default value handling
- Environment variable overrides
- Invalid configuration detection
- Settings singleton behavior
- Thread safety of settings access

**Dependencies**: TOML++, filesystem

#### 2.6 Demangle Tests (`test/unit/demangle_tests.cpp`)
**Module**: Demangle function in `src/orc.cpp` / `include/orc/orc.hpp`

**Test Cases**:
- Successful demangling of known symbols
- Handling of already-demangled symbols
- Invalid/malformed mangled names
- Thread-local storage behavior
- Performance for large symbol sets

**Dependencies**: cxxabi

#### 2.7 Fixed Vector Tests (`test/unit/fixed_vector_tests.cpp`)
**Status**: ✅ Already exists

**Action**: Move from `test/src/` to `test/unit/` and enhance:
- Add death tests for bounds violations
- Add exception safety tests
- Add move semantics validation
- Add constexpr tests (if applicable)

### Phase 3: Component Test Implementation (Weeks 7-12)

#### 3.1 Object File Registry Tests (`test/component/object_file_registry_tests.cpp`)
**Module**: `src/object_file_registry.cpp` / `include/orc/object_file_registry.hpp`

**Test Cases**:
- Object file registration
- Symbol lookup
- Duplicate handling
- Thread-safe concurrent registration
- Memory cleanup
- Integration with file parsers

**Test Data**: Small pre-built object files (store in `test/fixtures/`)

#### 3.2 DWARF Parser Tests (`test/component/dwarf_parser_tests.cpp`)
**Module**: `src/dwarf.cpp` / `include/orc/dwarf.hpp`

**Test Cases**:
- DIE (Debug Information Entry) parsing
- Attribute extraction
- Navigation of DIE trees
- Handling of abbreviation tables
- String table resolution
- Line number program processing
- Compressed sections (.zdebug_*)
- DWARF v2, v3, v4, v5 compatibility

**Test Data**: Minimal object files with known DWARF structures

#### 3.3 Mach-O Parser Tests (`test/component/macho_parser_tests.cpp`)
**Module**: `src/macho.cpp` / `include/orc/macho.hpp`

**Test Cases**:
- Header parsing (32-bit and 64-bit)
- Load command processing
- Section parsing
- Symbol table extraction
- Fat binary handling (multiple architectures)
- Segment loading
- String table access
- Error handling for corrupted files

**Test Data**: Minimal Mach-O binaries (x86_64 and arm64)

#### 3.4 COFF Parser Tests (`test/component/coff_parser_tests.cpp`)
**Module**: `src/coff.cpp` / `include/orc/coff.hpp`

**Test Cases**:
- COFF header parsing
- Section table processing
- Symbol table reading
- Relocation entries
- String table handling
- Error handling for malformed files

**Test Data**: Minimal COFF object files

#### 3.5 Archive (AR) Parser Tests (`test/component/ar_parser_tests.cpp`)
**Module**: `src/ar.cpp` / `include/orc/ar.hpp`

**Test Cases**:
- Archive header parsing
- Member extraction
- Symbol index reading
- Long filename support
- Nested object file processing
- Error handling

**Test Data**: Small .a archive files

#### 3.6 ODR Violation Detection Tests (`test/component/odrv_detection_tests.cpp`)
**Module**: Core logic in `src/orc.cpp`

**Test Cases**:
- Byte size mismatch detection
- Data member location conflicts
- Vtable element location mismatches
- Calling convention differences
- Access specifier conflicts
- False positive prevention (legitimate duplicates)
- Edge cases (forward declarations, incomplete types)

**Test Data**: Programmatically constructed DIE structures

#### 3.7 Report Generation Tests (`test/component/report_generation_tests.cpp`)
**Module**: Report generation in `src/orc.cpp` / `include/orc/orc.hpp`

**Test Cases**:
- Report formatting (text and JSON)
- Category string generation
- Symbol demangling in reports
- Location information accuracy
- Multiple conflict reporting
- Report filtering based on settings
- Output consistency

**Test Data**: Mock odrv_report objects

### Phase 4: Test Data and Fixtures (Weeks 11-12)

#### 4.1 Fixture Directory Structure
```
test/
├── fixtures/
│ ├── object_files/
│ │ ├── macho/
│ │ │ ├── simple_x86_64.o
│ │ │ ├── simple_arm64.o
│ │ │ └── fat_binary.o
│ │ ├── coff/
│ │ │ └── simple.obj
│ │ └── ar/
│ │ └── simple.a
│ ├── source/
│ │ └── simple_classes.cpp
│ └── dwarf/
│ └── sample_dies.json
└── helpers/
├── test_helpers.hpp
├── test_helpers.cpp
├── fixture_generator.cpp
└── mock_dies.cpp
```

#### 4.2 Test Helper Library
Create reusable test utilities:
- **DIE Construction**: Programmatically build DIE structures for testing
- **Object File Creation**: Generate minimal valid object files
- **Mock Data**: Factories for common test scenarios
- **Assertions**: Custom gtest matchers for ORC-specific types
- **Comparators**: Deep comparison of DWARF structures

Example: `test/helpers/test_helpers.hpp`
```cpp
namespace orc::testing {

// Create a minimal DIE with specified attributes
die* create_test_die(dw::tag tag, std::vector> attributes);

// Create a mock object file path with in-memory data
std::filesystem::path create_mock_object_file(const std::string& name, const std::vector& dies);

// Compare two DIE structures for equality
bool dies_equal(const die* a, const die* b);

// Custom gtest matcher for ODRV reports
MATCHER_P(HasCategory, category, "") {
return arg.reporting_categories().find(category) != std::string::npos;
}

} // namespace orc::testing
```

### Phase 5: CI/CD Integration (Week 13)

#### 5.1 Update GitHub Actions Workflow

**File**: `.github/workflows/build-and-test.yml`

Add steps to run all test suites:
```yaml
- name: Run Unit Tests
run: |
cmake --build build --target orc_unit_tests
./build/Debug/orc_unit_tests --gtest_output=xml:unit_test_results.xml

- name: Run Component Tests
run: |
cmake --build build --target orc_component_tests
./build/Debug/orc_component_tests --gtest_output=xml:component_test_results.xml

- name: Run Integration Tests
run: |
cmake --build build --target orc_test
./build/Debug/orc_test test/battery/ --json_mode

- name: Upload Test Results
uses: actions/upload-artifact@v3
if: always()
with:
name: test-results
path: |
unit_test_results.xml
component_test_results.xml
```

#### 5.2 Test Coverage Reporting

Add coverage generation and reporting:
```yaml
- name: Generate Coverage Report
run: |
# Install coverage tools
brew install lcov

# Re-run tests with coverage
just test-coverage

# Generate coverage report
lcov --capture --directory . --output-file coverage.info
lcov --remove coverage.info '/usr/*' '*/build/_deps/*' --output-file coverage.info

- name: Upload Coverage to Codecov
uses: codecov/codecov-action@v3
with:
files: ./coverage.info
fail_ci_if_error: true
```

### Phase 6: Documentation (Week 14)

#### 6.1 Testing Documentation

**Create**: `docs/testing.md`
- Overview of testing strategy
- How to run tests
- How to write new tests
- Test data management
- Debugging failed tests
- Coverage expectations

#### 6.2 Update README.md

Add testing section:
```markdown
## Testing

ORC has three levels of testing:

### Unit Tests
Test individual functions and classes in isolation.
```bash
just test-unit
```

### Component Tests
Test larger subsystems and their interactions.
```bash
just test-component
```

### Integration Tests
Test end-to-end ODR violation detection scenarios.
```bash
just test-integration
```

### Running All Tests
```bash
just test
```

For more details, see [docs/testing.md](docs/testing.md).

#### 6.3 Code Comment Standards

Establish documentation requirements for testable code:
- All public APIs must have doxygen comments
- Complex algorithms should include test case references
- Known limitations should be documented

## Test Coverage Goals

### Phase 1 Completion (Week 6)
- **Unit Test Coverage**: 60% of utility modules
- **Key Modules Covered**: str, hash, memory, dwarf_constants, settings

### Phase 2 Completion (Week 12)
- **Unit Test Coverage**: 80% of utility modules
- **Component Test Coverage**: 60% of core subsystems
- **Key Components Covered**: All file parsers, object registry, basic ODRV detection

### Phase 3 Completion (Week 14)
- **Unit Test Coverage**: 85%+
- **Component Test Coverage**: 75%+
- **Integration Test Coverage**: Maintained at current level
- **CI/CD**: Fully integrated with automated reporting

## Success Metrics

1. **Test Execution Time**: Total test suite should complete in < 2 minutes on CI
2. **Code Coverage**: Achieve 75%+ line coverage (excluding dependencies)
3. **Test Reliability**: < 1% flaky test rate
4. **Maintenance**: New code requires accompanying tests (enforced in code review)
5. **Documentation**: All test suites have clear documentation

## Risk Mitigation

### Risk: Tests are slow
- **Mitigation**: Use test fixtures to minimize file I/O
- **Mitigation**: Run tests in parallel where possible
- **Mitigation**: Profile slow tests and optimize

### Risk: Test data management is complex
- **Mitigation**: Create programmatic test data generators
- **Mitigation**: Use minimal fixtures where possible
- **Mitigation**: Document fixture creation process

### Risk: Breaking existing integration tests
- **Mitigation**: Run all test levels in CI
- **Mitigation**: Add unit/component tests before refactoring
- **Mitigation**: Maintain backward compatibility

### Risk: Low team adoption
- **Mitigation**: Provide clear documentation and examples
- **Mitigation**: Make test commands easy (`just test`)
- **Mitigation**: Show value through bug catches in code review

## Migration Path: From Current State to Full Test Coverage

### Week 1: Setup
1. Create directory structure: `test/unit/` and `test/component/`
2. Create `test/unit/main.cpp` and `test/component/main.cpp`
3. Update `CMakeLists.txt` with new test targets
4. Add `test` recipes to `justfile`
5. Verify builds and runs successfully (even with no tests)
6. Move `fixed_vector_tests.cpp` to `test/unit/`

**Validation**: `just test-unit` runs successfully

### Week 2: First Unit Tests
1. Implement `str_tests.cpp` with basic string utility tests
2. Implement `hash_tests.cpp` with basic hash function tests
3. Implement `dwarf_constants_tests.cpp` for enum conversions
4. Run tests locally and fix any issues

**Validation**: At least 20 passing unit tests

### Week 3-4: Core Unit Tests
1. Implement `settings_tests.cpp`
2. Implement `memory_tests.cpp`
3. Implement `demangle_tests.cpp`
4. Enhance `fixed_vector_tests.cpp` with additional coverage

**Validation**: 50+ passing unit tests, coverage > 50% for tested modules

### Week 5-6: Unit Test Polish
1. Add edge case tests
2. Add performance benchmarks where relevant
3. Fix any discovered bugs
4. Document test patterns and helpers

**Validation**: 80+ unit tests, all modules tested have >70% coverage

### Week 7-8: Test Fixtures and Helpers
1. Create `test/fixtures/` directory structure
2. Generate minimal test object files (Mach-O, COFF, AR)
3. Implement test helper library (`test/helpers/`)
4. Document fixture generation process

**Validation**: Test fixtures available for component tests

### Week 9-10: File Parser Component Tests
1. Implement `macho_parser_tests.cpp`
2. Implement `coff_parser_tests.cpp`
3. Implement `ar_parser_tests.cpp`
4. Implement `dwarf_parser_tests.cpp`

**Validation**: 50+ component tests, file parsers have >60% coverage

### Week 11-12: Core Logic Component Tests
1. Implement `object_file_registry_tests.cpp`
2. Implement `odrv_detection_tests.cpp`
3. Implement `report_generation_tests.cpp`
4. Fix any discovered bugs

**Validation**: 100+ component tests, core logic >60% coverage

### Week 13: CI/CD Integration
1. Update `.github/workflows/build-and-test.yml`
2. Add coverage reporting
3. Add test result artifacts
4. Test CI pipeline thoroughly

**Validation**: All tests run in CI, coverage reports generated

### Week 14: Documentation and Cleanup
1. Create `docs/testing.md`
2. Update `README.md`
3. Add code comment standards
4. Review and refactor tests for clarity
5. Team training/walkthrough

**Validation**: Complete documentation, team trained

## Example Test Files

### Example: `test/unit/main.cpp`
```cpp
// Google Test main for unit tests
#include

int main(int argc, char** argv) {
::testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}
```

### Example: `test/unit/str_tests.cpp` (Skeleton)
```cpp
#include "orc/str.hpp"
#include

namespace orc {

TEST(StrTest, BasicConversion) {
// Test basic string conversions
// EXPECT_EQ(...);
}

TEST(StrTest, EmptyString) {
// Test empty string handling
// EXPECT_TRUE(...);
}

TEST(StrTest, SpecialCharacters) {
// Test strings with special characters
// EXPECT_EQ(...);
}

} // namespace orc
```

### Example: `test/component/macho_parser_tests.cpp` (Skeleton)
```cpp
#include "orc/macho.hpp"
#include "test/helpers/test_helpers.hpp"
#include
#include

namespace orc {

class MachOParserTest : public ::testing::Test {
protected:
void SetUp() override {
// Set up test fixtures
test_object_path = std::filesystem::path("test/fixtures/object_files/macho/simple_x86_64.o");
}

std::filesystem::path test_object_path;
};

TEST_F(MachOParserTest, ParsesHeader) {
// Test that parser correctly reads Mach-O header
// ASSERT_TRUE(std::filesystem::exists(test_object_path));
// auto result = parse_macho_file(test_object_path);
// EXPECT_TRUE(result.has_value());
}

TEST_F(MachOParserTest, ExtractsSymbols) {
// Test symbol extraction
}

TEST_F(MachOParserTest, HandlesFatBinary) {
// Test multi-architecture binary
}

} // namespace orc
```

## Maintenance and Evolution

### Quarterly Reviews
- Review test coverage and identify gaps
- Update test fixtures as format support expands
- Evaluate test execution time and optimize if needed
- Update documentation based on team feedback

### Continuous Improvement
- Add tests for every bug fix (regression tests)
- Add tests for every new feature
- Refactor tests when production code changes
- Monitor and fix flaky tests immediately

### Team Practices
- Code review checklist includes test coverage
- New contributors must include tests with PRs
- Test failures block merge to main branch
- Regular test health dashboards

## Conclusion

This test plan provides a structured approach to implementing comprehensive unit and component testing for ORC. By following the 14-week timeline, the project will achieve:

1. **Robust test coverage** at multiple levels (unit, component, integration)
2. **Automated testing** in CI/CD pipelines
3. **Clear documentation** for test development and maintenance
4. **Improved code quality** through early bug detection
5. **Easier refactoring** with confidence in test coverage

The phased approach allows for incremental progress while maintaining development velocity on the main codebase. The justfile integration ensures tests are easy to run locally, promoting a test-driven development culture.

Start by reading the existing test setup in test/battery/, test/src/, CMakeLists.txt, and .github/workflows/build-and-test.yml. Then review the proposed test/unit/ and test/component/ structure and the associated source modules to divide the plan into bounded tasks. Done means the new targets, justfile recipes, and planned unit and component suites build and run alongside the existing integration tests.

Written by the indexing model from the issue text.