Discussion: CADGenBench generation task conflates visual perception with CAD generation

I want to raise a conceptual question about the current CADGenBench setup.

At the moment, the engineering drawings are provided as raster images. This means that the task is effectively:

**image → drawing understanding → CAD generation**

rather than directly evaluating:

**engineering drawing → CAD generation**

I wonder if this introduces a significant confounding factor. A model's performance may depend heavily on its ability to understand the image itself — detecting lines, circles, dimensions, text, views, etc. — rather than on its ability to reason about the geometry and generate the corresponding CAD model.

Would it make sense to distinguish these capabilities, for example by having separate evaluations for:

* **drawing understanding**: raster image → structured/vectorized drawing
* **CAD generation**: structured/vectorized drawing → 3D CAD
* **end-to-end image-to-CAD**: raster image → 3D CAD

I think the end-to-end task is definitely valuable, but I'm wondering whether calling it a CAD generation benchmark alone makes it difficult to interpret what the benchmark is actually measuring.

I'd be interested to hear what others think about this distinction.

The issue does not name any files, tests, or entry points. Start by reviewing the current CADGenBench task definition and benchmark inputs, then determine whether separate drawing-understanding, CAD-generation, and end-to-end evaluations are in scope; done would be an agreed evaluation design.

Written by the indexing model from the issue text.