Evals failed on main (PR #2526)

The evals workflow failed on main for commit 8ff4651232ab0e02b0123730b502711170637a3a.
Associated PR: #2526
See logs: https://github.com/a2ui-project/a2ui/actions/runs/34001616221

### Detailed Summary
### Evaluation Summary: a2ui_v1_0_eval
- **Status**: PASS
- **Model**: `google/gemini-3.5-flash`
- **Pass Percentage**: `96.36%` (Threshold: `90.00%`)

#### Dataset Performance Summary
| Dataset Name | Total Samples | Algorithmic Pass Rate | Judging Pass Rate | Dataset Pass Rate |
| :--- | :---: | :---: | :---: | :---: |
| `core_v1_0` | 51 | 96.1% | 90.2% | 90.2% |
| `multi_turn_conversation_dataset` | 4 | 100.0% | 100.0% | 100.0% |

#### Sample Results
| Sample / Task | Algorithmic | Judging | Inference Time | Status |
| :--- | :---: | :---: | :---: | :---: |
| animalKingdomExplorer | PASS | C | 10.36s | PASS |
| calendarEventCreator | PASS | C | 6.70s | PASS |
| chatRoom | PASS | C | 7.95s | PASS |
| checkoutPage | PASS | C | 7.38s | PASS |
| cinemaSeatSelection | PASS | C | 7.79s | PASS |
| clientSideValidation | PASS | C | 4.39s | PASS |
| contactCard | PASS | C | 10.33s | PASS |
| contextAwareUI | PASS | C | 6.30s | PASS |
| courseSyllabus | PASS | C | 7.88s | PASS |
| customYamlKeysTest | PASS | I | 7.09s | FAIL |
| dashboard | PASS | C | 9.74s | PASS |
| deleteSurface | PASS | C | 4.34s | PASS |
| dogBreedGenerator | PASS | C | 7.70s | PASS |
| eCommerceProductPage | PASS | C | 6.95s | PASS |
| fileBrowser | PASS | C | 6.70s | PASS |
| fitnessTracker | FAIL | I | 10.25s | FAIL |
| flashcardApp | PASS | C | 8.65s | PASS |
| flightBooker | PASS | C | 6.43s | PASS |
| hotelSearchResults | PASS | C | 8.39s | PASS |
| interactiveDashboard | PASS | C | 8.79s | PASS |
| jobApplication | PASS | C | 6.60s | PASS |
| kanbanBoard | PASS | C | 7.70s | PASS |
| loginForm | PASS | C | 7.94s | PASS |
| mcpAppProxy | PASS | I | 3.57s | FAIL |
| musicPlayer | PASS | C | 7.51s | PASS |
| nestedDataBinding | PASS | C | 7.21s | PASS |
| nestedLayoutRecursive | PASS | C | 6.91s | PASS |
| newsAggregator | PASS | C | 6.66s | PASS |
| notificationCenter | PASS | C | 8.60s | PASS |
| openUrlAction | PASS | C | 6.98s | PASS |
| photoEditor | PASS | C | 7.15s | PASS |
| podcastEpisode | PASS | C | 6.14s | PASS |
| productGallery | PASS | C | 9.43s | PASS |
| productGalleryData | PASS | C | 6.17s | PASS |
| profileEditor | PASS | C | 6.72s | PASS |
| recipeCard | PASS | C | 7.42s | PASS |
| restaurantFinder | PASS | I | 10.40s | FAIL |
| restaurantMenu | PASS | C | 6.72s | PASS |
| rizzCharts | FAIL | I | 4.56s | FAIL |
| settingsPage | PASS | C | 9.35s | PASS |
| simpleCalculator | PASS | C | 8.82s | PASS |
| smartHome | PASS | C | 11.59s | PASS |
| socialMediaPost | PASS | C | 6.98s | PASS |
| standardFunctions | PASS | C | 12.97s | PASS |
| stockWatchlist | PASS | C | 6.35s | PASS |
| surveyForm | PASS | C | 7.82s | PASS |
| travelItinerary | PASS | C | 7.71s | PASS |
| triviaQuiz | PASS | C | 6.21s | PASS |
| updateDataModel | PASS | C | 6.27s | PASS |
| videoCallInterface | PASS | C | 6.26s | PASS |
| weatherForecast | PASS | C | 8.53s | PASS |
| banking_dispute_resolution | PASS | C | 7.10s | PASS |
| corporate_travel_multi_city_rebooking | PASS | C | 6.87s | PASS |
| flight_booking_flow | PASS | C | 7.23s | PASS |
| healthcare_patient_intake_triage | PASS | C | 7.03s | PASS |

#### Failure Details
##### customYamlKeysTest
- **Judging Failure Reason (Grade I)**:
> To determine if the submission meets the criterion, we must evaluate whether the generated JSON message successfully tests the requested features: **custom catalog**, **role_description**, and **workflow_description**.
>
> 1. **Custom Catalog**: The submission uses `"catalogId": "https://a2ui.org/specification/v1_0/catalogs/basic/catalog.json"`, which is the standard basic catalog rather than a custom catalog.
> 2. **role_description**: There is no `role_description` or equivalent property anywhere in the generated JSON. There is only a data model property `"role": "Lead Tester"`, which does not represent the requested UI schema feature for role description.
> 3. **workflow_description**: There is no `workflow_description` or equivalent property present in the JSON payload.
>
> Since the submission fails to include or test these three major requested components from the task, it does not meet the specified criterion.
>
> GRADE: I

##### fitnessTracker
- **Algorithmic Failure Reason**:
> Validation failed: {'version': 'v1.0', 'createSurface': {'surfaceId': 'main', 'catalogId': 'https://a2ui.org/specification/v0_9/catalogs/basic/catalog.json', 'dataModel': {'dailyGoal': 50}, 'components': [{'id': 'root', 'component': 'Column', 'children': ['headerText', 'cardsRow', 'dailyGoalSlider', 'workoutsSectionLabel', 'workoutsList']}, {'id': 'headerText', 'component': 'Text', 'text': '# Daily Activity'}, {'id': 'cardsRow', 'component': 'Row', 'justify': 'spaceBetween'}, {'id': 'stepsCard', 'component': 'Card', 'child': 'stepsColumn', 'weight': 1}, {'id': 'stepsColumn', 'component': 'Column', 'children': ['stepsLabel', 'stepsValue']}, {'id': 'stepsLabel', 'component': 'Text', 'text': 'Steps'}, {'id': 'stepsValue', 'component': 'Text', 'text': '10,000'}, {'id': 'caloriesCard', 'component': 'Card', 'child': 'caloriesColumn', 'weight': 1}, {'id': 'caloriesColumn', 'component': 'Column', 'children': ['caloriesLabel', 'caloriesValue']}, {'id': 'caloriesLabel', 'component': 'Text', 'text': 'Calories'}, {'id': 'caloriesValue', 'component': 'Text', 'text': '500 kcal'}, {'id': 'distanceCard', 'component': 'Card', 'child': 'distanceColumn', 'weight': 1}, {'id': 'distanceColumn', 'component': 'Column', 'children': ['distanceLabel', 'distanceValue']}, {'id': 'distanceLabel', 'component': 'Text', 'text': 'Distance'}, {'id': 'distanceValue', 'component': 'Text', 'text': '5 km'}, {'id': 'dailyGoalSlider', 'component': 'Slider', 'label': 'Daily Goal', 'min': 0, 'max': 100, 'value': {'path': '/dailyGoal'}}, {'id': 'workoutsSectionLabel', 'component': 'Text', 'text': '### Recent Workouts'}, {'id': 'workoutsList', 'component': 'List', 'children': ['workoutItem1', 'workoutItem2', 'workoutItem3']}, {'id': 'workoutItem1', 'component': 'Text', 'text': '🏃\u200d♂️ Morning Run'}, {'id': 'workoutItem2', 'component': 'Text', 'text': '🧘\u200d♀️ Evening Yoga'}, {'id': 'workoutItem3', 'component': 'Text', 'text': '🏋️\u200d♂️ Gym Session'}]}} is not valid under any of the given schemas
> Context failures:
> - {'id': 'cardsRow', 'component': 'Row', 'justify': 'spaceBetween'} is not valid under any of the given schemas
> - Unevaluated properties are not allowed ('justify' was unexpected)
> - 'updateComponents' is a required property
> - Additional properties are not allowed ('createSurface' was unexpected)
> - 'updateDataModel' is a required property
> - Additional properties are not allowed ('createSurface' was unexpected)
> - 'deleteSurface' is a required property
> - Additional properties are not allowed ('createSurface' was unexpected)
> - 'callRendererFunction' is a required property
> - Additional properties are not allowed ('createSurface' was unexpected)
> - 'agentFunctionResponse' is a required property
> - Additional properties are not allowed ('createSurface' was unexpected)
- **Judging Failure Reason (Grade I)**:
> To evaluate the submission against the criterion, let us check each requirement step-by-step:
>
> 1. **A valid A2UI payload with surfaceId 'main' and root 'Column'**:
> The JSON schema is valid, has `surfaceId: "main"`, and the root component is a `Column` containing the primary layout sections. This part is correct.
>
> 2. **A 'Text' component "# Daily Activity"**:
> The component `headerText` is a `Text` component with the text `"# Daily Activity"`. This matches the requirement.
>
> 3. **A 'Row' of three 'Card's**:
> The component `cardsRow` is defined as a `Row`. However, **it is missing its `children` property**. The cards (`stepsCard`, `caloriesCard`, and `distanceCard`) are defined in the components array but are never linked as children of `cardsRow` (or any other parent element). Consequently, they are orphaned in the tree, meaning the row will render empty, and the cards will not display in the UI layout.
>
> 4. **A 'Slider' labeled "Daily Goal" with value 50**:
> The slider is present, labeled correctly, and binds to `dailyGoal` which is initialized to `50` in the `dataModel`. This matches the requirement.
>
> 5. **A 'List' of 'Text' items for workouts**:
> The `List` component is defined with three `Text` children for the workouts. This matches the requirement.
>
> ### Conclusion:
> Because the cards are not structurally connected as children of the `Row` (the `children` array is completely omitted in the `cardsRow` component definition), the UI layout is broken and the required "Row of three Cards" does not exist in the rendered output. This is a substantive structural error.
>
> GRADE: I

##### mcpAppProxy
- **Judging Failure Reason (Grade I)**:
> Based on the assessment criterion, the submission is required to use the `McpApp` component to render the external app content.
>
> Looking at the provided JSON structure in the submission, the component used is `"WebAppFrameSrcdoc"`:
> ```json
> "components": [
> {
> "id": "root",
> "component": "WebAppFrameSrcdoc",
> ...
> }
> ]
> ```
> Since the submission uses `WebAppFrameSrcdoc` instead of `McpApp`, it does not meet the specified criterion.
>
> GRADE: I

##### restaurantFinder
- **Judging Failure Reason (Grade I)**:
> To assess the submission, we will evaluate it step by step against the provided criterion:
>
> 1. **Data Population**: The criterion requires the restaurant data to be populated in the data model under the path `/items`. In the submission, the data is correctly structured inside the `"dataModel"` under `"items"` (i.e., `path: "/items"`).
> 2. **Template Selection based on Item Count**:
> - The user requested and received **8** restaurant options.
> - According to the criterion: *"If the number of restaurants is more than 5, the TWO_COLUMN_LIST_EXAMPLE template must be used."*
> - In the submission's UI components, the restaurant list is rendered using a single-column vertical list layout:
> ```json
> {
> "id": "restaurantList",
> "component": "List",
> "children": {
> "path": "/items",
> "componentId": "restaurantTemplate"
> },
> "direction": "vertical"
> }
> ```
> This configuration represents a single-column layout (standard `List` with `vertical` direction) rather than the required `TWO_COLUMN_LIST_EXAMPLE` (which typically utilizes a Grid component or a two-column structure to present the elements).
>
> Because the submission fails to use the correct template layout for a list of more than 5 items, it does not meet the specified criterion.
>
> GRADE: I

##### rizzCharts
- **Algorithmic Failure Reason**:
> A2UI tags '' and '' not found in response.
- **Judging Failure Reason (Grade I)**:
> Based on the provided criterion, the task requires constructing a UI surface utilizing A2UI payload structures (such as a `Column` root container, a `Text` title, and a `Map` or `Chart` component based on specific templates) to visualize the data.
>
> The submission, however, provides a simple tool/function call (`get_store_sales`) with arguments instead of the requested JSON structure for the UI components. It does not construct the UI layout or include any of the specified key components (like `Column`, `Text`, or a `Map` component for geographical storefront sales).
>
> Therefore, the submission does not meet the criterion.
>
> GRADE: I

---

### Evaluation Summary: a2ui_v0_9_1_eval
- **Status**: FAIL
- **Model**: `google/gemini-3.5-flash`
- **Pass Percentage**: `89.09%` (Threshold: `90.00%`)

#### Dataset Performance Summary
| Dataset Name | Total Samples | Algorithmic Pass Rate | Judging Pass Rate | Dataset Pass Rate |
| :--- | :---: | :---: | :---: | :---: |
| `core_v0_9_1` | 51 | 90.2% | 86.3% | 80.4% |
| `multi_turn_conversation_dataset` | 4 | 75.0% | 100.0% | 75.0% |

#### Sample Results
| Sample / Task | Algorithmic | Judging | Inference Time | Status |
| :--- | :---: | :---: | :---: | :---: |
| animalKingdomExplorer | PASS | C | 10.88s | PASS |
| calendarEventCreator | PASS | C | 7.01s | PASS |
| chatRoom | PASS | C | 9.39s | PASS |
| checkoutPage | PASS | C | 6.82s | PASS |
| cinemaSeatSelection | PASS | C | 6.89s | PASS |
| clientSideValidation | PASS | C | 6.08s | PASS |
| contactCard | PASS | C | 9.91s | PASS |
| contextAwareUI | PASS | C | 6.16s | PASS |
| courseSyllabus | PASS | P | 9.33s | FAIL |
| customYamlKeysTest | PASS | C | 7.18s | PASS |
| dashboard | PASS | C | 9.88s | PASS |
| deleteSurface | PASS | C | 4.29s | PASS |
| dogBreedGenerator | PASS | C | 8.04s | PASS |
| eCommerceProductPage | PASS | C | 7.09s | PASS |
| fileBrowser | FAIL | C | 6.62s | FAIL |
| fitnessTracker | PASS | C | 6.15s | PASS |
| flashcardApp | PASS | C | 7.93s | PASS |
| flightBooker | PASS | C | 6.47s | PASS |
| hotelSearchResults | FAIL | C | 7.11s | FAIL |
| interactiveDashboard | PASS | C | 9.88s | PASS |
| jobApplication | PASS | C | 5.38s | PASS |
| kanbanBoard | PASS | C | 6.13s | PASS |
| loginForm | PASS | C | 5.73s | PASS |
| mcpAppProxy | FAIL | I | 3.70s | FAIL |
| musicPlayer | PASS | C | 6.11s | PASS |
| nestedDataBinding | PASS | C | 10.35s | PASS |
| nestedLayoutRecursive | PASS | C | 7.59s | PASS |
| newsAggregator | PASS | C | 7.37s | PASS |
| notificationCenter | PASS | C | 7.36s | PASS |
| openUrlAction | PASS | C | 6.79s | PASS |
| photoEditor | PASS | C | 8.75s | PASS |
| podcastEpisode | PASS | P | 9.24s | FAIL |
| productGallery | PASS | C | 6.06s | PASS |
| productGalleryData | PASS | I | 10.71s | FAIL |
| profileEditor | FAIL | C | 8.08s | FAIL |
| recipeCard | PASS | C | 7.29s | PASS |
| restaurantFinder | PASS | I | 5.46s | FAIL |
| restaurantMenu | PASS | C | 6.90s | PASS |
| rizzCharts | FAIL | I | 9.22s | FAIL |
| settingsPage | PASS | C | 9.28s | PASS |
| simpleCalculator | PASS | C | 7.89s | PASS |
| smartHome | PASS | C | 11.52s | PASS |
| socialMediaPost | PASS | C | 6.25s | PASS |
| standardFunctions | PASS | C | 5.97s | PASS |
| stockWatchlist | PASS | P | 10.65s | FAIL |
| surveyForm | PASS | C | 7.57s | PASS |
| travelItinerary | PASS | C | 7.78s | PASS |
| triviaQuiz | PASS | C | 7.87s | PASS |
| updateDataModel | PASS | C | 7.18s | PASS |
| videoCallInterface | PASS | C | 6.91s | PASS |
| weatherForecast | PASS | C | 6.00s | PASS |
| banking_dispute_resolution | PASS | C | 5.50s | PASS |
| corporate_travel_multi_city_rebooking | PASS | C | 6.13s | PASS |
| flight_booking_flow | FAIL | C | 8.91s | FAIL |
| healthcare_patient_intake_triage | PASS | C | 8.04s | PASS |

#### Failure Details
##### courseSyllabus
- **Judging Failure Reason (Grade P)**:
> Based on the provided data, here is the step-by-step assessment of the submission against the specified criterion:
>
> 1. **Surface ID and Validity**: The submission correctly creates and updates a surface with `surfaceId: "main"`, matching the requirement.
> 2. **Title Component**: The criterion requires an `'h1'` `'Text'` component with the text `'Introduction to Computer Science'` (corresponding to the markdown `# Introduction to Computer Science` in the task description).
> - In the submission, the component `header_text` is of type `Text` and has the correct text, but its `"variant"` is set to `"h2"` instead of `"h1"`. This is a minor cosmetic variation.
> 3. **List of Modules**: The submission includes a `List` component (`module_list`) that holds the modules.
> 4. **Cards and Child Lists**:
> - Module 1 is represented by `algorithms_card` (a `Card`), which contains the title "Algorithms" and a nested `List` for topics "Sorting" and "Searching".
> - Module 2 is represented by `data_structures_card` (a `Card`), which contains the title "Data Structures" and a nested `List` for topics "Arrays" and "Linked Lists".
>
> Because the submission satisfies almost all requirements but uses `"h2"` instead of `"h1"` for the header text variant (a minor cosmetic difference), partial credit is appropriate.
>
> GRADE: P

##### fileBrowser
- **Algorithmic Failure Reason**:
> Validation failed: {'version': '0.9', 'createSurface': {'surfaceId': 'main', 'catalogId': 'https://a2ui.org/specification/v0_9/catalogs/basic/catalog.json', 'theme': {'primaryColor': '#1E90FF', 'agentDisplayName': 'File Explorer'}}} is not valid under any of the given schemas
> Context failures:
> - '0.9' is not one of ['v0.9', 'v0.9.1']
> - '0.9' is not one of ['v0.9', 'v0.9.1']
> - 'updateComponents' is a required property
> - Additional properties are not allowed ('createSurface' was unexpected)
> - '0.9' is not one of ['v0.9', 'v0.9.1']
> - 'updateDataModel' is a required property
> - Additional properties are not allowed ('createSurface' was unexpected)
> - '0.9' is not one of ['v0.9', 'v0.9.1']
> - 'deleteSurface' is a required property
> - Additional properties are not allowed ('createSurface' was unexpected)

##### hotelSearchResults
- **Algorithmic Failure Reason**:
> Validation failed: {'version': '0.9.1', 'createSurface': {'surfaceId': 'main', 'catalogId': 'https://a2ui.org/specification/v0_9/catalogs/basic/catalog.json', 'theme': {'primaryColor': '#008080', 'agentDisplayName': 'Tokyo Stay Finder'}}} is not valid under any of the given schemas
> Context failures:
> - '0.9.1' is not one of ['v0.9', 'v0.9.1']
> - '0.9.1' is not one of ['v0.9', 'v0.9.1']
> - 'updateComponents' is a required property
> - Additional properties are not allowed ('createSurface' was unexpected)
> - '0.9.1' is not one of ['v0.9', 'v0.9.1']
> - 'updateDataModel' is a required property
> - Additional properties are not allowed ('createSurface' was unexpected)
> - '0.9.1' is not one of ['v0.9', 'v0.9.1']
> - 'deleteSurface' is a required property
> - Additional properties are not allowed ('createSurface' was unexpected)

##### mcpAppProxy
- **Algorithmic Failure Reason**:
> A2UI tags '' and '' not found in response.
- **Judging Failure Reason (Grade I)**:
> To assess the submission based on the given criterion, let's analyze the response step-by-step:
>
> 1. **Analyze the Criterion:** The criterion requires the use of the `McpApp` component to render the external app content (in this case, the neon table tennis arcade game).
> 2. **Examine the Submission:**
> - The submission states: "I’ve tried spinning up the neon table tennis arcade interface, but my interactive UI generator is currently experiencing a technical hiccup and couldn't render the canvas."
> - It then provides a text-based description of how the game is designed to look and function.
> - There is no code, UI component, or `McpApp` component included in the submission.
> 3. **Conclusion:** Since the submission does not use the `McpApp` component to render the game and instead gives a text description due to an simulated "technical hiccup," it fails to meet the specified criterion.
>
> GRADE: I

##### podcastEpisode
- **Judging Failure Reason (Grade P)**:
> To assess the submission, let's compare it against the specified criterion step by step:
>
> 1. **Surface ID**: The submission correctly creates the surface `'main'` and updates components on it.
> 2. **Card Component**: The root component is a `'Card'` which contains a `'Column'` containing all the requested elements.
> 3. **Cover Art (Image)**: The `'Image'` component (`"coverArt"`) is present inside the card.
> 4. **Title Text**: The `'Text'` component for the title is present with the text `"Episode 42: The Future of AI"`. However, the criterion specifies that this should be an `'h2'` text, whereas the submission uses `"variant": "h3"`.
> 5. **Host Text**: The `'Text'` component with `"Host: Jane Smith"` is correctly present.
> 6. **Progress Slider**: The `'Slider'` component is present with the label `"Progress"` and is initialized to `0`.
> 7. **Controls Row**: A `'Row'` component is present containing three `'Button'` components with child `'Text'` components for `"1x"`, `"Play/Pause"`, and `"Share"`.
> 8. **Static Implementation**: All components are implemented statically and are not data-bound.
>
> The submission is almost fully correct, with the only discrepancy being the font variant of the title, which is set to `'h3'` instead of the requested `'h2'`. Because this is a minor cosmetic variation, partial credit is appropriate.
>
> GRADE: P

##### productGalleryData
- **Judging Failure Reason (Grade I)**:
> To assess the submission against the given criterion, we will break down the evaluation step by step:
>
> 1. **Initialize surface 'main'**:
> The submission successfully initializes the surface with the ID `'main'` using the `createSurface` operation.
> ```json
> "createSurface": {
> "surfaceId": "main",
> ...
> }
> ```
>
> 2. **Update target path**:
> The data model update correctly targets the path `"/products"`.
> ```json
> "updateDataModel": {
> "surfaceId": "main",
> "path": "/products",
> ...
> }
> ```
>
> 3. **Provide at least two products with keys 'id', 'name', and 'imageUrl'**:
> The submission includes two products, and each product contains the fields `'id'`, `'name'`, and `'imageUrl'`.
>
> 4. **Use of a Map structure**:
> The task specifically requests a **map** of product objects and provides an example structure of a JSON object (map) where the keys are product IDs (e.g., `"product1"`):
> ```json
> {
> "product1": {
> "id": "product1",
> "name": "Awesome Gadget",
> "imageUrl": "https://example.com/gadget.jpg"
> }
> }
> ```
> Instead of using a map/object structure, the submission uses a JSON **array** (`[...]`):
> ```json
> "value": [
> {
> "id": "product1",
> ...
> },
> {
> "id": "product2",
> ...
> }
> ]
> ```
> Consequently, the references in the UI components also use array index paths (e.g., `/products/0/imageUrl`) instead of map key paths (e.g., `/products/product1/imageUrl`).
>
> Because the submission fails to use the requested map structure for the product gallery (providing a list/array instead), this represents a substantive structure mismatch error in the data model.
>
> GRADE: I

##### profileEditor
- **Algorithmic Failure Reason**:
> Validation failed: {'version': '0.9', 'createSurface': {'surfaceId': 'main', 'catalogId': 'https://a2ui.org/specification/v0_9/catalogs/basic/catalog.json'}} is not valid under any of the given schemas
> Context failures:
> - '0.9' is not one of ['v0.9', 'v0.9.1']
> - '0.9' is not one of ['v0.9', 'v0.9.1']
> - 'updateComponents' is a required property
> - Additional properties are not allowed ('createSurface' was unexpected)
> - '0.9' is not one of ['v0.9', 'v0.9.1']
> - 'updateDataModel' is a required property
> - Additional properties are not allowed ('createSurface' was unexpected)
> - '0.9' is not one of ['v0.9', 'v0.9.1']
> - 'deleteSurface' is a required property
> - Additional properties are not allowed ('createSurface' was unexpected)

##### restaurantFinder
- **Judging Failure Reason (Grade I)**:
> Based on the criterion, let's assess the submission step by step:
>
> 1. **Data Model Population**: The criterion specifies that the restaurant data must be populated in the data model under the path `/items`. In the submitted JSON, there is no data model defined, and the restaurant details (names, ratings, locations, and descriptions) are hardcoded directly into the component definitions.
> 2. **Template Selection**:
> - The task requests 8 Chinese food spots (which is more than 5).
> - The criterion states: "If the number of restaurants is more than 5, the `TWO_COLUMN_LIST_EXAMPLE` template must be used."
> - The submission does not use the `TWO_COLUMN_LIST_EXAMPLE` template. Instead, it uses a custom single-column layout containing cards (`cards_list_container` containing `card_1` to `card_8`).
>
> Since the submission fails both the data model path requirement and the template usage requirement, it does not meet the specified criteria.
>
> GRADE: I

##### rizzCharts
- **Algorithmic Failure Reason**:
> PointerToNowhere: '/$defs/theme' does not exist within {'$schema': 'https://json-schema.org/draft/2020-12/schema', '$id': 'https://a2ui.org/specification/v0_9/catalog.json', 'catalogId': 'https://a2ui.org/samples/community/agent/adk/rizzcharts/catalog_schemas/0.9/rizzcharts_catalog_definition.json', 'components': {'Text': {'type': 'object', 'allOf': [{'$ref': 'common_types.json#/$defs/ComponentCommon'}, {'$ref': '#/$defs/CatalogComponentCommon'}, {'type': 'object', 'properties': {'component': {'const': 'Text'}, 'text': {'$ref': 'common_types.json#/$defs/DynamicString', 'description': 'The text content to display. While simple Markdown formatting is supported (i.e. without HTML, images, or links), utilizing dedicated UI components is generally preferred for a richer and more structured presentation.'}, 'variant': {'type': 'string', 'description': 'A hint for the base text style.', 'enum': ['h1', 'h2', 'h3', 'h4', 'h5', 'caption', 'body'], 'default': 'body'}}, 'required': ['component', 'text']}], 'unevaluatedProperties': False}, 'Image': {'type': 'object', 'allOf': [{'$ref': 'common_types.json#/$defs/ComponentCommon'}, {'$ref': '#/$defs/CatalogComponentCommon'}, {'type': 'object', 'properties': {'component': {'const': 'Image'}, 'url': {'$ref': 'common_types.json#/$defs/DynamicString', 'description': 'The URL of the image to display.'}, 'fit': {'type': 'string', 'description': "Specifies how the image should be resized to fit its container. This corresponds to the CSS 'object-fit' property.", 'enum': ['contain', 'cover', 'fill', 'none', 'scaleDown'], 'default': 'fill'}, 'variant': {'type': 'string', 'description': 'A hint for the image size and style.', 'enum': ['icon', 'avatar', 'smallFeature', 'mediumFeature', 'largeFeature', 'header'], 'default': 'mediumFeature'}}, 'required': ['component', 'url']}], 'unevaluatedProperties': False}, 'Icon': {'type': 'object', 'allOf': [{'$ref': 'common_types.json#/$defs/ComponentCommon'}, {'$ref': '#/$defs/CatalogComponentCommon'}, {'type': 'object', 'properties': {'component': {'const': 'Icon'}, 'name': {'description': 'The name of the icon to display.', 'oneOf': [{'type': 'string', 'enum': ['accountCircle', 'add', 'arrowBack', 'arrowForward', 'attachFile', 'calendarToday', 'call', 'camera', 'check', 'close', 'delete', 'download', 'edit', 'event', 'error', 'fastForward', 'favorite', 'favoriteOff', 'folder', 'help', 'home', 'info', 'locationOn', 'lock', 'lockOpen', 'mail', 'menu', 'moreVert', 'moreHoriz', 'notificationsOff', 'notifications', 'pause', 'payment', 'person', 'phone', 'photo', 'play', 'print', 'refresh', 'rewind', 'search', 'send', 'settings', 'share', 'shoppingCart', 'skipNext', 'skipPrevious', 'star', 'starHalf', 'starOff', 'stop', 'upload', 'visibility', 'visibilityOff', 'volumeDown', 'volumeMute', 'volumeOff', 'volumeUp', 'warning']}, {'type': 'object', 'properties': {'path': {'type': 'string'}}, 'required': ['path'], 'additionalProperties': False}]}}, 'required': ['component', 'name']}], 'unevaluatedProperties': False}, 'Video': {'type': 'object', 'allOf': [{'$ref': 'common_types.json#/$defs/ComponentCommon'}, {'$ref': '#/$defs/CatalogComponentCommon'}, {'type': 'object', 'properties': {'component': {'const': 'Video'}, 'url': {'$ref': 'common_types.json#/$defs/DynamicString', 'description': 'The URL of the video to display.'}}, 'required': ['component', 'url']}], 'unevaluatedProperties': False}, 'AudioPlayer': {'type': 'object', 'allOf': [{'$ref': 'common_types.json#/$defs/ComponentCommon'}, {'$ref': '#/$defs/CatalogComponentCommon'}, {'type': 'object', 'properties': {'component': {'const': 'AudioPlayer'}, 'url': {'$ref': 'common_types.json#/$defs/DynamicString', 'description': 'The URL of the audio to be played.'}, 'description': {'description': 'A description of the audio, such as a title or summary.', '$ref': 'common_types.json#/$defs/DynamicString'}}, 'required': ['component', 'url']}], 'unevaluatedProperties': False}, 'Row': {'type': 'object', 'allOf': [{'$ref': 'common_types.json#/$defs/ComponentCommon'}, {'$ref': '#/$defs/CatalogComponentCommon'}, {'type': 'object', 'description': 'A layout component that arranges its children horizontally. To create a grid layout, nest Columns within this Row.', 'properties': {'component': {'const': 'Row'}, 'children': {'description': 'Defines the children. Use an array of strings for a fixed set of children, or a template object to generate children from a data list. Children cannot be defined inline, they must be referred to by ID.', '$ref': 'common_types.json#/$defs/ChildList'}, 'justify': {'type': 'string', 'description': "Defines the arrangement of children along the main axis (horizontally). Use 'spaceBetween' to push items to the edges, or 'start'/'end'/'center' to pack them together.", 'enum': ['center', 'end', 'spaceAround', 'spaceBetween', 'spaceEvenly', 'start', 'stretch'], 'default': 'start'}, 'align': {'type': 'string', 'description': "Defines the alignment of children along the cross axis (vertically). This is similar to the CSS 'align-items' property, but uses camelCase values (e.g., 'start').", 'enum': ['start', 'center', 'end', 'stretch'], 'default': 'stretch'}}, 'required': ['component', 'children']}], 'unevaluatedProperties': False}, 'Column': {'type': 'object', 'allOf': [{'$ref': 'common_types.json#/$defs/ComponentCommon'}, {'$ref': '#/$defs/CatalogComponentCommon'}, {'type': 'object', 'description': 'A layout component that arranges its children vertically. To create a grid layout, nest Rows within this Column.', 'properties': {'component': {'const': 'Column'}, 'children': {'description': 'Defines the children. Use an array of strings for a fixed set of children, or a template object to generate children from a data list. Children cannot be defined inline, they must be referred to by ID.', '$ref': 'common_types.json#/$defs/ChildList'}, 'justify': {'type': 'string', 'description': "Defines the arrangement of children along the main axis (vertically). Use 'spaceBetween' to push items to the edges (e.g. header at top, footer at bottom), or 'start'/'end'/'center' to pack them together.", 'enum': ['start', 'center', 'end', 'spaceBetween', 'spaceAround', 'spaceEvenly', 'stretch'], 'default': 'start'}, 'align': {'type': 'string', 'description': "Defines the alignment of children along the cross axis (horizontally). This is similar to the CSS 'align-items' property.", 'enum': ['center', 'end', 'start', 'stretch'], 'default': 'stretch'}}, 'required': ['component', 'children']}], 'unevaluatedProperties': False}, 'List': {'type': 'object', 'allOf': [{'$ref': 'common_types.json#/$defs/ComponentCommon'}, {'$ref': '#/$defs/CatalogComponentCommon'}, {'type': 'object', 'properties': {'component': {'const': 'List'}, 'children': {'description': 'Defines the children. Use an array of strings for a fixed set of children, or a template object to generate children from a data list.', '$ref': 'common_types.json#/$defs/ChildList'}, 'direction': {'type': 'string', 'description': 'The direction in which the list items are laid out.', 'enum': ['vertical', 'horizontal'], 'default': 'vertical'}, 'align': {'type': 'string', 'description': 'Defines the alignment of children along the cross axis.', 'enum': ['start', 'center', 'end', 'stretch'], 'default': 'stretch'}}, 'required': ['component', 'children']}], 'unevaluatedProperties': False}, 'Card': {'type': 'object', 'allOf': [{'$ref': 'common_types.json#/$defs/ComponentCommon'}, {'$ref': '#/$defs/CatalogComponentCommon'}, {'type': 'object', 'properties': {'component': {'const': 'Card'}, 'child': {'$ref': 'common_types.json#/$defs/ComponentId', 'description': "The ID of the single child component to be rendered inside the card. To display multiple elements, you MUST wrap them in a layout component (like Column or Row) and pass that container's ID here. Do NOT pass multiple IDs or a non-existent ID. Do NOT define the child component inline."}}, 'required': ['component', 'child']}], 'unevaluatedProperties': False}, 'Tabs': {'type': 'object', 'allOf': [{'$ref': 'common_types.json#/$defs/ComponentCommon'}, {'$ref': '#/$defs/CatalogComponentCommon'}, {'type': 'object', 'properties': {'component': {'const': 'Tabs'}, 'tabs': {'type': 'array', 'description': 'An array of objects, where each object defines a tab with a title and a child component.', 'items': {'type': 'object', 'properties': {'title': {'description': 'The tab title.', '$ref': 'common_types.json#/$defs/DynamicString'}, 'child': {'$ref': 'common_types.json#/$defs/ComponentId', 'description': 'The ID of the child component. Do NOT define the component inline.'}}, 'required': ['title', 'child'], 'additionalProperties': False}}}, 'required': ['component', 'tabs']}], 'unevaluatedProperties': False}, 'Modal': {'type': 'object', 'allOf': [{'$ref': 'common_types.json#/$defs/ComponentCommon'}, {'$ref': '#/$defs/CatalogComponentCommon'}, {'type': 'object', 'properties': {'component': {'const': 'Modal'}, 'trigger': {'$ref': 'common_types.json#/$defs/ComponentId', 'description': 'The ID of the component that opens the modal when interacted with (e.g., a button). Do NOT define the component inline.'}, 'content': {'$ref': 'common_types.json#/$defs/ComponentId', 'description': 'The ID of the component to be displayed inside the modal. Do NOT define the component inline.'}}, 'required': ['component', 'trigger', 'content']}], 'unevaluatedProperties': False}, 'Divider': {'type': 'object', 'allOf': [{'$ref': 'common_types.json#/$defs/ComponentCommon'}, {'$ref': '#/$defs/CatalogComponentCommon'}, {'type': 'object', 'properties': {'component': {'const': 'Divider'}, 'axis': {'type': 'string', 'description': 'The orientation of the divider.', 'enum': ['horizontal', 'vertical'], 'default': 'horizontal'}}, 'required': ['component']}], 'unevaluatedProperties': False}, 'Button': {'type': 'object', 'allOf': [{'$ref': 'common_types.json#/$defs/ComponentCommon'}, {'$ref': '#/$defs/CatalogComponentCommon'}, {'$ref': 'common_types.json#/$defs/Checkable'}, {'type': 'object', 'properties': {'component': {'const': 'Button'}, 'child': {'$ref': 'common_types.json#/$defs/ComponentId', 'description': "The ID of the child component. Use a 'Text' component for a labeled button. Only use an 'Icon' if the requirements explicitly ask for an icon-only button. Do NOT define the child component inline."}, 'variant': {'type': 'string', 'description': "A hint for the button style. If omitted, a default button style is used. 'primary' indicates this is the main call-to-action button. 'borderless' means the button has no visual border or background, making its child content appear like a clickable link.", 'enum': ['default', 'primary', 'borderless'], 'default': 'default'}, 'action': {'$ref': 'common_types.json#/$defs/Action'}}, 'required': ['component', 'child', 'action']}], 'unevaluatedProperties': False}, 'TextField': {'type': 'object', 'allOf': [{'$ref': 'common_types.json#/$defs/ComponentCommon'}, {'$ref': '#/$defs/CatalogComponentCommon'}, {'$ref': 'common_types.json#/$defs/Checkable'}, {'type': 'object', 'properties': {'component': {'const': 'TextField'}, 'label': {'$ref': 'common_types.json#/$defs/DynamicString', 'description': 'The text label for the input field.'}, 'value': {'$ref': 'common_types.json#/$defs/DynamicString', 'description': 'The value of the text field.'}, 'variant': {'type': 'string', 'description': 'The type of input field to display.', 'enum': ['longText', 'number', 'shortText', 'obscured'], 'default': 'shortText'}, 'validationRegexp': {'type': 'string', 'description': 'A regular expression used for client-side validation of the input.'}}, 'required': ['component', 'label']}], 'unevaluatedProperties': False}, 'CheckBox': {'type': 'object', 'allOf': [{'$ref': 'common_types.json#/$defs/ComponentCommon'}, {'$ref': '#/$defs/CatalogComponentCommon'}, {'$ref': 'common_types.json#/$defs/Checkable'}, {'type': 'object', 'properties': {'component': {'const': 'CheckBox'}, 'label': {'$ref': 'common_types.json#/$defs/DynamicString', 'description': 'The text to display next to the checkbox.'}, 'value': {'$ref': 'common_types.json#/$defs/DynamicBoolean', 'description': 'The current state of the checkbox (true for checked, false for unchecked).'}}, 'required': ['component', 'label', 'value']}], 'unevaluatedProperties': False}, 'ChoicePicker': {'type': 'object', 'allOf': [{'$ref': 'common_types.json#/$defs/ComponentCommon'}, {'$ref': '#/$defs/CatalogComponentCommon'}, {'$ref': 'common_types.json#/$defs/Checkable'}, {'type': 'object', 'description': 'A component that allows selecting one or more options from a list.', 'properties': {'component': {'const': 'ChoicePicker'}, 'label': {'$ref': 'common_types.json#/$defs/DynamicString', 'description': 'The label for the group of options.'}, 'variant': {'type': 'string', 'description': 'A hint for how the choice picker should be displayed and behave.', 'enum': ['multipleSelection', 'mutuallyExclusive'], 'default': 'mutuallyExclusive'}, 'options': {'type': 'array', 'description': 'The list of available options to choose from.', 'items': {'type': 'object', 'properties': {'label': {'description': 'The text to display for this option.', '$ref': 'common_types.json#/$defs/DynamicString'}, 'value': {'type': 'string', 'description': 'The stable value associated with this option.'}}, 'required': ['label', 'value'], 'additionalProperties': False}}, 'value': {'$ref': 'common_types.json#/$defs/DynamicStringList', 'description': 'The list of currently selected values. This should be bound to a string array in the data model.'}, 'displayStyle': {'type': 'string', 'description': 'The display style of the component.', 'enum': ['checkbox', 'chips'], 'default': 'checkbox'}, 'filterable': {'type': 'boolean', 'description': 'If true, displays a search input to filter the options.', 'default': False}}, 'required': ['component', 'options', 'value']}], 'unevaluatedProperties': False}, 'Slider': {'type': 'object', 'allOf': [{'$ref': 'common_types.json#/$defs/ComponentCommon'}, {'$ref': '#/$defs/CatalogComponentCommon'}, {'$ref': 'common_types.json#/$defs/Checkable'}, {'type': 'object', 'properties': {'component': {'const': 'Slider'}, 'label': {'$ref': 'common_types.json#/$defs/DynamicString', 'description': 'The label for the slider.'}, 'min': {'type': 'number', 'description': 'The minimum value of the slider.', 'default': 0}, 'max': {'type': 'number', 'description': 'The maximum value of the slider.'}, 'value': {'$ref': 'common_types.json#/$defs/DynamicNumber', 'description': 'The current value of the slider.'}}, 'required': ['component', 'value', 'max']}], 'unevaluatedProperties': False}, 'DateTimeInput': {'type': 'object', 'allOf': [{'$ref': 'common_types.json#/$defs/ComponentCommon'}, {'$ref': '#/$defs/CatalogComponentCommon'}, {'$ref': 'common_types.json#/$defs/Checkable'}, {'type': 'object', 'properties': {'component': {'const': 'DateTimeInput'}, 'value': {'$ref': 'common_types.json#/$defs/DynamicString', 'description': 'The selected date and/or time value in ISO 8601 format. If not yet set, initialize with an empty string.'}, 'enableDate': {'type': 'boolean', 'description': 'If true, allows the user to select a date.', 'default': False}, 'enableTime': {'type': 'boolean', 'description': 'If true, allows the user to select a time.', 'default': False}, 'min': {'allOf': [{'$ref': 'common_types.json#/$defs/DynamicString'}, {'if': {'type': 'string'}, 'then': {'oneOf': [{'format': 'date'}, {'format': 'time'}, {'format': 'date-time'}]}}], 'description': 'The minimum allowed date/time in ISO 8601 format.'}, 'max': {'allOf': [{'$ref': 'common_types.json#/$defs/DynamicString'}, {'if': {'type': 'string'}, 'then': {'oneOf': [{'format': 'date'}, {'format': 'time'}, {'format': 'date-time'}]}}], 'description': 'The maximum allowed date/time in ISO 8601 format.'}, 'label': {'$ref': 'common_types.json#/$defs/DynamicString', 'description': 'The text label for the input field.'}}, 'required': ['component', 'value']}], 'unevaluatedProperties': False}, 'Canvas': {'type': 'object', 'allOf': [{'$ref': 'common_types.json#/$defs/ComponentCommon'}, {'$ref': '#/$defs/CatalogComponentCommon'}, {'type': 'object', 'description': 'Renders the UI element in a stateful panel next to the chat window.', 'properties': {'component': {'const': 'Canvas'}, 'children': {'$ref': 'common_types.json#/$defs/ChildList', 'description': 'Defines the children of the canvas.'}}, 'required': ['component', 'children']}], 'unevaluatedProperties': False}, 'Chart': {'type': 'object', 'allOf': [{'$ref': 'common_types.json#/$defs/ComponentCommon'}, {'$ref': '#/$defs/CatalogComponentCommon'}, {'type': 'object', 'description': 'An interactive chart that uses a hierarchical list of objects for its data.', 'properties': {'component': {'const': 'Chart'}, 'type': {'type': 'string', 'description': 'The type of chart to render.', 'enum': ['doughnut', 'pie']}, 'title': {'$ref': 'common_types.json#/$defs/DynamicString', 'description': 'The title of the chart.'}, 'chartData': {'description': 'The data for the chart, provided as a list of items. Can be a literal array or a data model path.', 'oneOf': [{'type': 'array', 'items': {'$ref': '#/$defs/ChartItem'}}, {'$ref': 'common_types.json#/$defs/DataBinding'}]}}, 'required': ['component', 'type', 'chartData']}], 'unevaluatedProperties': False}, 'GoogleMap': {'type': 'object', 'allOf': [{'$ref': 'common_types.json#/$defs/ComponentCommon'}, {'$ref': '#/$defs/CatalogComponentCommon'}, {'type': 'object', 'description': 'A component to display a Google Map with pins.', 'properties': {'component': {'const': 'GoogleMap'}, 'center': {'$ref': '#/$defs/DynamicLatLng', 'description': 'The center point of the map, containing latitude and longitude.'}, 'zoom': {'$ref': 'common_types.json#/$defs/DynamicNumber', 'description': 'The zoom level of the map.'}, 'pins': {'description': 'A list of pin objects to display on the map. Can be a literal array or a data model path.', 'oneOf': [{'type': 'array', 'items': {'$ref': '#/$defs/MapPin'}}, {'$ref': 'common_types.json#/$defs/DataBinding'}]}}, 'required': ['component', 'center', 'zoom']}], 'unevaluatedProperties': False}}, '$defs': {'CatalogComponentCommon': {'type': 'object', 'properties': {'weight': {'$ref': 'common_types.json#/$defs/DynamicNumber'}}}, 'ChartItem': {'type': 'object', 'properties': {'label': {'type': 'string'}, 'value': {'type': 'number'}, 'drillDown': {'type': 'array', 'description': 'An optional list of items for the next level of data.', 'items': {'type': 'object', 'properties': {'label': {'type': 'string'}, 'value': {'type': 'number'}}, 'required': ['label', 'value']}}}, 'required': ['label', 'value']}, 'DynamicLatLng': {'oneOf': [{'type': 'object', 'properties': {'lat': {'type': 'number'}, 'lng': {'type': 'number'}}, 'required': ['lat', 'lng'], 'additionalProperties': False}, {'$ref': 'common_types.json#/$defs/DataBinding'}]}, 'MapPin': {'type': 'object', 'properties': {'lat': {'type': 'number'}, 'lng': {'type': 'number'}, 'name': {'type': 'string'}, 'description': {'type': 'string'}, 'background': {'type': 'string', 'description': "Hex color code for the pin background (e.g., '#FBBC04')."}, 'borderColor': {'type': 'string', 'description': "Hex color code for the pin border (e.g., '#000000')."}, 'glyphColor': {'type': 'string', 'description': "Hex color code for the pin's glyph/icon (e.g., '#000000')."}}, 'required': ['lat', 'lng', 'name'], 'additionalProperties': False}, 'anyComponent': {'oneOf': [{'$ref': '#/components/AudioPlayer'}, {'$ref': '#/components/Button'}, {'$ref': '#/components/Canvas'}, {'$ref': '#/components/Card'}, {'$ref': '#/components/Chart'}, {'$ref': '#/components/CheckBox'}, {'$ref': '#/components/ChoicePicker'}, {'$ref': '#/components/Column'}, {'$ref': '#/components/DateTimeInput'}, {'$ref': '#/components/Divider'}, {'$ref': '#/components/GoogleMap'}, {'$ref': '#/components/Icon'}, {'$ref': '#/components/Image'}, {'$ref': '#/components/List'}, {'$ref': '#/components/Modal'}, {'$ref': '#/components/Row'}, {'$ref': '#/components/Slider'}, {'$ref': '#/components/Tabs'}, {'$ref': '#/components/Text'}, {'$ref': '#/components/TextField'}, {'$ref': '#/components/Video'}]}, 'anyFunction': {'enum': ['none']}}}
- **Judging Failure Reason (Grade I)**:
> Based on the criterion provided, let's assess the submission step by step:
>
> 1. **Analyze the Request**: The user is asking to visualize storefront sales across the "Pacific Northwest territory" (a regional/geographical request) to pinpoint underperforming "brick-and-mortar units" (store locations/geography-based performance).
> 2. **Analyze the Criterion**:
> * The criterion specifies that **Maps** must be used for requests about regional data, store locations, geography-based performance, or regional outliers.
> * Layout components like `Column` should be used as the `root`.
> 3. **Analyze the Submission**:
> * The root component is defined as `Canvas` with ID `root`, rather than a `Column` (though it does contain a column layout inside).
> * The submission contains KPI cards, a doughnut chart for sales distribution by state, a text-based operational efficiency profile, and a storefront registry table.
> * **Crucially, the submission does not include a Map component**, which is explicitly required by the criterion for regional data, store locations, and geography-based performance requests.
>
> Since the submission is missing the key visualization component (the Map) required for this type of regional query, it does not meet the core objectives of the criterion.
>
> GRADE: I

##### stockWatchlist
- **Judging Failure Reason (Grade P)**:
> To evaluate the submission against the criterion:
>
> 1. **Surface ID**: The submission correctly creates and updates a surface with the `surfaceId` set to `"main"`.
> 2. **Title Component**: The criterion asks for a `"Market Watch"` `"h1"` `"Text"` component. The submission includes a `Text` component with the text `"# Market Watch"` and the variant `"h2"`. While it captures the semantic meaning of the title, it uses `h2` instead of `h1` and includes the markdown symbol `#`. This is a minor cosmetic variation.
> 3. **List of Rows**: The submission includes a `List` component (`"watchlist_list"`) containing three `Row` components.
> 4. **Row Components**: Each of the three rows contains the respective stock symbol, price, and percentage change represented as `Text` components (e.g., `"**AAPL**"`, `"$150.00"`, and `"+1.2%"`).
>
> Because the submission is fully functional and correct with only minor cosmetic variations in the header text variant and style, it qualifies for partial credit.
>
> GRADE: P

##### flight_booking_flow
- **Algorithmic Failure Reason**:
> Validation failed: {'version': '0.9', 'createSurface': {'surfaceId': 'main', 'catalogId': 'https://a2ui.org/specification/v0_9/catalogs/basic/catalog.json'}} is not valid under any of the given schemas
> Context failures:
> - '0.9' is not one of ['v0.9', 'v0.9.1']
> - '0.9' is not one of ['v0.9', 'v0.9.1']
> - 'updateComponents' is a required property
> - Additional properties are not allowed ('createSurface' was unexpected)
> - '0.9' is not one of ['v0.9', 'v0.9.1']
> - 'updateDataModel' is a required property
> - Additional properties are not allowed ('createSurface' was unexpected)
> - '0.9' is not one of ['v0.9', 'v0.9.1']
> - 'deleteSurface' is a required property
> - Additional properties are not allowed ('createSurface' was unexpected)