python-jsonschema / python-jsonschema/check-jsonschema

Support for custom YAML tags

Open
#489 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
341
Forks
71
Avg merge
15d 20h
Merged PRs (30d)
4

Description

This issue requested support specifically for the !reference tag in a Gitlab CI file, however, there doesn't appear to be support for other tags. Specifically in my case, Ansible-specific tags like !unsafe or !vault.

Example content:

controller_templates:
  - name: My job template
    extra_vars:
      pass: !unsafe '{{ my_value }}'

Output:

Traceback (most recent call last):
  File "/home/user/.local/bin/check-jsonschema", line 8, in <module>
    sys.exit(main())
  File "/home/user/.local/lib/python3.6/site-packages/check_jsonschema/__init__.py", line 26, in main
    ret = checker.run()
  File "/home/user/.local/lib/python3.6/site-packages/check_jsonschema/checker.py", line 83, in run
    self._run()
  File "/home/user/.local/lib/python3.6/site-packages/check_jsonschema/checker.py", line 66, in _run
    errors = self._build_error_map()
  File "/home/user/.local/lib/python3.6/site-packages/check_jsonschema/checker.py", line 56, in _build_error_map
    for filename, doc in self._instance_loader.iter_files():
  File "/home/user/.local/lib/python3.6/site-packages/check_jsonschema/loaders/instance/__init__.py", line 63, in iter_files
    data = loadfunc(fp)
  File "/home/user/.local/lib/python3.6/site-packages/ruamel/yaml/main.py", line 343, in load
    return constructor.get_single_data()
  File "/home/user/.local/lib/python3.6/site-packages/ruamel/yaml/constructor.py", line 113, in get_single_data
    return self.construct_document(node)
  File "/home/user/.local/lib/python3.6/site-packages/ruamel/yaml/constructor.py", line 123, in construct_document
    for _dummy in generator:
  File "/home/user/.local/lib/python3.6/site-packages/ruamel/yaml/constructor.py", line 723, in construct_yaml_map
    value = self.construct_mapping(node)
  File "/home/user/.local/lib/python3.6/site-packages/ruamel/yaml/constructor.py", line 440, in construct_mapping
    return BaseConstructor.construct_mapping(self, node, deep=deep)
  File "/home/user/.local/lib/python3.6/site-packages/ruamel/yaml/constructor.py", line 255, in construct_mapping
    value = self.construct_object(value_node, deep=deep)
  File "/home/user/.local/lib/python3.6/site-packages/ruamel/yaml/constructor.py", line 146, in construct_object
    data = self.construct_non_recursive_object(node)
  File "/home/user/.local/lib/python3.6/site-packages/ruamel/yaml/constructor.py", line 181, in construct_non_recursive_object
    data = constructor(self, node)
  File "/home/user/.local/lib/python3.6/site-packages/ruamel/yaml/constructor.py", line 743, in construct_undefined
    node.start_mark,
ruamel.yaml.constructor.ConstructorError: could not determine a constructor for the tag '!unsafe'
  in "job_templates.yml", line 337, column 18

The workaround in my case is to convert the content to JSON with yq then validate--which isn't ideal.

Is it possible to add support for custom tags like these? Possibly by allowing the user to pass in a custom constructor for their respective tags that can be passed through to the ruamel.yaml parser.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in loaders/instance/init.py and trace the ruamel.yaml loading path shown in the report. Reproduce the Ansible !unsafe example, then determine the supported behavior for custom tags and how validation should proceed. Done means tagged YAML no longer fails solely because its tag lacks a constructor.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, yaml
Domain
cli
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.