Validation fails with multi-file schemas with relative file names

I have a complex web of JSON schemas that serve to specify a data model. Since we do not want to make the schemas world visible, and want to be able to move the entire code tree without breaking, we want to use relative file names for $ref. We are using a Registry instance to find the schema files. The $refs are of the form "../somedir/schemafile.json", as the schemas are put into parallel directories, being too numerous to fit well into a single directory.

It turns out that in some cases the $ref strings are mangled, so that the lookup fails, for which file varies a bit between different tests.

There might be a better way of achieving the desired result that I just do not know, but this behaviour does not look right.

Reproducing the problem

The problem can be reproduced with the following files:

top/test,py

import json
import jsonschema
from pathlib import Path
from referencing import Registry, Resource
from referencing.exceptions import NoSuchResource

BASE = Path(__file__).parent
def retrieve_from_filesystem(uri: str):
    if not uri.startswith("../"):
        raise NoSuchResource(ref=uri)
    path = BASE / Path(uri.removeprefix("../"))
    contents = json.loads(path.read_text())
    return Resource.from_contents(contents)

registry = Registry(retrieve=retrieve_from_filesystem)


if __name__ == "__main__":
    instance = BASE / "dttest" / "tst_instance.json"
    schema = BASE / "schem" / "DropImageData.json"
    jsonschema.validate(
        instance=json.load(open(instance)),
        schema=json.load(open(schema)),
        registry=registry
    )

top/dttest/tst_instance.json

{
  "mimeType":"image/jpeg",
  "data": "some UUencoded data would be here"
}

top/schem/DropImage.json

{
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
      "mimeType": {
        "allOf": [
          {
            "$ref": "../schem/ImageMimeType.json"
          }
        ]
      }
    },
    "required": ["mimeType"]
}

top/schem/DropImageData.json

{
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "description": "DropImage, containing image as attached data.",
    "title": "DropImageData",
    "type": "object",
    "allOf": [
        {
            "$ref": "../schem/DropImage.json"
        }
    ]
}

top/schem/ImageMimetype.json

{
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "title": "ImageMimetype",
    "type": "string",
    "enum": [
                "image/png",
                "image/jpeg"
    ]
}

Error output:

Traceback (most recent call last):
  File "/home/rhfogh/Software/miniconda3/envs/mxlims/lib/python3.10/site-packages/jsonschema/validators.py", line 462, in _validate_reference
    resolved = self._resolver.lookup(ref)
  File "/home/rhfogh/Software/miniconda3/envs/mxlims/lib/python3.10/site-packages/referencing/_core.py", line 684, in lookup
    raise exceptions.Unresolvable(ref=ref) from None
referencing.exceptions.Unresolvable: ../schem/ImageMimeType.json

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/rhfogh/pycharm/mxlims_data_model/mxlims/test2/test.py", line 21, in <module>
    jsonschema.validate(
  File "/home/rhfogh/Software/miniconda3/envs/mxlims/lib/python3.10/site-packages/jsonschema/validators.py", line 1330, in validate
    error = exceptions.best_match(validator.iter_errors(instance))
  File "/home/rhfogh/Software/miniconda3/envs/mxlims/lib/python3.10/site-packages/jsonschema/exceptions.py", line 479, in best_match
    best = max(errors, key=key, default=None)
  File "/home/rhfogh/Software/miniconda3/envs/mxlims/lib/python3.10/site-packages/jsonschema/validators.py", line 383, in iter_errors
    for error in errors:
  File "/home/rhfogh/Software/miniconda3/envs/mxlims/lib/python3.10/site-packages/jsonschema/_keywords.py", line 334, in allOf
    yield from validator.descend(instance, subschema, schema_path=index)
  File "/home/rhfogh/Software/miniconda3/envs/mxlims/lib/python3.10/site-packages/jsonschema/validators.py", line 431, in descend
    for error in errors:
  File "/home/rhfogh/Software/miniconda3/envs/mxlims/lib/python3.10/site-packages/jsonschema/_keywords.py", line 275, in ref
    yield from validator._validate_reference(ref=ref, instance=instance)
  File "/home/rhfogh/Software/miniconda3/envs/mxlims/lib/python3.10/site-packages/jsonschema/validators.py", line 431, in descend
    for error in errors:
  File "/home/rhfogh/Software/miniconda3/envs/mxlims/lib/python3.10/site-packages/jsonschema/_keywords.py", line 296, in properties
    yield from validator.descend(
  File "/home/rhfogh/Software/miniconda3/envs/mxlims/lib/python3.10/site-packages/jsonschema/validators.py", line 431, in descend
    for error in errors:
  File "/home/rhfogh/Software/miniconda3/envs/mxlims/lib/python3.10/site-packages/jsonschema/_keywords.py", line 334, in allOf
    yield from validator.descend(instance, subschema, schema_path=index)
  File "/home/rhfogh/Software/miniconda3/envs/mxlims/lib/python3.10/site-packages/jsonschema/validators.py", line 431, in descend
    for error in errors:
  File "/home/rhfogh/Software/miniconda3/envs/mxlims/lib/python3.10/site-packages/jsonschema/_keywords.py", line 275, in ref
    yield from validator._validate_reference(ref=ref, instance=instance)
  File "/home/rhfogh/Software/miniconda3/envs/mxlims/lib/python3.10/site-packages/jsonschema/validators.py", line 464, in _validate_reference
    raise exceptions._WrappedReferencingError(err) from err
jsonschema.exceptions._WrappedReferencingError: Unresolvable: ../schem/ImageMimeType.json

Some limited debugging shows that the validation fails when referencing._core.lookup is called, in a case where
self._base_uri == ref
As a result ref is converted from '../schem/ImageMimeType.json' to a uri of 'schem/ImageMimeType.json' , which cannot be found in the Registry lookup. The problem happens in line 680 of _core.py, where the urljoin function removes the '../' prefix.
It is notable that this does not happen the first time the program gets to this line, so that the ref '../schem/DropImage.json' is processed correctly, presumably because self._base_url is an empty string at this point.

Versions used

jsonschema version 4.26.0
Python 3.10
JSONschema version 2020-12

Start with the reproducer in top/test.py and inspect referencing/_core.py around line 680, where urljoin changes the relative $ref during lookup. Confirm the behavior with the provided schema tree and determine why the nested reference is resolved differently; done means validation succeeds for the reproducer and regression coverage protects relative references across nested schemas.