jacebrowning / jacebrowning/datafiles

Best way to map existing files to Model instances

Open
#236 6 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Python
Stars
213
Forks
23
Avg merge
1d 22h
Merged PRs (30d)
5

Description

Is there a way to map existing files with the same schema _that do not match a repeatable pattern on disk_ to a datafiles Model instance manually? The use case is config files spread across arbitrary-depth subfolders below a top-level project directory. Using glob I can find the files I am interested in mapping, but I am not having much success creating mapped instances of those discovered files.

I have tried:

- Overriding `Model.Meta.dataclass_pattern` with each discovered files path and calling `Model.objects.get()`
- Creating instances of a model with default values and no `pattern` defined and then overriding both the `instance.Meta.datafiles_pattern` and `instance.datafile.path` attributes on the instance, with the correct path for the discovered file, before calling `instance.datafile.load()`.

However in both cases this results in an odd behaviour where all instances with nested attributes contain pointers to the most recently loaded files' nested object rather than their own 🤦‍♂️

Is this a completely unsupported use-case, or is there another way to use `datafiles` to map files discovered outside of the supported 'pattern' construct to instances of a datafiles Model? Thank you!

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the nested-object pointer behavior using the glob-discovered paths and the attempted Model.Meta.dataclass_pattern, Model.objects.get(), and instance.datafile.load() approaches. Read the datafiles model and instance-mapping entry points around instance.Meta.datafiles_pattern and instance.datafile.path; done means establishing whether arbitrary discovered files can be mapped independently or documenting that this use case is unsupported.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.