Should represent Table's fields declaration and rows instances as a class?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 886
- Forks
- 137
- PR merge metrics
- No merged PRs in 30d
Description
Currently we use two data types to represent something that could be represented in one class. The first is the fields parameter received by import_from_* (which are passed to utils.create_table), like:
UWSGI_FIELDS = OrderedDict([('pid', rows.fields.IntegerField),
('ip', rows.fields.UnicodeField),
('datetime', rows.fields.DatetimeField),
('http_verb', rows.fields.UnicodeField),
('http_path', rows.fields.UnicodeField),
('generation_time', rows.fields.FloatField),
('http_version', rows.fields.FloatField),
('http_status', rows.fields.IntegerField)])
Second is the Table.Row (created in Table.__init__), which is a named tuple containing row data.
We could use an approach similar to ORMs and use a class to define the fields, like Django does. We could start with something like this:
class UwsgiLog(rows.Row):
pid = rows.fields.IntegerField()
ip = rows.fields.UnicodeField()
datetime = rows.fields.DatetimeField()
http_verb = rows.fields.UnicodeField()
http_path = rows.fields.UnicodeField()
generation_time = rows.fields.FloatField()
http_version = rows.fields.FloatField()
http_status = rows.fields.IntegerField()
And the Table rows (returned when we iterate over it) will be instances of UwsgiLog.
Pros:
- This syntax is more flexible since we can create utility methods inside the class
- More declarative
Cons:
- We may not have access to the field order in this case (which is very important)
namedtupleis probably faster than any other customized class- We'll need to add more complexity to the code
Note: check if we can integrate this feature with scrapy so it'll easier to parse data using rows in a scrapy project.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the fields passed to import_from_*, utils.create_table, and Table.__init__, then inspect rows.fields and the current Table.Row named tuple. Review the possible Scrapy integration mentioned in the issue. Done requires an agreed design for class-based field declarations and row instances, including field ordering and compatibility with existing behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100