cognitedata / cognitedata/python-extractor-utils

SequenceUploadQueue add_to_upload_queue incorrectly describes column_external_ids

Open
#212 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
13
Forks
6
Avg merge
2d 6h
Merged PRs (30d)
11

Description

https://github.com/cognitedata/python-extractor-utils/blob/3e99e07943a7bd0c16ca3f4bd224a81c2052480f/cognite/extractorutils/uploader/time_series.py#L428-L452

The parameter description states:
column_external_ids: List of external id for the columns of the sequence

But the parameter takes in:
column_external_ids: Optional[List[dict]] = None

As opposed to a List[str]. It then will try to plug it into the columns parameter SequenceData(id=id, external_id=id, rows=rows, columns=column_external_ids) which will later result in error:

return [cast(str, c.get("externalId")) for c in self.columns]

SequenceData is expecting an API like dict for columns:
https://docs.cognite.com/api/v1/#tag/Sequences/operation/createSequence

Possible options:

  1. Change the parameter for add_to_upload_queue from column_external_ids to columns but that will change the look and feel. The SDK feel is to insert a list of str as column_external_ids.
  2. Keep it as column_external_ids and in add_to_upload_queue, generate the API like dict for columns from the list of str column_external_ids.
  3. Within the SequenceUploadQueue, there is a function set_sequence_column_definition that sets self.column_definitions but it isn't used unless a create is being called. It feels super odd when you want to create on missing Sequences and call set_sequence_column_definition with column definitions, but when calling add_to_upload_queue, you must additionally add column definitions again.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with add_to_upload_queue in cognite/extractorutils/uploader/time_series.py at the linked lines, then inspect SequenceData and set_sequence_column_definition. Compare the documented column_external_ids contract with the columns value passed to SequenceData and the referenced Sequences API format. Done means choosing one consistent contract and updating the affected behavior and description accordingly.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.