ISA-tools / ISA-tools/isa-api

Are Multiple "<entity> Name" Columns Allowed?

Open
#500 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
48
Forks
42
PR merge metrics
No merged PRs in 30d

Description

I am trying to figure out how to create valid ISA-Tab/ISA-JSON with a more complex sample lineage than what the examples show. The examples are pretty much just source -> sample -> extract, but what about something like source -> sample1 -> sample2 -> extract? Is this allowed or do you have to reduce things down to 1 sample?

The last sentence here https://isa-specs.readthedocs.io/en/latest/isatab.html#study-table-file suggests to me that it should be possible:
"Node properties, such as Characteristics (for Material nodes), Parameter Value (for Process nodes) and additional Name columns for special cases of Process node to disambiguate Protocol REF entries of MUST follow the named node of context."
What does "additional Name columns" refer to if not additional Sample Name columns?

Additionally, I can modify a JSON example and convert it to Tab so that it produces a study file with an additional Sample Name column, but if I try to convert a modified Tab to JSON with an additional Sample Name column there is an error.

Code to modify the BII-I-1.json example and convert to Tab:

with open('C:/Users/Sparda/Desktop/Moseley Lab/Code/MESSES/isadatasets/json/BII-I-1/BII-I-1.json', 'r') as jsonFile:
    isa_example = json.load(jsonFile)
    
samples = []
for sample in isa_example["studies"][0]["materials"]["samples"]:
    samples.append({"@id":sample["@id"]})

growth_protocol2 = {
          "@id": "#protocol/growth_protocol_2",
          "components": [],
          "description": "1. Biomass samples (45 ml) were taken via the sample port of the Applikon fermenters. The cells were pelleted by centrifugation for 5 min at 5000 rpm. The supernatant was removed and the RNA pellet resuspended in the residual medium to form a slurry. This was added in a dropwise manner directly into a 5 ml Teflon flask (B. Braun Biotech, Germany) containing liquid nitrogen and a 7 mm-diameter tungsten carbide ball. After allowing evaporation of the liquid nitrogen the flask was reassembled and the cells disrupted by agitation at 1500 rpm for 2 min in a Microdismembranator U (B. Braun Biotech, Germany) 2. The frozen powder was then dissolved in 1 ml of TriZol reagent (Sigma-Aldrich, UK), vortexed for 1 min, and then kept at room temperature for a further 5min. 3. Chloroform extraction was performed by addition of 0.2 ml chloroform, shaking vigorously or 15 s, then 5min incubation at room temperature. 4. Following centrifugation at 12,000 rpm for 5 min, the RNA (contained in the aqueous phase) was precipitated with 0.5 vol of 2-propanol at room temperature for 15 min. 5. After further centrifugation (12,000 rpm for 10 min at 4 C) the RNA pellet was washed twice with 70 % (v/v) ethanol, briefly air-dried, and redissolved in 0.5 ml diethyl pyrocarbonate (DEPC)-treated water. 6. The single-stranded RNA was precipitated once more by addition of 0.5 ml of LiCl buffer (4 M LiCl, 20 mM Tris-HCl, pH 7.5, 10 mM EDTA), thus removing tRNA and DNA from the sample. 7. After precipitation (20 C for 1h) and centrifugation (12,000 rpm, 30 min, 4 C), the RNA was washed twice in 70 % (v/v) ethanol prior to being dissolved in a minimal volume of DEPC-treated water. 8. Total RNA quality was checked using the RNA 6000 Nano Assay, and analysed on an Agilent 2100 Bioanalyser (Agilent Technologies). RNA was quantified using the Nanodrop ultra low volume spectrophotometer (Nanodrop Technologies).",
          "name": "growth protocol 2",
          "parameters": [],
          "protocolType": {
            "annotationValue": "growth",
            "termAccession": "",
            "termSource": ""
          },
          "uri": "",
          "version": ""
        }
isa_example["studies"][0]["protocols"].append(growth_protocol2)

new_process = {
          "@id": "#process/growth_protocol_2_1",
          "comments": [],
          "date": "",
          "executesProtocol": {
            "@id": "#protocol/growth_protocol_2"
          },
          "inputs": [
            {
              "@id": "#sample/sample-E-0.07-aliquot1"
            }
          ],
          "outputs": [
            {
              "@id": "#sample/sample-E-0.07-aliquot1_1"
            }
          ],
          "parameterValues": [],
          "performer": "",
          "previousProcess": {"@id": "#process/growth_protocol13"}
        }
isa_example["studies"][0]["processSequence"].append(new_process)

new_sample = {
            "@id": "#sample/sample-E-0.07-aliquot1_1",
            "characteristics": [
              {
                "category": {
                  "@id": "#characteristic_category/Material_Type"
                },
                "value": {
                  "annotationValue": "internal",
                  "termAccession": "",
                  "termSource": ""
                }
              }
            ],
            "derivesFrom": [
            ],
            "factorValues": [
              {
                "category": {
                  "@id": "#factor/limiting_nutrient"
                },
                "value": {
                  "annotationValue": "ethanol",
                  "termAccession": "",
                  "termSource": ""
                }
              },
              {
                "category": {
                  "@id": "#factor/rate"
                },
                "unit": {
                  "@id": "#Unit/l/hour"
                },
                "value": 0.07
              }
            ],
            "name": "sample-E-0.07-aliquot1_1"
          }
isa_example["studies"][0]["materials"]["samples"].append(new_sample)

with open('C:/Users/Sparda/Desktop/Moseley Lab/Code/MESSES/isadatasets/BII-I-1_testing.json', 'w') as out_fp:
     json.dump(isa_example, out_fp, indent=2)

with open('C:/Users/Sparda/Desktop/Moseley Lab/Code/MESSES/isadatasets/BII-I-1_testing.json') as file_pointer:
    json2isatab.convert(file_pointer, 'C:/Users/Sparda/Desktop/Moseley Lab/Code/MESSES/isadatasets/BII-I-1_testing/', validate_first=False)

I modified the BII-S-1 tabular example in 2 different ways, one that adds a new sample before the existing one and another that adds a new sample after the existing one. I have attached them as s_BII-S-1_a.txt and s_BII-S-1_b.txt, respectively, and attached the investigation file with the added protocol as well.
s_BII-S-1_a.txt
s_BII-S-1_b.txt
i_investigation.txt

s_BII-S-1_a.txt will not convert because the validation fails after not finding the samples from the assays in the study file.
s_BII-S-1_b.txt fails with a traceback:

Traceback (most recent call last):

  File "C:\Users\Sparda\AppData\Local\Temp\ipykernel_5600\2959741035.py", line 1, in <cell line: 1>
    isa_json = isatab2json.convert('C:/Users/Sparda/Desktop/Moseley Lab/Code/MESSES/isadatasets/tab/BII-I-1_conversion_testing2', use_new_parser=True, validate_first=False)

  File "C:\Python310\lib\site-packages\isatools\convert\isatab2json.py", line 56, in convert
    ISA = isatab.load(fp)

  File "C:\Python310\lib\site-packages\isatools\isatab\load\core.py", line 283, in load
    ).create_from_df(study_tfile_df)

  File "C:\Python310\lib\site-packages\isatools\isatab\load\ProcessSequenceFactory.py", line 398, in create_from_df
    if source_node_context not in sample_node_context.derives_from:

AttributeError: 'NoneType' object has no attribute 'derives_from'

If I try to convert s_BII-S-1_a.txt with validate_first=False it will have the same traceback.

I investigated the traceback and the issue seems to be an assumption in the ProcessSequenceFactory that there will be one and only one Source Name and Sample Name for part of the code, but not the other part. Certain parts of the code directly look for 'Sample Name' and 'Source Name', but the code that actually produces the error instead uses object_label.startswith('Sample Name'). It is clear that some parts are aware that there can be multiple columns with the same name evidenced by the use of the startswith method, but when initially determining sources and samples only the literal 'Source Name' and 'Sample Name' columns are used. Multiple 'Source Name' and 'Sample Name' columns are not looked for.

So, are multiple " Name" columns allowed?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in isatools/isatab/load/ProcessSequenceFactory.py, especially create_from_df, and compare the literal Source Name/Sample Name handling with the startswith-based path. Reproduce the failure using the attached s_BII-S-1_a.txt and s_BII-S-1_b.txt files through isatab2json.convert; done means the supported behavior for multiple entity Name columns is established and conversion no longer raises the reported error if they are valid.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.