dandi / dandi/dandi-cli

Metadata / validation not caught before attempting to upload

Open
#1,270 48 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
28
Forks
37
Avg merge
1d 17h
Merged PRs (30d)
9

Description

I am getting a sequence of error when some metadata is missing :

(nwb) jerome.lecoq@OSXLTCYGQCV upload % nwbinspector ./to_upload --config dandi

**************************************************
NWBInspector Report Summary

Timestamp: 2023-04-05 13:50:51.651946-07:00
Platform: macOS-12.6.3-arm64-arm-64bit
NWBInspector version: 0.4.26

Found 17 issues over 1 files:
2 - BEST_PRACTICE_VIOLATION
15 - BEST_PRACTICE_SUGGESTION
**************************************************

0 BEST_PRACTICE_VIOLATION
==========================

0.0 to_upload/Rorb-IRES2-Cre_590168381_590168385.nwb: check_index_series_points_to_image - 'IndexSeries' object at location '/stimulus/presentation/natural_movie_three_stimulus'
Message: Pointing an IndexSeries to a TimeSeries will be deprecated. Please point to an Images container instead.

0.1 to_upload/Rorb-IRES2-Cre_590168381_590168385.nwb: check_index_series_points_to_image - 'IndexSeries' object at location '/stimulus/presentation/natural_movie_one_stimulus'
Message: Pointing an IndexSeries to a TimeSeries will be deprecated. Please point to an Images container instead.

1 BEST_PRACTICE_SUGGESTION
===========================

1.2 to_upload/Rorb-IRES2-Cre_590168381_590168385.nwb: check_description - 'Device' object at location '/general/devices/eye-tracking camera'
Message: Description is missing.

1.3 to_upload/Rorb-IRES2-Cre_590168381_590168385.nwb: check_description - 'Device' object at location '/general/devices/display monitor'
Message: Description is missing.

1.4 to_upload/Rorb-IRES2-Cre_590168381_590168385.nwb: check_description - 'Device' object at location '/general/devices/Microscope'
Message: Description is missing.

1.5 to_upload/Rorb-IRES2-Cre_590168381_590168385.nwb: check_description - 'Device' object at location '/general/devices/2-photon microscope'
Message: Description is missing.

1.6 to_upload/Rorb-IRES2-Cre_590168381_590168385.nwb: check_description - 'Images' object with name 'SegmentationImages'
Message: Description ('no description') is a placeholder.

1.7 to_upload/Rorb-IRES2-Cre_590168381_590168385.nwb: check_description - 'GrayscaleImage' object with name 'mean'
Message: Description is missing.

1.8 to_upload/Rorb-IRES2-Cre_590168381_590168385.nwb: check_description - 'GrayscaleImage' object with name 'correlation'
Message: Description is missing.

1.9 to_upload/Rorb-IRES2-Cre_590168381_590168385.nwb: check_small_dataset_compression - 'OpticalSeries' object at location '/stimulus/templates/natural_movie_three_image_stack'
Message: data is not compressed. Consider enabling compression when writing a dataset.

1.10 to_upload/Rorb-IRES2-Cre_590168381_590168385.nwb: check_small_dataset_compression - 'OpticalSeries' object at location '/stimulus/templates/natural_movie_one_image_stack'
Message: data is not compressed. Consider enabling compression when writing a dataset.

1.11 to_upload/Rorb-IRES2-Cre_590168381_590168385.nwb: check_experimenter_exists - 'NWBFile' object at location '/'
Message: Experimenter is missing.

1.12 to_upload/Rorb-IRES2-Cre_590168381_590168385.nwb: check_experiment_description - 'NWBFile' object at location '/'
Message: Experiment description is missing.

1.13 to_upload/Rorb-IRES2-Cre_590168381_590168385.nwb: check_keywords - 'NWBFile' object at location '/'
Message: Metadata /general/keywords is missing.

1.14 to_upload/Rorb-IRES2-Cre_590168381_590168385.nwb: check_column_binary_capability - 'TimeIntervals' object with name 'trials'
Message: Column 'blank_sweep' uses 'float32' but has binary values [0. 1.]. Consider making it boolean instead and renaming the column to start with 'is_'; doing so will save 1.88KB.

1.15 to_upload/Rorb-IRES2-Cre_590168381_590168385.nwb: check_column_binary_capability - 'PlaneSegmentation' object with name 'PlaneSegmentation'
Message: Column 'Accepted' uses 'integers' but has binary values [0 1]. Consider making it boolean instead and renaming the column to start with 'is_'; doing so will save 13.02KB.

1.16 to_upload/Rorb-IRES2-Cre_590168381_590168385.nwb: check_column_binary_capability - 'PlaneSegmentation' object with name 'PlaneSegmentation'
Message: Column 'Rejected' uses 'integers' but has binary values [0 1]. Consider making it boolean instead and renaming the column to start with 'is_'; doing so will save 13.02KB.

(nwb) jerome.lecoq@OSXLTCYGQCV upload % cd 000459
(nwb) jerome.lecoq@OSXLTCYGQCV 000459 % dandi organize ../to_upload
2023-04-05 13:51:11,061 [ WARNING] A newer version (0.52.0) of dandi/dandi-cli is available. You are using 0.51.0
2023-04-05 13:51:11,490 [ INFO] NumExpr defaulting to 8 threads.
2023-04-05 13:51:12,251 [ INFO] Loading metadata from 1 files
[Parallel(n_jobs=-1)]: Using backend LokyBackend with 8 concurrent workers.
[Parallel(n_jobs=-1)]: Done 1 tasks | elapsed: 2.6s
[Parallel(n_jobs=-1)]: Done 1 out of 1 | elapsed: 2.6s finished
2023-04-05 13:51:14,851 [ WARNING] Completely empty record for ../to_upload/Rorb-IRES2-Cre_590168381_590168385.nwb
2023-04-05 13:51:14,851 [ INFO] Logs saved in /Users/jerome.lecoq/Library/Logs/dandi-cli/20230405205110Z-75206.log
Error: 1 out of 1 files were found not containing all necessary metadata: ../to_upload/Rorb-IRES2-Cre_590168381_590168385.nwb

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.