NEONScience / NEONScience/NEON-IS-data-processing

tempSpecificDepthLakes_level1_grp_reshape does not appear to handle sensor swaps properly

Open
#1,205 0 comments 0 reactions 1 assignee View on GitHub

@ncatolico is already working on this.

Since Jun 3, 2026.

Dominant language
R
Stars
13
Forks
8
Avg merge
3h 38m
Merged PRs (30d)
23

Description

Error in tempSpecificDepthLakes_level1_grp_reshape at datum: /2026/05/05/temp-specific-depths-lakes_FLNT103100

Logs:

INFO  [2026-05-08 00:27:48.360] Processing path to datum: /tmp/pfs/filter_joined/2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885
ERROR [2026-05-08 00:27:48.362] In def.validate.json::::: Error, the file does not exist, /tmp/pfs/filter_joined/2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/location/tchain_51220_locations.json
ERROR [2026-05-08 00:27:48.362] In def.validate.json::::: Error, the file does not exist, /tmp/pfs/filter_joined/2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/location/tchain_51655_locations.json
ERROR [2026-05-08 00:27:48.363]
Error resulted from call to:
NEONprocIS.base::def.loc.geo.hist(locFile, log = NULL)
Call stack:
[1] "tryCatch(withCallingHandlers(wrap.file.comb.tsdl.splt(InfoDirIn = InfoDirIn"
[2] "tryCatchList(expr, classes, parentenv, handlers)"
[3] "tryCatchOne(expr, names, parentenv, handlers[[1]])"
[4] "doTryCatch(return(expr), name, parentenv, handler)"
[5] "withCallingHandlers(wrap.file.comb.tsdl.splt(InfoDirIn = InfoDirIn, DirOut "
[6] "wrap.file.comb.tsdl.splt(InfoDirIn = InfoDirIn, DirOut = Para$DirOut, fileP"
[7] "NEONprocIS.base::def.loc.geo.hist(locFile, log = NULL)"
[8] "stop()"
[9] ".handleSimpleError(function (err) \n{\n call.stack <- base::sys.calls()\n N"
[10] "h(simpleError(msg, call))"
INFO [2026-05-08 00:27:48.365] Re-routing failed datum path /tmp/pfs/filter_joined/2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885 to /pfs/out/errored_datums
INFO [2026-05-08 00:27:48.367] Removed partial output for errored datum: /pfs/out/2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885

There was a sensor swap this day. Repo structure of tempSpecificDepthLakes_group_path at this datum:

$ pc glob file tempSpecificDepthLakes_group_path@master:/2026/05/05/*FLNT*/**
SIZE    NAME
750B    /2026/05/05/temp-specific-depths-lakes_FLNT103100/group/
750B    /2026/05/05/temp-specific-depths-lakes_FLNT103100/group/CFGLOC113885.json
54.12KiB        /2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/
54.12KiB        /2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/
9.722KiB        /2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/data/
9.722KiB        /2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/data/tchain_parsed_CFGLOC113885_2026-05-05.parquet
14.14KiB        /2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/flags/
14.14KiB        /2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/flags/tchain_parsed_CFGLOC113885_2026-05-05_flagsCal.parquet
4.419KiB        /2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/location/
655B    /2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/location/CFGLOC113885.json
1.89KiB /2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/location/tchain_51220_locations.json
1.89KiB /2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/location/tchain_51655_locations.json
9.419KiB        /2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/uncertainty_coef/
9.419KiB        /2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/uncertainty_coef/tchain_parsed_CFGLOC113885_2026-05-05_uncertaintyCoef.json
16.42KiB        /2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/uncertainty_data/
16.42KiB        /2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/uncertainty_data/tchain_parsed_CFGLOC113885_2026-05-05_uncertaintyData.parquet

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.