Security / Administrative Review

I scanned the project to get a feel for things and have not found any critical issues from an administrative or security point of view that would block making the code open source in the DOI-USGS organization.

An AI-generated security scan is pasted below... I don't think any of the issues it identified warrant follow up but offer this as evidence of what exists in the repository.

Click to expand/collapse report

# Security Scan Report — HyTEST Repository

*Scan date: 2026-02-18 | Scope: all git-tracked files*

No actual credentials, private keys, or tokens were found. The issues below are personal information and internal infrastructure details that may warrant review before public exposure.

---

## MEDIUM — Personal Usernames in HPC Home Directory Paths

These are hardcoded personal home-directory paths for named users, which will silently fail for anyone else running the code and expose internal usernames.

| File | Line | Content |
|------|------|---------|
| `environment_set_up/Start_Dask_Cluster_Tallgrass.ipynb` | 36 | `shared_temp_directory='/home/asnyder/tmp'` |
| `dataset_processing/tutorials/niwaa_wrfhydro_monthly_huc12_agg/02_Spatial_Aggregation/01_2D_spatial_aggregation.ipynb` | 250 | `shared_temp_directory='/home/lstaub/tmp'` |

**Recommendation:** Replace with a dynamically resolved path such as `os.path.expanduser('~/tmp')` or an environment variable.

---

## MEDIUM — Personal Username in Production Data Path (Catalog)

A personal username (`rsignell`) appears in production data paths registered in the intake catalog. This may indicate data is stored in a personal project directory rather than a shared project directory.

| File | Line | Content |
|------|------|---------|
| `dataset_catalog/hytest_intake_catalog.yml` | 68 | `/caldera/projects/usgs/hazards/cmgp/woodshole/rsignell/conus404/zarr/nwis_chanobs.zarr` |
| `dataset_catalog/hytest_intake_catalog.yml` | 87 | `/caldera/projects/usgs/hazards/cmgp/woodshole/rsignell/conus404/zarr/chanobs.zarr` |
| `dataset_processing/demos/nwis_to_nwm_gages_rechunking.ipynb` | 321 | (commented out) same `rsignell` path |

**Recommendation:** Confirm data has been or should be migrated to a shared project directory. Update catalog paths if so.

---

## MEDIUM — Personal Email Addresses

Multiple named individuals' email addresses appear in source files. While these may be intentional contact information, they should be reviewed to confirm they are appropriate for a public repository.

| File | Line | Address |
|------|------|---------|
| `code.json` | 40 | `asnyder@usgs.gov` |
| `environment_set_up/QuickStart-Cloud-Nebari.md` | 13 | `asnyder@usgs.gov` |
| `doc/Notebook_review_guide.md` | 103 | `alaws@contractor.usgs.gov` |
| `evaluation/tutorials/CONUS404/1_CONUS404_DRB_Data_Prep.ipynb` | 9 | `alaws@usgs.gov` |
| `evaluation/tutorials/CONUS404/2_CONUS404_DRB_Analysis.ipynb` | 11 | `alaws@usgs.gov` |
| `dataset_catalog/STAC.ipynb` | 322 | `mdmf@usgs.gov` |
| `essential_reading/gdp_data_prep.md` | 18 | `dblodgett@usgs.gov` |

**Recommendation:** Consider replacing individual addresses with a team/project inbox or GitHub Discussions link to reduce personal exposure and maintenance burden when staff changes.

---

## LOW — Internal HPC Hostname Disclosed

The USGS HPC hostname is referenced in documentation and scripts.

| File | Line | Content |
|------|------|---------|
| `environment_set_up/jupyter-start.sh` | 29 | `$USER@denali.cr.usgs.gov` |

**Recommendation:** This is likely intentional in setup docs, but confirm this hostname is appropriate to publish publicly.

---

## LOW — Internal Filesystem Paths (`/caldera/...`) Widely Referenced

Many catalog YAML files and notebooks contain absolute paths rooted at `/caldera/`, the USGS HPC shared storage system. These paths are used for HPC-local data access and reveal internal directory structure.

Key files affected (many entries each):

- `dataset_catalog/subcatalogs/conus404-catalog.yml`
- `dataset_catalog/subcatalogs/nhm-v1.0-daymet-catalog.yml`
- `dataset_catalog/subcatalogs/nhm-v1.1-gridmet-catalog.yml`
- `dataset_catalog/subcatalogs/nhm-v1.1-c404-bc-catalog.yml`
- `dataset_processing/tutorials/niwaa_wrfhydro_monthly_huc12_agg/` (multiple scripts and notebooks)
- `dataset_processing/tutorials/niwaa_wrfhydro_monthly_huc12_agg/02_Spatial_Aggregation/usgs_common.py` (lines 47–62, hardcoded)

**Recommendation:** This is somewhat expected for HPC-targeted tutorials but is worth documenting clearly. Paths like the ones in `usgs_common.py` are particularly fragile — consider making them configurable rather than hardcoded.

---

## LOW — SLURM Project Account Name Hardcoded

| File | Line | Content |
|------|------|---------|
| `environment_set_up/Start_Dask_Cluster_Tallgrass.ipynb` | 30 | `account='hytest'` |

This is a fallback value if `SLURM_JOB_ACCOUNT` is not set. Not a secret, but documents an internal HPC allocation name.

---

## NOT AN ISSUE — AWS Credential Handling Code

Several notebooks contain code that reads `~/.aws/credentials` and sets `AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY` environment variables. **No actual credential values are present** — only variable names and standard boto/fsspec patterns. This is expected behavior for AWS credential bootstrapping.

---

## Summary

| Severity | Count | Description |
|----------|-------|-------------|
| HIGH | 0 | No hardcoded secrets, tokens, or private keys found |
| MEDIUM | 3 | Personal usernames in paths, personal emails, username in production data path |
| LOW | 3 | Internal hostname, internal filesystem paths, SLURM account name |