ESMCI / ESMCI/cprnc

Non-UTF8 characters are output when a scalar changes between files

Open
#15 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Fortran
Stars
4
Forks
7
PR merge metrics
No merged PRs in 30d

Description

When I was working on new input files for [ESCOMP/CAM #1029](https://github.com/ESCOMP/CAM/pull/1029) I was getting unexpected failures of the BASELINE for tests using the new files. Instead of finding a "DIFF" the tests were reporting "`Exception during BASELINE:`".

Example TestStatus.log output:
```
2024-09-11 20:22:32: Exception during BASELINE:
'utf-8' codec can't decode byte 0xf5 in position 6813: invalid start byte
Traceback (most recent call last):
File "/glade/derecho/scratch/gdicker/2024Jun04_CAMMPASv8Grids_2/CAM_prtest/cime/CIME/SystemTests/system_tests_common.py", line 618, in _phase_modifying_call
function()
File "/glade/derecho/scratch/gdicker/2024Jun04_CAMMPASv8Grids_2/CAM_prtest/cime/CIME/SystemTests/system_tests_common.py", line 758, in _compare_baseline
self._log_cprnc_output_tail(pattern)
File "/glade/derecho/scratch/gdicker/2024Jun04_CAMMPASv8Grids_2/CAM_prtest/cime/CIME/SystemTests/system_tests_common.py", line 540, in _log_cprnc_output_tail
cprnc_log_tail = fin.readlines()[-20:]
File "/glade/u/apps/derecho/23.09/opt/._view/ki3gspoeflu225iuymcch2cddghxcain/lib/python3.10/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf5 in position 6813: invalid start byte
```

This is because the following text is in the \*.cprnc.out file that was being read and causing the python `read` or `readlines` functions to fail:
```
cf1
1 1.346970915794373E+00õÃ5X1<8d>õ?
RMS cf1 2.9020E-08 NORMALIZED 0.0000E+00

cf2
1 -5.392833948135376E-01Ù^U(TÏAá¿
RMS cf2 3.7545E-08 NORMALIZED 0.0000E+00

cf3
1 1.923125386238098E-01»7ò<8e>²<9d>È?
RMS cf3 6.9598E-09 NORMALIZED 0.0000E+00
```

The dimensionless cf1, cf2, and cf3 variables changed between files (as expected), but apparently were not handled well when CPRNC printed results.

Example to reproduce:

Using version after tag cam6_4_033 from ESCOMP/CAM, run a ERS_Ln9_P36x1.mpasa480_mpasa480.F2000climo.derecho_intel.cam-outfrq9s_mpasa480 test and compare against the cam6_4_032 baseline.

```bash
# From within a fully checked out CAM clone, assuming Derecho
./cime/scripts/create_test \
--compare "/glade/campaign/cesm/community/amwg/cam_baselines/cam6_4_032_intel" \
ERS_Ln9_P36x1.mpasa480_mpasa480.F2000climo.derecho_intel.cam-outfrq9s_mpasa480
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.