ccpem / ccpem/mrcfile

Writing/reading extended header seems to be broken in latest version

Open
#58 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
86
Forks
24
PR merge metrics
No merged PRs in 30d

Description

Hi @colinpalmer

I am having an issue with the latest version of mrcfile on pypi. It looks like there is some issue with the extended header being written.

If I write an mrcfile and then try to read the file back, the extended header does not seem to be interpreted correctly. Instead of an array of records I just get something like "[b'\x00' b'\x00' b'\x00' b'\x00' b'\x00' b'\x00' b'\x00' b'\x00' b'\x00' ... ] as the extended header.

Indeed, it looks like the bug existed before this version. However, previously it only affected files not written using the new_mmap function. I never noticed this because I have been writing mrcfile using new_mmap; however, in the new version of mrcfile the behaviour is consistent but unfortunately consistently broken!

Here is some python code to reproduce the problem:

```
import mrcfile
import numpy as np

def write():

N = 10
header_dtype = mrcfile.dtypes.get_ext_header_dtype(b"FEI1")

data = np.zeros((N, 10, 10), dtype="float16")
header = np.zeros(N, dtype=header_dtype)

handle = mrcfile.new_mmap(
"test_mmap.mrc",
shape=data.shape,
mrc_mode=mrcfile.utils.mode_from_dtype(data.dtype),
overwrite=True,
extended_header=header,
exttyp="FEI1",
)

handle = mrcfile.new("test.mrc", overwrite=True)
handle.set_data(data)
handle.set_extended_header(header)

def read():

print("TEST: test_mmap.mrc")
handle = mrcfile.open("test_mmap.mrc")
print(handle.extended_header.dtype)
print(handle.extended_header)
print("")
print("TEST: test.mrc")
handle = mrcfile.open("test.mrc")
print(handle.extended_header.dtype)
print(handle.extended_header)

if __name__ == '__main__':
write()
read()
```

In version v1.4.3 when reading test_mmap.mrc I get the following output:

TEST: test_mmap.mrc
[('Metadata size', '

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the supplied Python reproduction using mrcfile.dtypes.get_ext_header_dtype, new_mmap, new, set_extended_header, and open; compare how each path writes and reads the FEI1 extended header. Done means both test_mmap.mrc and test.mrc reopen with the structured extended-header dtype and records shown in the v1.4.3 output, rather than a byte array.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.