MetPX / MetPX/Sundew

crash when a file disappears...

Open
#18 1 comment 0 reactions 1 assignee View on GitHub

@petersilva is already working on this.

Since Nov 8, 2022.

Dominant language
Python
Stars
6
Forks
3
PR merge metrics
No merged PRs in 30d

Description

```
2022-11-05 23:56:23,627 [INFO] 2 files will be ingested
2022-11-05 23:56:23,631 [INFO] (944 Bytes) Ingested in DB as /var/spool/px/db/20221105/WXAPP-HOURLY_OBS/DMS/CMC_dms-op1-host05/2400633.yte.hourly_obs.en.json:DMS:CMC_dms-op1-host05:WXAPP-HOURLY_OBS:JSON::20221105235623
2022-11-05 23:56:23,631 [INFO] Queued for: wxo-ops-be02-wxapp wxo-ops-be01-wxapp ddsr-MSC-DMS-OP-PDS wxo-op-be06-wxapp wxo-op-be07-wxapp pxi-dms wxo-op-be05-wxapp
2022-11-05 23:56:23,635 [INFO] (947 Bytes) Ingested in DB as /var/spool/px/db/20221105/WXAPP-HOURLY_OBS/DMS/CMC_dms-op1-host05/2400633.yte.hourly_obs.fr.json:DMS:CMC_dms-op1-host05:WXAPP-HOURLY_OBS:JSON::20221105235623
2022-11-05 23:56:23,635 [INFO] Queued for: wxo-ops-be02-wxapp wxo-ops-be01-wxapp ddsr-MSC-DMS-OP-PDS wxo-op-be06-wxapp wxo-op-be07-wxapp pxi-dms wxo-op-be05-wxapp
Traceback (most recent call last):
File "/usr/bin/pxReceiver", line 150, in
source.ingestor.ingestSingleFile(igniter)
File "/usr/lib/px/Ingestor.py", line 376, in ingestSingleFile
reader.read()
File "/usr/lib/px/DiskReader.py", line 101, in read
self.sort()
File "/usr/lib/px/DiskReader.py", line 303, in sort
templist = [ (os.stat(file)[ST_MTIME],file) for file in self.files]
OSError: [Errno 2] No such file or directory: '/var/spool/px/rxq/dms-v2-products/kjfk.202211052351.hourly_avipads.xml:DMS:CMC_dms-op1-host02:AVIPADS-HOURLYOBS:XML'
Traceback (most recent call last):
File "/usr/bin/pxReceiver", line 130, in
igniter = PXIgniter('receiver', sourceName, command, lockPath, logger)
File "/usr/lib/px/PXIgniter.py", line 37, in __init__
eval("self." + cmd)() # Execute the command directly
File "/usr/lib/px/Igniter.py", line 136, in restart
self.stop()
File "/usr/lib/px/PXIgniter.py", line 80, in stop
os.kill(self.lockpid, signal.SIGTERM)
OSError: [Errno 1] Operation not permitted
Traceback (most recent call last):
File "/usr/bin/pxReceiver", line 150, in
source.ingestor.ingestSingleFile(igniter)
File "/usr/lib/px/Ingestor.py", line 376, in ingestSingleFile
reader.read()
File "/usr/lib/px/DiskReader.py", line 101, in read
self.sort()
File "/usr/lib/px/DiskReader.py", line 303, in sort
templist = [ (os.stat(file)[ST_MTIME],file) for file in self.files]
OSError: [Errno 2] No such file or directory: '/var/spool/px/rxq/dms-v2-products/1060r0k.wwl.hourly_obs.en.json:DMS:CMC_dms-op1-host05:WXAPP-HOURLY_OBS:JSON'
```

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.