NordicHPC / NordicHPC/slurm2sql

OverflowError: Python int too large to convert to SQLite INTEGER

Open
#14 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
19
Forks
10
PR merge metrics
No merged PRs in 30d

Description

I came across this when trying to export data from some years.
But its not related to running over a long time period because it fails the same way even when I have history-start the day this issue occurs or days later (and then if fails immediately).
By testing, it seems there is a specific period from 15 November 2021 to 6 December 2021 it fails

Traceback (most recent call last): File "/cluster/home/haagen/slurm/slurm2sql.py", line 798, in <module> exit(main(sys.argv[1:])) File "/cluster/home/haagen/slurm/slurm2sql.py", line 564, in main errors = get_history(db, sacct_filter=sacct_filter, File "/cluster/home/haagen/slurm/slurm2sql.py", line 635, in get_history errors += slurm2sql(db, sacct_filter=new_filter, update=True, jobs_only=jobs_only, File "/cluster/home/haagen/slurm/slurm2sql.py", line 752, in slurm2sql c.execute('INSERT %s INTO slurm (%s) VALUES (%s)'%( OverflowError: Python int too large to convert to SQLite INTEGER

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The failure occurs in slurm2sql.py at line 752 during the SQLite INSERT reached through get_history and slurm2sql. Reproduce an export covering 15 November to 6 December 2021 and inspect which inserted value exceeds SQLite INTEGER limits. Done means that this period exports without the reported OverflowError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, sqlite
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.