Icinga / Icinga/icinga2

Daily one Error: std::bad_alloc

Open
#10,410 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
2.2k
Forks
616
Avg merge
5d 6h
Merged PRs (30d)
10

Description

Describe the bug

A clear and concise description of what the bug is.

I updated to r2.14.5-1 from r2.14.3-1, because of an another bug (JSON). Now the Icinga stops once a day with error. Do you have idea what causes this kind of error:

Caught unhandled exception.
Current time: 2025-04-13 03:04:56 +0200

Application version: r2.14.5-1

System information:
Platform: CentOS Linux
Platform version: 7 (Core)
Kernel: Linux
Kernel version: 3.10.0-1160.90.1.el7.x86_64
Architecture: x86_64

Build information:
Compiler: GNU 11.2.1
Build host: b10faeb05acc
OpenSSL version: OpenSSL 1.0.2k-fips 26 Jan 2017

Application information:

General paths:
Config directory: /etc/icinga2
Data directory: /var/lib/icinga2
Log directory: /var/log/icinga2
Cache directory: /var/cache/icinga2
Spool directory: /var/spool/icinga2
Run directory: /run/icinga2

Old paths (deprecated):
Installation root: /usr
Sysconf directory: /etc
Run directory (base): /run
Local state directory: /var

Internal paths:
Package data directory: /usr/share/icinga2
State path: /var/lib/icinga2/icinga2.state
Modified attributes path: /var/lib/icinga2/modified-attributes.conf
Objects path: /var/cache/icinga2/icinga2.debug
Vars path: /var/cache/icinga2/icinga2.vars
PID path: /run/icinga2/icinga2.pid

Error: std::bad_alloc

Stacktrace:
0# __cxa_throw in /usr/lib64/icinga2/sbin/icinga2
1# operator new(unsigned long) in /lib64/libstdc++.so.6
2# std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) in /lib64/libstdc++.so.6
3# char* std::string::_S_construct<char*>(char*, char*, std::allocator const&, std::forward_iterator_tag) in /lib64/libboost_program_options.so.1.69.0
4# std::basic_stringbuf<char, std::char_traits, std::allocator >::str() const in /lib64/libstdc++.so.6
5# 0x00000000009B7DF0 in /usr/lib64/icinga2/sbin/icinga2
6# icinga::IdoPgsqlConnection::Query(icinga::String const&) in /usr/lib64/icinga2/sbin/icinga2
7# icinga::IdoPgsqlConnection::InternalExecuteQuery(icinga::DbQuery const&, int) in /usr/lib64/icinga2/sbin/icinga2
8# icinga::WorkQueue::RunTaskFunction(std::function<void ()> const&) in /usr/lib64/icinga2/sbin/icinga2
9# icinga::WorkQueue::WorkerThreadProc() in /usr/lib64/icinga2/sbin/icinga2
10# 0x00007F8397A91954 in /lib64/libboost_thread.so.1.69.0
11# 0x00007F8395B89EA5 in /lib64/libpthread.so.0
12# clone in /lib64/libc.so.6


  • This would indicate a runtime problem or configuration error. If you believe this is a bug in Icinga 2
  • please submit a bug report at https://github.com/Icinga/icinga2 and include this stack trace as well as any other
  • information that might be useful in order to reproduce this problem.

This kind of error wasn't before I upgraded.

To Reproduce

Currently I don't know how to reproduce

Expected behavior

Non crashing.

Screenshots

If applicable, add screenshots to help explain your problem.

Your Environment

Include as many relevant details about the environment you experienced the problem in

  • Version used (icinga2 --version):

icinga2 - The Icinga 2 network monitoring daemon (version: r2.14.5-1)

Copyright (c) 2012-2025 Icinga GmbH (https://icinga.com/)
License GPLv2+: GNU GPL version 2 or later https://gnu.org/licenses/gpl2.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

System information:
Platform: CentOS Linux
Platform version: 7 (Core)
Kernel: Linux
Kernel version: 3.10.0-1160.90.1.el7.x86_64
Architecture: x86_64

Build information:
Compiler: GNU 11.2.1
Build host: b10faeb05acc
OpenSSL version: OpenSSL 1.0.2k-fips 26 Jan 2017

Application information:

General paths:
Config directory: /etc/icinga2
Data directory: /var/lib/icinga2
Log directory: /var/log/icinga2
Cache directory: /var/cache/icinga2
Spool directory: /var/spool/icinga2
Run directory: /run/icinga2

Old paths (deprecated):
Installation root: /usr
Sysconf directory: /etc
Run directory (base): /run
Local state directory: /var

Internal paths:
Package data directory: /usr/share/icinga2
State path: /var/lib/icinga2/icinga2.state
Modified attributes path: /var/lib/icinga2/modified-attributes.conf
Objects path: /var/cache/icinga2/icinga2.debug
Vars path: /var/cache/icinga2/icinga2.vars
PID path: /run/icinga2/icinga2.pid

  • Operating System and version:

NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"

  • Enabled features (icinga2 feature list):

Disabled features: compatlog elasticsearch graphite icingadb influxdb2 journald livestatus opentsdb syslog
Enabled features: api checker command debuglog gelf ido-pgsql influxdb mainlog notification perfdata

  • Icinga Web 2 version and modules (System - About):

  • Config validation (icinga2 daemon -C):
    [2025-04-14 08:37:32 +0200] information/ConfigItem: Instantiated 1 NotificationComponent.
    [2025-04-14 08:37:32 +0200] information/ConfigItem: Instantiated 1 IdoPgsqlConnection.
    [2025-04-14 08:37:32 +0200] information/ConfigItem: Instantiated 1 ExternalCommandListener.
    [2025-04-14 08:37:32 +0200] information/ConfigItem: Instantiated 1 CheckerComponent.
    [2025-04-14 08:37:32 +0200] information/ConfigItem: Instantiated 3121 Dependencies.
    [2025-04-14 08:37:32 +0200] information/ConfigItem: Instantiated 16 UserGroups.
    [2025-04-14 08:37:32 +0200] information/ConfigItem: Instantiated 11 TimePeriods.
    [2025-04-14 08:37:32 +0200] information/ConfigItem: Instantiated 21 Users.
    [2025-04-14 08:37:32 +0200] information/ConfigItem: Instantiated 7 ServiceGroups.
    [2025-04-14 08:37:32 +0200] information/ConfigItem: Instantiated 5307 Services.
    [2025-04-14 08:37:32 +0200] information/ConfigItem: Instantiated 293 Zones.
    [2025-04-14 08:37:32 +0200] information/ConfigItem: Instantiated 879 ScheduledDowntimes.
    [2025-04-14 08:37:32 +0200] information/ConfigItem: Instantiated 1 InfluxdbWriter.
    [2025-04-14 08:37:32 +0200] information/ConfigItem: Instantiated 6 NotificationCommands.
    [2025-04-14 08:37:32 +0200] information/ConfigItem: Instantiated 11101 Notifications.
    [2025-04-14 08:37:32 +0200] information/ConfigItem: Instantiated 1 FileLogger.
    [2025-04-14 08:37:32 +0200] information/ConfigItem: Instantiated 1 IcingaApplication.
    [2025-04-14 08:37:32 +0200] information/ConfigItem: Instantiated 239 HostGroups.
    [2025-04-14 08:37:32 +0200] information/ConfigItem: Instantiated 1 EventCommand.
    [2025-04-14 08:37:32 +0200] information/ConfigItem: Instantiated 345 Hosts.
    [2025-04-14 08:37:32 +0200] information/ConfigItem: Instantiated 1674 Downtimes.
    [2025-04-14 08:37:32 +0200] information/ConfigItem: Instantiated 291 Endpoints.
    [2025-04-14 08:37:32 +0200] information/ConfigItem: Instantiated 132 Comments.
    [2025-04-14 08:37:32 +0200] information/ConfigItem: Instantiated 13 ApiUsers.
    [2025-04-14 08:37:32 +0200] information/ConfigItem: Instantiated 1 ApiListener.
    [2025-04-14 08:37:32 +0200] information/ConfigItem: Instantiated 315 CheckCommands.
    [2025-04-14 08:37:32 +0200] information/ConfigItem: Instantiated 1 GelfWriter.
    [2025-04-14 08:37:32 +0200] information/ScriptGlobal: Dumping variables to file '/var/cache/icinga2/icinga2.vars'
    [2025-04-14 08:37:33 +0200] information/cli: Finished validating the configuration file(s).

  • If you run multiple Icinga 2 instances, the zones.conf file (or icinga2 object list --type Endpoint and icinga2 object list --type Zone) from all affected nodes.

Additional context

Same happaned since update:
-rw-r--r--. 1 icinga icinga 2533 Apr 9 08:45 report.1744181155.761729
-rw-r--r--. 1 icinga icinga 2505 Apr 12 04:35 report.1744425335.259736
-rw-r--r--. 1 icinga icinga 2505 Apr 13 03:04 report.1744506296.342264

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the stack trace entry points IdoPgsqlConnection::Query and InternalExecuteQuery, then investigate why query-string construction reaches std::bad_alloc. Review the upgrade-related behavior and available database diagnostics; done means identifying a reproducible cause, preventing the daemon crash, and adding a regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, postgresql
Domain
backend, databases
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.