centreon-engine slower than ndo ?

[romainc11]

Hi,
I wanted to switch from ndo to centreon broker to improve performance.
I will stick with nagios as it would be very hard to changes the many pollers on my architecture.

Before doing so, I benched centreon-broker vs ndo on a poller in a KVM virtual machine (hosting mysql, centreon and the poller, with 5000 local dummy services).
But centreon-broker seems slower on my bench, see this image (concerning services latencies) :

http://imageshack.us/photo/my-images...mparaison.png/

1) Cold start with centreon-broker
2) Hot swap from centreon-broker to ndo (restarting poller obviously)
3) Cold start with centreon-broker
4) Hot swap from ndo to centreon-broker
5) Hot swap from centreon-broker to ndo (where latency drops before going up again)

Why is so ?
Can I make things better, better than ndo ?

Thanks

[DonKiShoot]

Thanks for your benchs.
I will go back here for Centreon team reply

[mkermagoret]

Hi,
Before doing so, I benched centreon-broker vs ndo on a poller in a vwmare virtual machine (hosting mysql, centreon and the poller, with 5000 local dummy services).
But centreon-broker seems slower on my bench, see this image (concerning services latencies) :

Hi,

Thanks for testing Centreon Broker and for giving us the results of your tests. Until today, we've not experienced latency issues as you describe. Can you provide more information on the benchmarks you ran ? Especially VM sizing, Nagios, Broker and MySQL configuration files. If you do not want to make them public you can send them to mkermagoret@merethis.com . We'll look at it and tell you what gone wrong.

Best regards,

[romainc11]

Here are the infos :

Physical machine : Intel xeon x5670 @ 2,93 Ghz 6 cores (12 logical cores):
Linux 2.6.18-308.1.1.el5, 16 Go of ram, 16 Go of swap
Virtual Machine : Install of ces standard, 2 Go of ram, 2 Go of swap

Packages version :
nagios-3.2.3-1
nagios-core-3.2.3-1

centreon-broker-2.0.2-1
centreon-broker-cbmod-2.0.2-1
centreon-broker-storage-2.0.2-1
centreon-broker-core-2.0.2-1
centreon-broker-cbd-2.0.2-1

centreon-2.3.4-7
centreon-database-2.3.4-7
centreon-clapi-1.2-4

Centreon-broker configuration :
http://dl.dropbox.com/u/25195348/cen...r-conf.tar.bz2

Nagios configuration :
http://dl.dropbox.com/u/25195348/cen...s-conf.tar.bz2

MySQL configuration (ces default) :

[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
# Default to using old password format for compatibility with mysql 3.x
# clients (those using the mysqlclient10 compatibility package).
old_passwords=1

# Disabling symbolic-links is recommended to prevent assorted security risks;
# to do so, uncomment this line:
# symbolic-links=0

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

##################

Thanks

[romainc11]

Does anyone have an answer ?
Meanwhile we managed to improve drastically ndo performances by using MySQL MEMORY engine on some ndo tables (the ones that are actually completely refilled at each poller (re)start).

[mkermagoret]

Does anyone have an answer ?

Indeed the performance penalty you encounter comes from a feature of Centreon Broker : the asynchronous event processing. This feature allows Centreon Broker to process events after they've been generated where NDOUtils processes them on the fly. Asynchronous processing usually prevent Nagios from being slowed by the event broker.

Asynchronous event processing requires to copy each generated event. This performance hit is usually unnoticeable, as the scheduling of Nagios is not a big part of the machine load. However when the share of load of the scheduling process becomes important, as in your platform (average latency of 50s), the event copy slows Nagios down.

To make things better I would recommend better platform sizing so that average check latency without any broker modules gets close to 0.

Best regards.