Saturday, December 11, 2010

Cacti fixed. Ooops. :)

I finally tracked down what happen.  This is the problem with changing too many things at the same time.  I looked over the permissions for the cacti user in the MySQL database and decided that it didn't need the full permissions, so I removed the ability to create temporary tables and some other things.  I think normally Cacti doesn't use temporary tables, but I added the Boost plugin and it DOES need that.

The SNMP results get put in the database and written to disk later.  To do this, Boost uses temporary tables.  Since it was unable to flush the table, the database kept filling up.  It got up to 8 million rows before I figured out how to fix it.

I also learned that you can't "backfill" data into an RRD file, at least not using the poller.php.  When I was finally able to flush the data out of the boost table, I lost a lot of the data because of the feature that writes data to the RRD file as someone requests the graph.  Once I started checking all the graphs, I pretty much locked them out of receiving old updates that were stored in the table.

I really need to make some kind of alert for the poller tables in Cacti.

0 comments: