Archive for the ‘reporting’ Category

Unqualified COUNT(*) speed PBXT vs InnoDB

Май 27th, 2010

So this is about a SELECT COUNT(*) FROM tblname without a WHERE clause. MyISAM has an optimisation for that since it maintains a rowcount for each table. InnoDB and PBXT can’t do that (at least not easily) because of their multi-versioned nature… different transactions may see a different number of rows for the table table!

So, it’s kinda known but nevertheless often ignored that this operation on InnoDB is costly in terms of time; what InnoDB has to do to figure out the exact number of rows is scan the primary key and just tally. Of course it’s faster if it doesn’t have to read a lot of the blocks from disk (i.e. smaller dataset or a large enough buffer pool).

I was curious about PBXT’s performance on this, and behold it appears to be quite a bit faster! For a table with 50 million rows, PBXT took about 20 minutes whereas the same table in InnoDB took 30 minutes. Interesting!

From those numbers you can tell that doing the query at all is not an efficient thing to do, and definitely not something a frontend web page should be doing. Usually you just need a ballpark figure so running the query in a cron job and putting the value into memcached (or just an include file) will work well in such cases.

If you do use a WHERE clause, all engines (including MyISAM) are in the same boat… they might be able to use an index to filter on the conditions – but the bigger the table, the more work it is for the engine. PBXT being faster than InnoDB for this task makes it potentially interesting for reporting purposes as well, where otherwise you might consider using MyISAM – we generally recommend using a separate reporting slave with particular settings anyway (fewer connections but larger session-specific buffers), but it’s good to have extra choices for the task.

(In case you didn’t know, it’s ok for a slave to use a different engine from a master – so you can really make use of that ability for specialised tasks such as reporting.)


PlanetMySQL Voting: Vote UP / Vote DOWN

Kontrollbase – revision 297 fixes Reporter-CLI “alert_22″ sub-routine

Апрель 13th, 2010
Quick note to let our users know that there was an XML tag closure error on the “alert_22″ subroutine in the “bin/kontroll-reporter-cli.pl” script. This does not affect the webapp portion of Kontrollbase – only reports generated via the command line reporter script. It is not a fatal error but will cause the XML file to [...]
PlanetMySQL Voting: Vote UP / Vote DOWN

Kontrollbase – graph “no data to display” on new install has been fixed

Март 25th, 2010
If you have been wondering why the overview and graphs pages say “no data to display” on the graphs when you first install Kontrollbase, it’s because there’s no data in the database being returned from the queries that generate the graphs – this is because a new install has no data to graph. This has [...]
PlanetMySQL Voting: Vote UP / Vote DOWN

Kontrollsoft’s uptime checks now managed by Pingdom

Март 17th, 2010
Website health checks are a crucial service to an operations team. In addition to in-house monitoring and service state reporting it’s also important, even critical, to have an impartial third party to run checks to test your customer facing services. There are a lot of companies in this arean that would be glad to have [...]
PlanetMySQL Voting: Vote UP / Vote DOWN

Advanced reporting options for MySQL

Март 8th, 2010

I’m seeking help from the MySQL community for what tools are used today to generate complex reports for enterprise applications that use MySQL. In an Oracle world, you have Oracle Report Writer, in Microsoft Crystal Reports.

In the open source world there is Jasper Reports, Pentaho Reports and BIRT however I don’t know the power of complex reporting with these.

If anybody has experience using or evaluating these tools please let me know. This may lead to possible work.


PlanetMySQL Voting: Vote UP / Vote DOWN

Kontrollbase – queries to update your max_connections alert

Февраль 26th, 2010
If you have been reading the Kontrollbase performance reports and noticed that one alert says your connection usage vs max connections ration is too high but then recommends you to decrease the max_connections variable, then you will find this fix handy. Its two simple queries that execute on the Kontrollbase schema to update the max_connections [...]
PlanetMySQL Voting: Vote UP / Vote DOWN

Kontrollbase – scripts being rewritten in Python, request improvements now!

Февраль 4th, 2010
The time has come for major performance improvements to the reporter, stats-gather, alerter, and client scripts. This means that I will be rewriting the scripts in Python. A couple of reasons for this; to cut down on the number of modules that are required for the installation process (which also makes distributing the client script [...]
PlanetMySQL Voting: Vote UP / Vote DOWN

Kontrollbase 2.0.1 revision 281 available for download

Январь 22nd, 2010
A new version of Kontrollbase – the enterprise monitoring, analytics, reporting, and historical analysis webapp for MySQL database administrators and advanced users of MySQL databases – is available for download. See the downloads page or run “svn update” to get your new version today. http://kontrollsoft.com/software-downloads
PlanetMySQL Voting: Vote UP / Vote DOWN

Advanced analytics report added to Kontrollbase CLI-reporter

Январь 14th, 2010
It didn’t take long, but the advanced analytics reporting code has been added to the command line reporter version of Kontrollbase. Click here to see an example of the tuning and analytics report output (html format). Now you have all of the reporting features (minus the growth over time) features that are offered in the [...]
PlanetMySQL Voting: Vote UP / Vote DOWN

Kontrollbase revision 270 is available for download

Январь 12th, 2010
The latest version of Kontrollbase – the MySQL analytics and performance tuning application –  is available for download. Since the previous one, version 225, there have been a lot of great changes. The most significant being the development and inclusion of the Reporter CLI script – which is a combination of the client, stats-gather, and [...]
PlanetMySQL Voting: Vote UP / Vote DOWN