PlanetMysql.ru — информация о СУБД MySQL

In order to benchmark a Drupal site performance you need to see all the database queries related to your Drupal site. In case you don’t have access to the my.cnf file If you don’t have access to the my.cnf file, you can log the queries from the Drupal code itself: cp includes/database.mysql-common.inc includes/database.mysql-common.inc.backup.orig vim includes/database.mysql-common.inc [...]
PlanetMySQL Voting: Vote UP / Vote DOWN

As part of my upcoming FREE my.cnf check advice I first need to ask people to provide the current MySQL configuration file commonly found as a file named my.cnf

If only that question was easy to answer!

Use of configuration files

MySQL will by default use at least one configuration file from the following defaults. MySQL also uses a cascade approach for configuration files. When you have multiple files in the appropriate paths you can see unexpected behavior when you override certain values in different files.

You can however for example specify –no-defaults to use no configuration file, or add options to your command line execution, so even looking at all configuration files is no guarantee of your operating configuration.

However for most environments, these complexities do not exist.

Default Location

By default and on single instance MySQL servers you are most likely to find this file called my.cnf and found at:

• /etc/my.cnf
• /etc/mysql/my.cnf

These are known as the global options files.

Alternative Locations

MySQL has both instance specific and user specific locations. For the inclusion of an instance specific file, the location is:

• $MYSQL_HOME/my.cnf

where MYSQL_HOME is a defined environment variable. Historical MySQL versions also looked at [datadir]/my.cnf however I am unaware if this is applicable in 5.x versions.

You can also specific options on a per user basis for default inclusion. These are found at:

• $HOME/.my.cnf

Distro specific locations

Ubuntu for example also provides an ability to add options via an include directory.

Specifying a configuration at runtime

While you may have these default files, you may elect to start mysql with a specific configuration file as specified by –defaults-file. This option will override all global/instance/user locations and use just this configuration file. You can also specify additional configuration that supplements and not overrides the default with –defaults-extra-file.

What files are on my system?

Again, assuming the default names you can perform a brute force check with:

$ sudo find / -name "*my*cnf"

This is actually worthwhile, especially if you find a /root/.my.cnf file which is default MySQL settings for the Operating System ‘root’ user.

MySQL recommendations

MySQL by default provides a number of recommended files however these are generally outdated especially for newer hardware. These files include my-huge.cnf, my-large.cnf, my-medium.cnf, my-small.cnf and my-innodb-heavy-4G.cnf. Don’t assume replacing your configuration with one of these files will make your system perform better.

MySQL made some attempt to correct these and at least some very poor defaults with MySQL 5.4 however I am unsure what’s in MySQL 5.5

MySQL Configuration at runtime

While several commands can help with identifying your configuration files and print defaults etc, it’s also possible to change your configuration at runtime. It’s possible that these changes are not reflected in your configuration files and pose an additional mismatch.

References

• Using Option Files
• Server Command Options

PlanetMySQL Voting: Vote UP / Vote DOWN

Today, while on IRC in #pentaho I came across a discussion and a published my.cnf. In this configuration I found some grossly incorrect values for per session buffers (see below).

It doesn’t take a MySQL expert to spot the issues, however there is plenty of bad information available on the Internet and developers not knowing MySQL well can easily be mislead. This has spurred me to create a program to rid the world of bad MySQL configuration. While my task is potential infinite, it will enable me to give back and hopefully do a small amount of good. You never know, saving those CPU cycles may save energy and help the planet.

Stay tuned for more details of my program.

[mysqld]
...
sort_buffer_size = 6144K
myisam_sort_buffer_size = 1G
join_buffer_size = 1G
bulk_insert_buffer_size = 1G
read_buffer_size     = 6144K
read_rnd_buffer_size = 6144K
key_buffer_size		= 1024M
max_allowed_packet	= 32M
thread_stack		= 192K
thread_cache_size       = 256

query_cache_limit	= 512M
query_cache_size        = 512M
...

PlanetMySQL Voting: Vote UP / Vote DOWN

Ronald Bradford’s recent warning to be sure to know your my.cnf sections reminded me of a similar issue that I ran into last summer, where putting the “group” option in both the [mysqld_safe] and [mysqld] directives resulted in a mostly silent problem.

I started noticing this in MySQL 5.1 and it affected both the official MySQL binary and the Percona binary. In trying to be conscientious, I had the following set:

[mysqld_safe]
user=mysql
group=mysql

[mysqld]
user=mysql
group=mysql


However, when the MySQL server started up, the error log showed

[Warning] option 'group_concat_max_len': unsigned value 0 adjusted to 4

This was obviously a problem, but I only started noticing it during MySQL restarts, which was mostly during upgrades to MySQL 5.1. I tracked it down and realized that when I removed the “group” option from the [mysqld] directive, the warning did not come up.

The problem is that [safe_mysqld] sees “group” as the “group” option, but [mysqld] does not know about the “group” option. The MySQL server allows the shortest unique identifier of an option to *be* that option. Thus, “group” is an acceptable abbreviation for “group_concat_max_len”.

So mysqld was taking:
group=mysql

and translating it to:
group_concat_max_len=mysql

but “mysql” is a string, not a number, so MySQL tried to be helpful by converting to a number….so it was as if I stated:
group_concat_max_len=0

I filed a bug for this back in June:
http://bugs.mysql.com/bug.php?id=45379. The response was “If 3 different people ask about removing this feature reclassifying report to feature request with new synopsis.”

So, a second moral: make a bug report if you want things to get changed, and if you see a bug report for a problem you’re encountering, make sure to add your voice so that MySQL understands that an issue is indeed serious.

PlanetMySQL Voting: Vote UP / Vote DOWN

Ronald Bradford’s recent warning to be sure to know your my.cnf sections reminded me of a similar issue that I ran into last summer, where putting the “group” option in both the [mysqld_safe] and [mysqld] directives resulted in a mostly silent problem.

I started noticing this in MySQL 5.1 and it affected both the official MySQL binary and the Percona binary. In trying to be conscientious, I had the following set:

[mysqld_safe]
user=mysql
group=mysql

[mysqld]
user=mysql
group=mysql


However, when the MySQL server started up, the error log showed

[Warning] option 'group_concat_max_len': unsigned value 0 adjusted to 4

This was obviously a problem, but I only started noticing it during MySQL restarts, which was mostly during upgrades to MySQL 5.1. I tracked it down and realized that when I removed the “group” option from the [mysqld] directive, the warning did not come up.

The problem is that [safe_mysqld] sees “group” as the “group” option, but [mysqld] does not know about the “group” option. The MySQL server allows the shortest unique identifier of an option to *be* that option. Thus, “group” is an acceptable abbreviation for “group_concat_max_len”.

So mysqld was taking:
group=mysql

and translating it to:
group_concat_max_len=mysql

but “mysql” is a string, not a number, so MySQL tried to be helpful by converting to a number….so it was as if I stated:
group_concat_max_len=0

I filed a bug for this back in June:
http://bugs.mysql.com/bug.php?id=45379. The response was “If 3 different people ask about removing this feature reclassifying report to feature request with new synopsis.”

So, a second moral: make a bug report if you want things to get changed, and if you see a bug report for a problem you’re encountering, make sure to add your voice so that MySQL understands that an issue is indeed serious.

PlanetMySQL Voting: Vote UP / Vote DOWN

The MySQL configuration file, e.g. /etc/my.cnf has a number of different section headings including [mysql], [mysqld], [mysqld_safe]. It is important that you ensure you put the right variables into the right section. For example, the following my.cnf configuration file will not operate as the user probably expects.

[mysqld]
...
log-bin=mysql-bin
server-id=1
query_cache_size = 100M
query_cache_type = 1

...

[mysqld_safe]
...
key_buffer_size=600M
skip-innodb
...

In this example, this configuration does not give you a MyISAM key buffer of 600M, it’s actually the default of 8M.

mysql> show global variables like 'key_buffer_size';
+-----------------+---------+
| Variable_name   | Value   |
+-----------------+---------+
| key_buffer_size | 8388600 |
+-----------------+---------+

Be sure to add the right options to the [mysqld] section.

What I didn’t know until yesterday was that some programs read from multiple groups. From the 5.1.2. Server Command Options MySQL reference manual page. In helping the describe the problem for the benefit of readers I actually learned something new myself.

mysqld reads options from the [mysqld] and [server] groups. mysqld_safe reads options from the [mysqld], [server], [mysqld_safe], and [safe_mysqld] groups. mysql.server reads options from the [mysqld] and [mysql.server] groups.

I have for example always put log-error in both the [mysqld_safe] and [mysql]d sections because both of these write different errors. Seems that is unnecessary.

PlanetMySQL Voting: Vote UP / Vote DOWN

The MySQL configuration file, e.g. /etc/my.cnf has a number of different section headings including [mysql], [mysqld], [mysqld_safe]. It is important that you ensure you put the right variables into the right section. For example, the following my.cnf configuration file will not operate as the user probably expects.

[mysqld]
...
log-bin=mysql-bin
server-id=1
query_cache_size = 100M
query_cache_type = 1

...

[mysqld_safe]
...
key_buffer_size=600M
skip-innodb
...

In this example, this configuration does not give you a MyISAM key buffer of 600M, it’s actually the default of 8M.

mysql> show global variables like 'key_buffer_size';
+-----------------+---------+
| Variable_name   | Value   |
+-----------------+---------+
| key_buffer_size | 8388600 |
+-----------------+---------+

Be sure to add the right options to the [mysqld] section.

What I didn’t know until yesterday was that some programs read from multiple groups. From the 5.1.2. Server Command Options MySQL reference manual page. In helping the describe the problem for the benefit of readers I actually learned something new myself.

mysqld reads options from the [mysqld] and [server] groups. mysqld_safe reads options from the [mysqld], [server], [mysqld_safe], and [safe_mysqld] groups. mysql.server reads options from the [mysqld] and [mysql.server] groups.

I have for example always put log-error in both the [mysqld_safe] and [mysql]d sections because both of these write different errors. Seems that is unnecessary.

PlanetMySQL Voting: Vote UP / Vote DOWN

Many configs have thread_stack_size configured explicitly, but that can cause rather bad trouble:

• if the stack inside a thread it’s too small, you can get segfault crashes (stack overflow, essentially). Particularly on 64-bit.
• if the stack is too large, your system cannot handle as many connections since it all eats RAM.

Let mysqld sort it out, on startup it does a calculation based on the CPU architecture, and that’s actually the most sensible. So for almost all setups, remove any thread_stack_size=… line you might have in my.cnf.

PlanetMySQL Voting: Vote UP / Vote DOWN

As a consultant I would be rich if I made money every time when asking “Have you checked the MySQL error log?”

Today’s special found in a 13GB MySQL server error log.

090819 22:49:37InnoDB: Warning: difficult to find free blocks from
InnoDB: the buffer pool (1101071 search iterations)! Consider
InnoDB: increasing the buffer pool size.
InnoDB: It is also possible that in your Unix version
InnoDB: fsync is very slow, or completely frozen inside
InnoDB: the OS kernel. Then upgrading to a newer version
InnoDB: of your operating system may help. Look at the
InnoDB: number of fsyncs in diagnostic info below.
InnoDB: Pending flushes (fsync) log: 0; buffer pool: 0
InnoDB: 167 OS file reads, 1 OS file writes, 1 OS fsyncs
InnoDB: Starting InnoDB Monitor to print further
InnoDB: diagnostics to the standard output.
090819 22:49:37InnoDB: Warning: difficult to find free blocks from
InnoDB: the buffer pool (1101051 search iterations)! Consider
InnoDB: increasing the buffer pool size.
InnoDB: It is also possible that in your Unix version
InnoDB: fsync is very slow, or completely frozen inside
InnoDB: the OS kernel. Then upgrading to a newer version
InnoDB: of your operating system may help. Look at the
InnoDB: number of fsyncs in diagnostic info below.
InnoDB: Pending flushes (fsync) log: 0; buffer pool: 0
InnoDB: 167 OS file reads, 1 OS file writes, 1 OS fsyncs
InnoDB: Starting InnoDB Monitor to print further
InnoDB: diagnostics to the standard output.
090819 22:49:37InnoDB: Warning: difficult to find free blocks from
InnoDB: the buffer pool (1101072 search iterations)! Consider
InnoDB: increasing the buffer pool size.
InnoDB: It is also possible that in your Unix version
InnoDB: fsync is very slow, or completely frozen inside
InnoDB: the OS kernel. Then upgrading to a newer version
InnoDB: of your operating system may help. Look at the
InnoDB: number of fsyncs in diagnostic info below.
InnoDB: Pending flushes (fsync) log: 0; buffer pool: 0
InnoDB: 167 OS file reads, 1 OS file writes, 1 OS fsyncs
InnoDB: Starting InnoDB Monitor to print further
InnoDB: diagnostics to the standard output.

As you can see the same error is occuring, and in this example 3 times in the last second. To find the cause of the error I didn’t have to look far because I had already checked the /etc/my.cnf file.

$ cat /etc/my.cnf
[mysqld]
set-variable = max_connections=500
innodb_buffer_pool_size = 500
safe-show-database

Where do people come up with these my.cnf files? A decision was made to create one, and not use either no file or at least the default that was provided with the installation that is still on the system.

$ find / -name *my*cnf
/home/dontcold/.my.cnf
/etc/my.cnf
/root/.my.cnf
/usr/local/cpanel/whostmgr/my.cnf
/usr/share/doc/MySQL-server-5.0.77/my-small.cnf
/usr/share/doc/MySQL-server-5.0.77/my-medium.cnf
/usr/share/doc/MySQL-server-5.0.77/my-innodb-heavy-4G.cnf
/usr/share/doc/MySQL-server-5.0.77/my-large.cnf
/usr/share/doc/MySQL-server-5.0.77/my-huge.cnf
/usr/share/mysql/my-small.cnf
/usr/share/mysql/my-medium.cnf
/usr/share/mysql/my-innodb-heavy-4G.cnf
/usr/share/mysql/my-large.cnf
/usr/share/mysql/my-huge.cnf

What is interesting is that I’ve seen nearly the same file on a previous installation and I documented in For MySQL DBA fame and glory. Prize included.

PlanetMySQL Voting: Vote UP / Vote DOWN