Archive for the ‘xtradb’ Category

Building XtraDB on Windows

Февраль 6th, 2012

As you may or may not know, Windows is not yet a supported platform for XtraDB.

I thought I’d try to build it, and see what happens:

  1. Download XtraDB 5.5 Source
  2. cd C:\xtradb-5.5
  3. mkdir bld
  4. cd bld
  5. cmake ..
  6. VS08: File -> Open -> Solution -> C:\xtradb-5.5\bld\MySQL.sln

Build Ended With:

========== Build: 70 succeeded, 17 failed, 2 up-to-date, 10 skipped ==========

The first failure had to do with innobase:

18>Generating Code...
18>Build log was saved at "file://c:\..\innobase.dir\Debug\BuildLog.htm"
18>innobase - 9 error(s), 3 warning(s)

I checked the innobase build log and found this:

sql_prepare.cc
..\sql_prepare.cc(2199) : error C3861: 'gettimeofday': identifier not found
..\sql_prepare.cc(2232) : error C3861: 'gettimeofday': identifier not found
..\sql_prepare.cc(2639) : error C3861: 'gettimeofday': identifier not found
..\sql_prepare.cc(2672) : error C3861: 'gettimeofday': identifier not found
..\sql_prepare.cc(2808) : error C3861: 'gettimeofday': identifier not found
..\sql_prepare.cc(2848) : error C3861: 'gettimeofday': identifier not found
..\sql_prepare.cc(2940) : error C3861: 'gettimeofday': identifier not found
..\sql_prepare.cc(2973) : error C3861: 'gettimeofday': identifier not found
...
sql_parse.cc
..\sql_parse.cc(5750) : error C3861: 'gettimeofday': identifier not found
..\sql_parse.cc(5827) : error C3861: 'gettimeofday': identifier not found

The ‘gettimeofday’ function is a Unix-only (i.e., there is no direct analog of the gettimeofday() in Windows) .. more details here.

So, this part of the code has simply not been ported to Windows yet.

I got to looking, and it is a reported bug already:

https://bugs.launchpad.net/percona-server/+bug/737895
https://bugs.launchpad.net/percona-patches/+bug/421925
https://bugs.launchpad.net/percona-patches/+bug/390156

But, do not worry. If you want to take advantage of the InnoDB enhancements found in XtraDB on Windows, you can simply use MariaDB.

There are pre-built MariaDB binaries for Windows, and MariaDB contains XtraDB+ (which is XtraDB plus some additional enhancements on top of it).

Hope this helps.


PlanetMySQL Voting: Vote UP / Vote DOWN

Building MariaDB 5.1 on Windows

Январь 23rd, 2012

Recently, I found myself needing MariaDB 5.1.60 for Windows for some testing purposes. Therefore, I needed to build it from source. I ended up using what I’d call a “blend” of the commands listed in this “how-to” and the readme file INSTALL-WIN-SOURCE, so I thought I’d post those steps.

  1. Download 5.1.60 MariaDB source from here.
  2. cd C:\mariadb-5.1
  3. win\configure.js
  4. cmake .
  5. VS: File -> Open -> Solution -> MySql.sln
  6. VS: Build -> Build Solution
  7. VS: Right-click “PACKAGE” -> Build (in “Solution Explorer” View)

That’s it.

Let’s fire it up:

MariaDB> select version();
+----------------------+
| version()            |
+----------------------+
| 5.1.60-MariaDB-debug |
+----------------------+

MariaDB> show global variables like 'innodb_version';
+----------------+-------------+
| Variable_name  | Value       |
+----------------+-------------+
| innodb_version | 1.0.17-13.0 |
+----------------+-------------+

MariaDB> show engines;
+------------+---------+---------------------------...
| Engine     | Support | Comment		   ...
+------------+---------+---------------------------...
| CSV        | YES     | CSV storage engine	   ...
| InnoDB     | DEFAULT | Percona-XtraDB, Supports t...
| PBXT       | YES     | High performance, multi-ve...
| MARIA      | YES     | Crash-safe tables with MyI...
| MyISAM     | YES     | Default engine as of MySQL...
| FEDERATED  | YES     | FederatedX pluggable stora...
| MRG_MYISAM | YES     | Collection of identical My...
| MEMORY     | YES     | Hash based, stored in memo...
+------------+---------+---------------------------...

..

For reference, here are my full outputs:

C:\Users\Chris>cd ..\..\mariadb-5.1

C:\mariadb-5.1>win\configure.js

C:\mariadb-5.1>cmake .
-- Check for working C compiler: cl
-- Check for working C compiler: cl -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: cl
-- Check for working CXX compiler: cl -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
build CSV as static library (libcsv.lib)
build FEDERATEDX as static library (libfederatedx.lib)
build HEAP as static library (libheap_s.lib)
build MARIA as static library (libmaria_s.lib)
build MYISAM as static library (libmyisam_s.lib)
build MYISAMMRG as static library (libmyisammrg_s.lib)
build PBXT as static library (libpbxt_s.lib)
build XTRADB as static library (libxtradb.lib)
build ARCHIVE as DLL (ha_archive.dll)
build BLACKHOLE as DLL (ha_blackhole.dll)
build EXAMPLE as DLL (ha_example.dll)
build FEDERATED as DLL (ha_federated.dll)
build INNODB_PLUGIN as DLL (ha_innodb_plugin.dll)
-- Configuring done
-- Generating done
-- Build files have been written to: C:/mariadb-5.1

Open Visual Studio -> File -> Open -> Project/Solution -> Select C:\mariadb-5.1.60\MySql.sln

Build Solution Output:

========== Build: 79 succeeded, 0 failed, 2 up-to-date, 2 skipped ==========

Package Build Output:

1>------ Build started: Project: PACKAGE, Configuration: Debug Win32 ------
1>
1>Performing Post-Build Event...
1>CPack: Create package using NSIS
1>CPack: Install projects
1>CPack: - Install project: MySql
1>CPack: -   Install component: Unspecified
1>CPack: -   Install component: headers
1>CPack: -   Install component: mysqltest
1>CPack: -   Install component: runtime
1>CPack: -   Install component: scripts
1>CPack: -   Install component: sqlbench
1>CPack: Compress package
1>CPack: Finalize package
1>CPack: Package C:/mariadb-5.1/MariaDB-5.1.60-win32.exe generated.
1>Build log was saved at "file://c:\mariadb-5.1\PACKAGE.dir\Debug\BuildLog.htm"
1>PACKAGE - 0 error(s), 0 warning(s)
========== Build: 1 succeeded, 0 failed, 81 up-to-date, 0 skipped ==========

 
 


PlanetMySQL Voting: Vote UP / Vote DOWN

InnoDB Plugin Version History

Декабрь 1st, 2011

I’m often wondering what version of the InnoDB Plugin is included with which version of MySQL (or MariaDB). The MySQL changelogs used to denote which version of the InnoDB plugin was included with that particular release of MySQL, but sadly this is no longer the case.

Therefore I’ve compiled a comprehensive list which contains all of this info, and then some (and note all InnoDB Plugin changelog links are provided at the bottom).

Hope you find it helpful :)

MySQL 5.5:

MySQL     Plugin     Status Date
5.5.18 1.1.8 GA 11/16/2011
5.5.17 1.1.8 GA 10/19/2011
5.5.16 1.1.8 GA 09/15/2011
5.5.15 1.1.8 GA 07/28/2011
5.5.14 1.1.8 GA 07/05/2011
5.5.13 1.1.7 GA 05/31/2011
5.5.12 1.1.6 GA 05/05/2011
5.5.11 1.1.6 GA 04/07/2011
5.5.10 1.1.5 GA 03/15/2011
5.5.9 1.1.5 GA 02/07/2011
5.5.8 1.1.4 GA 12/03/2010
5.5.7 1.1.3 GA 10/14/2010
5.5.6 1.1.2 GA 09/13/2010
5.5.5 1.1.1 Early Adopter    06/06/2010
5.5.4 1.1.0 Beta 04/09/2010
5.5.3 1.0.6 RC 03/24/2010
5.5.2 1.0.6 RC 02/12/2010
5.5.1 1.0.6 RC 01/04/2010
5.5.0 1.0.5 RC 12/07/2009

MySQL 5.1:

MySQL     Plugin     Status     Date
5.1.60 1.0.17 GA 11/16/2011
5.1.59 1.0.17 GA 09/15/2011
5.1.58 1.0.17 GA 07/05/2011
5.1.57 1.0.16 GA 05/05/2011
5.1.56 1.0.15 GA 03/01/2011
5.1.55 1.0.15 GA 02/07/2011
5.1.54 1.0.14 GA 11/26/2010
5.1.53 1.0.13 GA 11/03/2010
5.1.52 1.0.13 GA 10/11/2010
5.1.51 1.0.12 GA 09/10/2010
5.1.50 1.0.11 GA 08/03/2010
5.1.49 1.0.10 GA 07/09/2010
5.1.48 1.0.9 GA 06/02/2010
5.1.47 1.0.8 GA 05/06/2010
5.1.46 1.0.7 GA 04/06/2010
5.1.45 1.0.6 RC 03/01/2010
5.1.44 1.0.6 RC 02/04/2010
5.1.43 1.0.6 RC 01/15/2010
5.1.42 1.0.6 RC 12/15/2009
5.1.41 1.0.5 RC 11/05/2009
5.1.40 1.0.4 Beta 10/06/2009
5.1.39 1.0.4 Beta 09/04/2009
5.1.38 1.0.4 Beta 09/01/2009
x.x.xx 1.0.3 Alpha 03/11/2009
x.x.xx 1.0.2 Alpha 12/01/2008
x.x.xx 1.0.1 Alpha 05/08/2008
x.x.xx 1.0.0 Alpha 04/15/2008

MariaDB:

MariaDB     Plugin     Status   Date Comments
5.1.55 1.0.15 GA 03/01/11 If use InnoDB Plugin
5.1.55 1.0.15-12.5   GA 03/01/11 If use built-in InnoDB, which is XtraDB in MariaDB
5.2.9 1.0.15-12.7 GA 09/22/11  
5.3.2 1.0.15-12.7 GA 10/14/11  

 
Changelogs:

Note: The plugin versions had their own changelogs initially, but eventually the changes were just mixed into the general changelogs for the MySQL Server.

Note: Also, some plugin versions span multiple MySQL versions, so those have multiple links below.

Plugin 1.0:

1.0.0	http://dev.mysql.com/doc/innodb-plugin/1.0/en/innodb-changes-100.html
1.0.1	http://dev.mysql.com/doc/innodb-plugin/1.0/en/innodb-changes-101.html
1.0.2	http://dev.mysql.com/doc/innodb-plugin/1.0/en/innodb-changes-102.html
1.0.3	http://dev.mysql.com/doc/innodb-plugin/1.0/en/innodb-changes-103.html
1.0.4	http://dev.mysql.com/doc/innodb-plugin/1.0/en/innodb-changes-104.html
1.0.5	http://dev.mysql.com/doc/innodb-plugin/1.0/en/innodb-changes-105.html
1.0.6	http://dev.mysql.com/doc/innodb-plugin/1.0/en/innodb-changes-106.html
1.0.7	http://dev.mysql.com/doc/innodb-plugin/1.0/en/innodb-changes-107.html
1.0.8	http://dev.mysql.com/doc/innodb-plugin/1.0/en/innodb-changes-108.html
1.0.9	http://dev.mysql.com/doc/refman/5.1/en/news-5-1-48.html
1.0.10	http://dev.mysql.com/doc/refman/5.1/en/news-5-1-49.html
1.0.11	http://dev.mysql.com/doc/refman/5.1/en/news-5-1-50.html
1.0.12	http://dev.mysql.com/doc/refman/5.1/en/news-5-1-51.html
1.0.13	http://dev.mysql.com/doc/refman/5.1/en/news-5-1-52.html
	http://dev.mysql.com/doc/refman/5.1/en/news-5-1-53.html
1.0.14	http://dev.mysql.com/doc/refman/5.1/en/news-5-1-54.html
1.0.15	http://dev.mysql.com/doc/refman/5.1/en/news-5-1-55.html
	http://dev.mysql.com/doc/refman/5.1/en/news-5-1-56.html
1.0.16	http://dev.mysql.com/doc/refman/5.1/en/news-5-1-57.html
1.0.17	http://dev.mysql.com/doc/refman/5.1/en/news-5-1-58.html
	http://dev.mysql.com/doc/refman/5.1/en/news-5-1-59.html
	http://dev.mysql.com/doc/refman/5.1/en/news-5-1-60.html

Plugin 1.1:

1.1.0	http://dev.mysql.com/doc/innodb/1.1/en/innodb-changes-11.html
1.1.1	http://dev.mysql.com/doc/refman/5.5/en/news-5-5-5.html
1.1.2	http://dev.mysql.com/doc/refman/5.5/en/news-5-5-6.html
1.1.3	http://dev.mysql.com/doc/refman/5.5/en/news-5-5-7.html
1.1.4	http://dev.mysql.com/doc/refman/5.5/en/news-5-5-8.html
1.1.5	http://dev.mysql.com/doc/refman/5.5/en/news-5-5-9.html
	http://dev.mysql.com/doc/refman/5.5/en/news-5-5-10.html
1.1.6	http://dev.mysql.com/doc/refman/5.5/en/news-5-5-11.html
	http://dev.mysql.com/doc/refman/5.5/en/news-5-5-12.html
1.1.7	http://dev.mysql.com/doc/refman/5.5/en/news-5-5-13.html
1.1.8	http://dev.mysql.com/doc/refman/5.5/en/news-5-5-14.html
	http://dev.mysql.com/doc/refman/5.5/en/news-5-5-15.html
	http://dev.mysql.com/doc/refman/5.5/en/news-5-5-16.html
	http://dev.mysql.com/doc/refman/5.5/en/news-5-5-17.html
	http://dev.mysql.com/doc/refman/5.5/en/news-5-5-18.html

Misc. Notes:

5.1.38 is first release of the plugin

http://dev.mysql.com/doc/refman/5.1/en/news-5-1-38.html

And for reference, here is the text from that announcement regarding the InnoDB Plugin:

C.1.1. Changes in MySQL 5.1.38

  As of MySQL 5.1.38, the InnoDB Plugin is included in MySQL
  releases, in addition to the built-in version of InnoDB that
  has been included in previous releases. This version of the
  InnoDB Plugin is 1.0.4 and is considered of Beta quality.

  The InnoDB Plugin offers new features, improved performance
  and scalability, enhanced reliability and new capabilities
  for flexibility and ease of use. Among the features of the
  InnoDB Plugin are "Fast index creation," table and index
  compression, file format management, new INFORMATION_SCHEMA
  tables, capacity tuning, multiple background I/O threads, and
  group commit.

  For information about these features, see the InnoDB Plugin
  Manual at

http://www.innodb.com/products/innodb_plugin/plugin-documentation

  For general information about using InnoDB in MySQL,
  see Section 13.6, "The InnoDB Storage Engine."

  The InnoDB Plugin is included in source and binary
  distributions, except RHEL3, RHEL4, SuSE 9 (x86, x86_64,
  ia64), and generic Linux RPM packages.

  To use the InnoDB Plugin, you must disable the built-in
  version of InnoDB that is also included and instruct the
  server to use InnoDB Plugin instead. To accomplish this, use
  the following lines in your my.cnf file:

    [mysqld]
    ignore-builtin-innodb
    plugin-load=innodb=ha_innodb_plugin.so

  For the plugin-load option, innodb is the name to associate
  with the plugin and ha_innodb_plugin.so is the name of the
  shared object library that contains the plugin code. The
  extension of .so applies for Unix (and similar) systems. For
  HP-UX on HPPA (11.11) or Windows, the extension should be .sl
  or .dll, respectively, rather than .so.

  If the server has problems finding the plugin when it starts
  up, specify the pathname to the plugin directory. For
  example, if plugins are located in the lib/mysql/plugin
  directory under the MySQL installation directory and you have
  installed MySQL at /usr/local/mysql, use these lines in your
  my.cnf file:

    [mysqld]
    ignore-builtin-innodb
    plugin-load=innodb=ha_innodb_plugin.so
    plugin_dir=/usr/local/mysql/lib/mysql/plugin

  The previous examples show how to activate the storage engine
  part of InnoDB Plugin, but the plugin also implements several
  InnoDB-related INFORMATION_SCHEMA tables. (For information
  about these tables, see

http://www.innodb.com/doc/innodb_plugin-1.0/innodb-information-schema.html)

  To enable these tables, include additional name=library
  pairs to the plugin-load option:

    [mysqld]
    ignore-builtin-innodb
    plugin-load=innodb=ha_innodb_plugin.so
      ;innodb_trx=ha_innodb_plugin.so
      ;innodb_locks=ha_innodb_plugin.so
      ;innodb_cmp=ha_innodb_plugin.so
      ;innodb_cmp_reset=ha_innodb_plugin.so
      ;innodb_cmpmem=ha_innodb_plugin.so
      ;innodb_cmpmem_reset=ha_innodb_plugin.so

  The plugin-load option here is formatted on multiple lines
  for display purposes but should be written in my.cnf using a
  single line without spaces in the option value. On Windows,
  substitute .dll for each instance of the .so extension.

  After the server starts up, verify that InnoDB Plugin has
  been loaded by using the SHOW PLUGINS statement. For example,
  if you have loaded the storage engine and the
  INFORMATION_SCHEMA tables, the output should include lines
  similar to these:

    mysql> SHOW PLUGINS;
    +---------------------+--------+--------------------+---------------------...
    | Name                | Status | Type               | Library             ...
    +---------------------+--------+--------------------+---------------------...
    | InnoDB              | ACTIVE | STORAGE ENGINE     | ha_innodb_plugin.so ...
    | INNODB_TRX          | ACTIVE | INFORMATION SCHEMA | ha_innodb_plugin.so ...
    | INNODB_LOCKS        | ACTIVE | INFORMATION SCHEMA | ha_innodb_plugin.so ...
    | INNODB_CMP          | ACTIVE | INFORMATION SCHEMA | ha_innodb_plugin.so ...
    | INNODB_CMP_RESET    | ACTIVE | INFORMATION SCHEMA | ha_innodb_plugin.so ...
    | INNODB_CMPMEM       | ACTIVE | INFORMATION SCHEMA | ha_innodb_plugin.so ...
    | INNODB_CMPMEM_RESET | ACTIVE | INFORMATION SCHEMA | ha_innodb_plugin.so ...
    +---------------------+--------+--------------------+---------------------...

  If you build MySQL from a source distribution, InnoDB Plugin
  is one of the storage engines that is built by default. Build
  MySQL the way you normally do; for example, by using the
  instructions at Section 2.10, "MySQL Installation Using a
  Source Distribution." After the build completes, you should
  find the plugin shared object file under the
  storage/innodb_plugin directory, and make install should
  install it in the plugin directory. Configure MySQL to use
  InnoDB Plugin as described earlier for binary distributions.

Misc Links:

http://dev.mysql.com/doc/innodb-plugin/1.0/en/innodb-plugin-introduction.html
http://dev.mysql.com/doc/innodb-plugin/1.0/en/innodb-changes.html
http://dev.mysql.com/doc/refman/5.5/en/innodb-5-5.html
http://dev.mysql.com/doc/innodb-plugin/1.0/en/innodb-plugin-installation.html

 
 


PlanetMySQL Voting: Vote UP / Vote DOWN

How well does your table fits in innodb buffer pool ?

Декабрь 10th, 2010

Understanding how well your tables and indexes fit to buffer pool are often very helpful to understand why some queries are IO bound and others not - it may be because the tables and indexes they are accessing are not in cache, for example being washed away by other queries. MySQL Server does not provide any information of this type, Percona Server however adds number of tables to Information Schema which makes this information possible. It is just few queries away:

SQL:
  1. mysql> SELECT `schema` AS table_schema,innodb_sys_tables.name AS table_name,innodb_sys_indexes.name AS index_name,cnt,dirty,hashed,round(cnt*100/index_size,2) fit_pct   FROM (SELECT index_id,count(*) cnt,sum(dirty=1) dirty ,sum(hashed=1) hashed FROM innodb_buffer_pool_pages_index GROUP BY index_id) bp JOIN innodb_sys_indexes ON id=index_id JOIN innodb_sys_tables ON table_id=innodb_sys_tables.id JOIN innodb_index_stats ON innodb_index_stats.table_name=innodb_sys_tables.name AND innodb_sys_indexes.name=innodb_index_stats.index_name AND innodb_index_stats.table_schema=innodb_sys_tables.schema  ORDER BY cnt DESC LIMIT 20;
  2. +--------------+--------------+--------------+------+-------+--------+---------+
  3. | table_schema | table_name   | index_name   | cnt  | dirty | hashed | fit_pct |
  4. +--------------+--------------+--------------+------+-------+--------+---------+
  5. | test         | a            | c            | 7976 |     0 |      0 |   13.73 |
  6. | test         | a            | PRIMARY      |   59 |     0 |      0 |    0.08 |
  7. | sbtest       | sbtest#P#p1  | PRIMARY      |   22 |     0 |      0 |   22.68 |
  8. | sbtest       | sbtest#P#p0  | PRIMARY      |   22 |     0 |      0 |   22.68 |
  9. | sbtest       | sbtest#P#p2  | PRIMARY      |   21 |     0 |      0 |   21.65 |
  10. | sbtest       | sbtest#P#p3  | PRIMARY      |   18 |     0 |      0 |   18.56 |
  11. | sbtest       | sbtest#P#p3  | k            |    4 |     0 |      0 |  100.00 |
  12. | sbtest       | sbtest#P#p2  | k            |    4 |     0 |      0 |  100.00 |
  13. | sbtest       | sbtest#P#p1  | k            |    4 |     0 |      0 |  100.00 |
  14. | sbtest       | sbtest#P#p0  | k            |    4 |     0 |      0 |  100.00 |
  15. | stats        | TABLES       | PRIMARY      |    2 |     0 |      0 |   66.67 |
  16. | stats        | TABLES       | TABLE_SCHEMA |    1 |     0 |      0100.00 |
  17. | percona      | transactions | PRIMARY      |    1 |     0 |      0100.00 |
  18. +--------------+--------------+--------------+------+-------+--------+---------+
  19. 13 rows IN SET (0.04 sec)

This query shows information about how many pages are in buffer pool for given table (cnt), how many of them are dirty (dirty), and what is the percentage of index fits in memory (fit_pct)
For illustration purposes I've created one table with partitions to show you will have the real "physical" table name which identifies table down to partition, which is very helpful for analyzes of your
access to partitions - you can actually check if your "hot" partitions really end up in the cache and "cold" are out of the cache, or is something happening which pushes them away from the cache.

You can use this feature to tune buffer pool invalidation strategy, for example play with innodb_old_blocks_pct and innodb_old_blocks_time actually observing data stored in buffer pool rather than using some form of temporary measures.

I often check these stats during warmup to see what is really getting warmed up first as well as how buffer pool is affected by batch jobs, alter tables, optimize table etc - the lasting impact these may have on system performance is often caused by impact they have on buffer pool which may take hours to recover.

This tool can be also helpful for capacity planning/performance management. In many cases you would learn you need a certain fit to buffer pool for tables/indexes for reasonable performance, you may try to count it too but it may be pretty hard as there are a lot of variables, including page fill factors etc.


Entry posted by Peter Zaitsev | No comment

Add to: delicious | digg | reddit | netscape | Google Bookmarks


PlanetMySQL Voting: Vote UP / Vote DOWN

Thinking about running OPTIMIZE on your Innodb Table ? Stop!

Декабрь 10th, 2010

Innodb/XtraDB tables do benefit from being reorganized often. You can get data physically laid out in primary key order as well as get better feel for primary key and index pages and so using less space,
it is just OPTIMIZE TABLE might not be best way to do it.

If you're running Innodb Plugin on Percona Server with XtraDB you get benefit of a great new feature - ability to build indexes by sort instead of via insertion. This process can be a lot faster, especially for large indexes which would get inserts in very random order, such as indexes on UUID column or something similar. It also produces a lot better fill factor. The problem is.... OPTIMIZE TABLE for Innodb tables does not get advantage of it for whatever reason.

Lets take a look at little benchmark I done by running OPTIMIZE for a second time on a table which is some 10 times larger than amount of memory I allocated for buffer pool:

SQL:
  1. CREATE TABLE `a` (
  2.   `id` int(10) UNSIGNED NOT NULL AUTO_INCREMENT,
  3.   `c` char(64) DEFAULT NULL,
  4.   PRIMARY KEY (`id`),
  5.   KEY `c` (`c`)
  6. ) ENGINE=InnoDB AUTO_INCREMENT=12582913 DEFAULT CHARSET=latin1
  7.  
  8. mysql> SELECT * FROM a ORDER BY id LIMIT 10;
  9. +----+------------------------------------------+
  10. | id | c                                        |
  11. +----+------------------------------------------+
  12. 1 | 813cf02d7d65de2639014dd1fb574d4c481ecac7 |
  13. 2 | 62960f5d5d50651e5a5983dacaedfa9a73a9ee87 |
  14. 3 | cea33998792ffe28b16b9272b950102a9633439f |
  15. 4 | 8346a7afa0a0791693338d96a07a944874340a1c |
  16. 5 | b00faaa432f507a0d16d2940ca8ec36699f141c8 |
  17. 6 | 8e00926cf6c9b13dc8e0664a744b7116c5c61036 |
  18. 7 | f151fe34b66fd4d28521d5e7ccb68b0d5d81f21b |
  19. 8 | 7fceb5afa200a27b81cab45f94903ce04d6f24db |
  20. 9 | 0397562dc35b5242842d68de424aa9f0b409d60f |
  21. | 10 | af8efbaef7010a1a3bfdff6609e5c233c897e1d5 |
  22. +----+------------------------------------------+
  23. 10 rows IN SET (0.04 sec)
  24.  
  25. # This is just random SHA(1) hashes
  26.  
  27. mysql> OPTIMIZE TABLE a;
  28. +--------+----------+----------+-------------------------------------------------------------------+
  29. | TABLE  | Op       | Msg_type | Msg_text                                                          |
  30. +--------+----------+----------+-------------------------------------------------------------------+
  31. | test.a | OPTIMIZE | note     | TABLE does NOT support OPTIMIZE, doing recreate + analyze instead |
  32. | test.a | OPTIMIZE | STATUS   | OK                                                                |
  33. +--------+----------+----------+-------------------------------------------------------------------+
  34. 2 rows IN SET (3 hours 3 min 35.15 sec)
  35.  
  36. mysql> ALTER TABLE a DROP KEY c;
  37. Query OK, 0 rows affected (0.46 sec)
  38. Records: 0  Duplicates: 0  Warnings: 0
  39.  
  40. mysql> OPTIMIZE TABLE a;
  41. +--------+----------+----------+-------------------------------------------------------------------+
  42. | TABLE  | Op       | Msg_type | Msg_text                                                          |
  43. +--------+----------+----------+-------------------------------------------------------------------+
  44. | test.a | OPTIMIZE | note     | TABLE does NOT support OPTIMIZE, doing recreate + analyze instead |
  45. | test.a | OPTIMIZE | STATUS   | OK                                                                |
  46. +--------+----------+----------+-------------------------------------------------------------------+
  47. 2 rows IN SET (4 min 5.52 sec)
  48.  
  49. mysql> ALTER TABLE a ADD KEY(c);
  50. Query OK, 0 rows affected (5 min 51.83 sec)
  51. Records: 0  Duplicates: 0  Warnings: 0

That's right ! Optimizing table straight away takes over 3 hours, while dropping indexes besides primary key, optimizing table and adding them back takes about 10 minutes, which is close than 20x speed difference and more compact index in the end.

So if you're considering running OPTIMIZE on your tables consider using this trick, it is especially handy when you're running it on the Slave where it is OK table is exposed without indexes for some time.
Note though nothing stops you from using LOCK TABLES on Innodb table to ensure there is not ton of queries starting reading table with no indexes and bringing box down.

You can also use this trick for ALTER TABLE which requires table rebuild. Dropping all indexes; doing ALTER and when adding them back can be a lot faster than straight ALTER TABLE.

P.S I do not know why this was not done when support for creating index by sorting was implemented. It looks very strange to me to have this feature implemented but majority of high level commands
or tools (like mysqldump) do not get advantage of it and will use old slow method of building indexes by insertion.


Entry posted by peter | 2 comments

Add to: delicious | digg | reddit | netscape | Google Bookmarks


PlanetMySQL Voting: Vote UP / Vote DOWN

Speaking on San Francisco Meetup 14-Dec : What’s new in XtraDB/InnoDB-plugin 5.1+

Ноябрь 30th, 2010

Erin and Mike, organizers of SF Meetup generously invited me to talk on coming SF Meetup on Dec-14 about new features in InnoDB in MySQL 5.1 and 5.5 and, what is pay attention to, when you upgrade from MySQL 5.0.
Although I personally mostly in 5.1->5.5 upgrade area, Erin insured me that upgrade from MySQL 5.0 is still actually question for many users, so I accepted invitation. Event details are on Meetup.com page. So if you are up for free pizza, soda and do not mind to listen to me, welcome to join!


Entry posted by Vadim | No comment

Add to: delicious | digg | reddit | netscape | Google Bookmarks


PlanetMySQL Voting: Vote UP / Vote DOWN

Percona Server 5.1.52-rel11.6

Ноябрь 23rd, 2010

Percona Server version 5.1.52-rel11.6 is now available for download.

The main purpose of this release is to update the current Percona stable release to the latest version of MySQL 5.1.

Functionality Added or Changed

  •  Percona Server 5.1.52-rel11.6 is now based on MySQL 5.1.52.
  •  New Features Added: None
  •  Other Changes: None

Bugs Fixed

  • Bug #671764innochecksum wasn’t distributed with RPM and .DEB packages. (Aleksandr Kuzminsky)
  • Bug #673426 – Use of some system variables as command-line options caused a crash or undefined behavior. (Oleg Tsarev)
  • Bug #673929 – Query cache misses were being reported for some queries when hits were actually occurring. (Oleg Tsarev)
  • Bug #676146 – The development environment test of log_slow_verbosity=innodb on a slave for row-based replication was not working correctly. (Oleg Tsarev)
  • Bug #676147 – The development environment test of option log_slow_slave_statements for row-based replication was not working correctly. (Oleg Tsarev)
  • Bug #676148 – Similar to Bug #676147. A check is now made for the replication type to test. (Oleg Tsarev)
  • Bug #676158 – Setting the query cache size to 512M caused test failure on low memory systems. (Aleksandr Kuzminsky)

Release Notes for this and previous releases can be found in our Wiki.

The latest downloads are available on our website. The latest source code for Percona Server, including the development branch, can be found on Launchpad.

Please report any bugs found at Bugs in Percona Server.

For general questions, use our Percona Discussions Group, and for development questions our Percona Development Group.

For support, commercial, and sponsorship inquiries, contact Percona.


Entry posted by Fred Linhoss | No comment

Add to: delicious | digg | reddit | netscape | Google Bookmarks


PlanetMySQL Voting: Vote UP / Vote DOWN

Moving from MyISAM to Innodb or XtraDB. Basics

Ноябрь 23rd, 2010

I do not know if it is because we’re hosting a free webinar on migrating MyISAM to Innodb or some other reason but recently I see a lot of questions about migration from MyISAM to Innodb.

Webinar will cover the process in a lot more details though I would like to go over basics in advance. You can also check my old post on this topic about Moving MyISAM to Innodb as well as searching the blog – We’ve blogged so much on this topic.

So what are the basics ?

Regression Benchmarks – Make sure to run regression benchmarks in particular in terms of concurrent behavior. You may have hidden dependencies of MyISAM table lock behavior in your applications, also check if your application handled deadlocks well. MyISAM will not produce deadlocks for Innodb you should always see deadlocks as a probability as long as you write to the database. They may be rate but it is rather hard to guaranty you will never run into them.

Performance Benchmarks – Innodb and MyISAM have different performance properties and you can’t really say one is faster than other it is very much workload dependent, and again concurrent tests should be important here. Innodb also may result in different plans for some queries which is rather easy to check with mk-upgrade

Feature Differences – There are some feature differences between MyISAM and Innodb though well it is typically easily spotted by converting tables to Innodb on restored backup. Full text search indexes, GIS, multi-column auto increment keys are great examples. There are also different limits for MyISAM and Innodb – it is possible to have some rows which can be stored in MyISAM but would not fit to Innodb, though this is an exception.

Space Innodb Tables tend to be larger. Again converting schema will show you some of this. Though the best is to take a look at the size after stressing system with load for a while as depending on the schema and usage Innodb tables may increase in size significantly due to fragmentation.

Usage Differences What works well for MyISAM may not work for Innodb and vice versa. You may benefit from different index structure for Innodb, including different primary key setup, as well as you may want to structure workload differently. With MyISAM it is often for people to do updates in small chunks, almost row by row to avoid holding table lock for long time in Innodb you want larger updates to reduce cost of transaction commit. You also may want to avoid excessive SELECT COUNT(*) FROM TBL (with no where clause) which is very fast for MyISAM but does table/index scan for Innodb.

Defaults You need to know two things about defaults for Innodb. First Depending on MySQL version they may be somewhere from suboptimal to absolutely disastrous, you do not want to try to run Innodb or XtraDB with them. Second Innodb is tuned to be ACID by default – if you’re moving from MyISAM often you do not need such strong guarantees and can at least change innodb_flush_log_at_trx_commit=2. It still will be much more secure than storing data in MyISAM. The 3 most important values to check are innodb_flush_log_at_trx_commit, innodb_buffer_pool_size and innodb_log_file_size. There are a lot more options for fine tuning but make sure at least these are right.


Entry posted by peter | No comment

Add to: delicious | digg | reddit | netscape | Google Bookmarks


PlanetMySQL Voting: Vote UP / Vote DOWN

Percona XtraBackup 1.4

Ноябрь 22nd, 2010

Percona XtraBackup 1.4 is now available for download.

Version 1.4 fixes problems related to incremental backups. If you do incremental backups, it’s strongly recommended that you upgrade to this release.

Functionality Added or Changed

  • Incremental backups have changed and now allow the restoration of full backups containing certain rollback transactions that previously caused problems. Please see Preparing the Backups and the --apply-log-only option. (From innobackupex, the --redo-only option should be used). (Yasufumi Kinoshita)
  • The XtraBackup Test Suite was implemented and is now a standard part of each distribution. (Aleksandr Kuzminsky)
  • Other New Features:
    • The --prepare option now reports xtrabackup_binlog_pos_innodb if the information exists. (Yasufumi Kinoshita)
    • When --prepare is used to restore a partical backup, the data dictionary is now cleaned and contains only tables that exist in the backup. (Yasufumi Kinoshita))
    • The --table option was extended to accept several regular expression arguments, separated by commas. (Yasufumi Kinoshita)
  • Other Changes:
    • Ported to the Percona Server 5.1.47-11 code base. (Yasufumi Kinoshita)
    • XtraBackup now uses the memory allocators of the host operating system, rather than the built-in InnoDB allocators (see Using Operating System Memory Allocators). (Yasufumi Kinoshita)

Bugs Fixed

  • Bug #595770 – Binaries are stripped by rpmbuild, so __os_install_post is redefined to change the default behaviour. (Aleksandr Kuzminsky)
  • Bug #589639 – Fixed a problem of hanging when tablespaces were deleted during the recovery process. (Yasufumi Kinoshita)
  • Bug #611960 – Fixed a segmentation fault in “xtrabackup”. (Yasufumi Kinoshita)
  • Miscellaneous important fixes related to incremental backups.

Release Notes for this and previous releases of Percona Xtrabackup can be found in our Wiki.

The latest downloads are available on our website. The latest source code can be found on Launchpad.

Please report any bugs found at Bugs in Percona XtraBackup.

For general questions, use our Percona Discussions Group, and for development questions our Percona Development Group.

For support, commercial, and sponsorship inquiries, contact Percona.


Entry posted by Fred Linhoss | No comment

Add to: delicious | digg | reddit | netscape | Google Bookmarks


PlanetMySQL Voting: Vote UP / Vote DOWN

DBJ – MySQL Hotbackups with Xtrabackup

Ноябрь 16th, 2010

The open-source xtrabackup tool from Percona brings much needed hot backup functionality to MySQL deployments.  In this database journal article we discuss logical, cold, and hot backups, then explain how to use xtrabackup on your MyISAM, InnoDB, and XtraDB tables to create at-the-ready backups.  We then take you through the step-by-step process to restore them, and even the process of point-in-time recovery too.

Database Journal – Hotbackups with Percona’s Xtrabackup


PlanetMySQL Voting: Vote UP / Vote DOWN