Archive for the ‘Installation’ Category

Installing MySQL from source/binary tarball as a Linux service

Май 1st, 2012

I've written before I prefer to do a manual install of MySQL over a repository one. I still do: I typically install from binary tarball or by compiling from source.

I'd like to share my setup procedure for Linux installation and service setup. I've done this dozens of times, on different Linux flavors, and it works well for me.

Installing from source

To get this straight: you sometimes have to compile the source files. I, for example, happen to use the Sphinx MySQLSE extension. You can only use it if compiled with MySQL. You had to compile a "vanilla" 5.1 version without query cache in order to completely remove the cache's mutex contention.

Anyway, I find the easiest way is to install onto a path associated with the server version. For example, I would install a 5.5 server onto /usr/local/mysql55

This way, a new version gets its own path, and no ambiguity.

To do that, use the prefix option on configuration step:

cd /path/to/extracted/source/tarball
sh BUILD/autorun.sh
./configure --prefix=/usr/local/mysql55
make
sudo make install

Once this is complete, you have everything under /usr/local/mysql55. This means binaries, libraries, scripts, etc.

To install the MySQL server as a service, copy the mysql.server script to /etc/init.d:

sudo cp /usr/local/mysql55/support-files/mysql.server /etc/init.d/mysql55

Again, I'm naming the script after the MySQL version. This avoids conflict with possible past or future installations of the MySQL server, which typically create a service named mysql or mysqld.

A thing to note about the mysql.server script is that it allows you (at around line #45) to set two variables:

  • basedir: path to your installation directory. When compiling from source this is already setup with the path provided to the configure script. Thus, in our example, you can expect this variable to read /usr/local/mysql55. So basically nothing to do here.
  • datadir: path to your data directory. If you're putting your my.cnf file in /etc or /etc/mysql, then setting datadir in my.cnf suffices. However, if you're going to put my.cnf itself on the data directory (e.g. so as to avoid collisions) then make sure to set the variable in the mysql.server init script.

Depending on your $PATH configuration, it is also a good idea to specify basedir variable on your my.cnf's [mysqld] section.

Which leads us to $PATH: your linux system is still unaware of the many binaries you've got in there. I typically add the following line at the end of /etc/bash.bashrc:

export PATH=/usr/local/mysql55:${PATH}

This is the most global PATH settings one can do. Alternatively, use /etc/profile, ~/.bashrc etc. (you may have noticed by now I'm working with bash).

Finally, need to setup the init script to run at startup and stop at shutdown.

  • On Debian/Ubuntu/related I use rcconf (I'm too lazy to remember the command line setup).
  • On RedHat/CentOS/related I use chkconfig --add mysql55, or  linuxconf (since I'm lazy).

Installing from binary tarball

The only difference is that the mysql.server script is unaware of our deployment path. So the basedir variable must be set in that file. Other than that, follow same steps as for source installation (oh, of course no need to configure & make...).


PlanetMySQL Voting: Vote UP / Vote DOWN

how to setup gitorious on ubuntu server 11.10

Апрель 4th, 2012

In this tutorial I will describe how to setup gitorious on Ubuntu 11.10. Gitorious – a Ruby on Rails web application – can be used to conclude git projects in an easy to manage user interface. In the README of the gitorious repository I found the evidence “One of the main challenges in Gitorious is its installation process. It is anything but trivial.” – It appears correct ;) During the installation I got some errors by the sphinx search engine and some ruby gems. You will find the workarounds in this tutorial.

This tutorial is based on the gitorious – UbuntuInstallation tutorial.

In this setup I will use the hostname gitorious.linux-adm.de – change this hostname in your installation.

software installation

Required packages

apt-get install build-essential zlib1g-dev tcl-dev libexpat-dev \
    libcurl4-openssl-dev postfix apache2 mysql-server mysql-client \
    apg geoip-bin libgeoip1 libgeoip-dev sqlite3 libsqlite3-dev \
    imagemagick libpcre3 libpcre3-dev zlib1g zlib1g-dev libyaml-dev \
    libmysqlclient15-dev apache2-dev libonig-dev ruby-dev rubygems \
    libopenssl-ruby phpmyadmin libdbd-mysql-ruby libmysql-ruby \
    libmagick++-dev  zip unzip memcached git-core git-svn git-doc \
    git-cvs irb libxslt1.1 libxslt1-dev

Ruby, sphinxsearch, stompserver

apt-get install libchronic-ruby librmagick-ruby libdaemons-ruby \
ruby-hoe ruby-echoe libyadis-ruby ruby-mime-types libopenid-ruby \
libdifflcs-ruby libjson-ruby librack-ruby libhmac-ruby rake stompserver \
libapache2-mod-passenger rails sphinxsearch sphinxbase-utils

fetch gitorious

git clone git://gitorious.org/gitorious/mainline.git /var/www/gitorious

install missing ruby gems

cd /var/www/gitorious/
gem install bundler
bundle install

git user

Create a git user with the homedir in /var/www/gitorious/ and set the correct ownership.

adduser --system --home /var/www/gitorious/ --no-create-home --group --shell /bin/bash git
chown git:git -R /var/www/gitorious/

Create a directory for the repositories and a cache directory. Gitorious will add the SSH public keys of the registered users to the authorized_keys file, therefore we have to create a .ssh file and an empty authorized_keys file with the correct permissions.

su - git
git submodule init && \
git submodule update
mkdir -p tmp/pids
mkdir repositories
mkdir tarballs
mkdir .ssh
touch .ssh/authorized_keys
chmod 700 .ssh/
chmod 600 .ssh/authorized_keys

Put gitorious binary on path:

cat >> /var/www/gitorious/.bash_profile <<EOF
export PATH="$PATH:/var/www/gitorious/script/"
EOF

gitorious configuration

prepare configuration files

cd /var/www/gitorious
cp config/database.sample.yml config/database.yml
cp config/gitorious.sample.yml config/gitorious.yml
cp config/broker.yml.example config/broker.yml

create a secret key

Create a secret key for the session cookie. (we need this key later in the next step.

apg -m 32

setup config/gitorious.yml

Setup gitorious in /var/www/gitorious/config/gitorious.yml:


repository_base_path: "/var/www/gitorious/repositories"

cookie_secret: xxx # add a secret key here the command "apg -m 32"

gitorious_client_port should be 80

gitorious_host: gitorious.linux-adm.de # change to your hostname

archive_cache_dir: "/var/www/gitorious/tarballs"

archive_work_dir: "/tmp/tarballs-work"

hide_http_clone_urls: true

is_gitorious_dot_org: false

Create the logroate configuration and the init scripts.

cp /var/www/gitorious/doc/templates/ubuntu/gitorious-logrotate /etc/logrotate.d/

cp /var/www/gitorious/doc/templates/ubuntu/git-ultrasphinx /etc/init.d/
cp /var/www/gitorious/doc/templates/ubuntu/git-daemon /etc/init.d/
cp /var/www/gitorious/doc/templates/ubuntu/git-poller /etc/init.d/

Set in /etc/init.d/git-daemon and in /etc/init.d/git-poller the RUBY_HOME variable to “/usr”

#RUBY_HOME="/opt/ruby-enterprise"
RUBY_HOME="/usr"

Activate the new gitorious init scripts

update-rc.d git-daemon defaults && \
    update-rc.d git-poller defaults && \
    update-rc.d git-ultrasphinx defaults

fix gem spec files

Try to start /etc/init.d/git-ultrasphinx
If you get the following error message you have to fix the spec files:

root@ubuntu03:/var/www/gitorious# /etc/init.d/git-ultrasphinx start
Invalid gemspec in [/var/lib/gems/1.8/specifications/json-1.5.4.gemspec]: invalid date format in specification: "2011-08-31 00:00:00.000000000Z"
Invalid gemspec in [/var/lib/gems/1.8/specifications/childprocess-0.2.1.gemspec]: invalid date format in specification: "2011-08-11 00:00:00.000000000Z"
Invalid gemspec in [/var/lib/gems/1.8/specifications/paperclip-2.4.5.gemspec]: invalid date format in specification: "2011-10-21 00:00:00.000000000Z"
(in /var/www/gitorious)
Invalid gemspec in [/var/lib/gems/1.8/specifications/json-1.5.4.gemspec]: invalid date format in specification: "2011-08-31 00:00:00.000000000Z"
Invalid gemspec in [/var/lib/gems/1.8/specifications/childprocess-0.2.1.gemspec]: invalid date format in specification: "2011-08-11 00:00:00.000000000Z"
Invalid gemspec in [/var/lib/gems/1.8/specifications/paperclip-2.4.5.gemspec]: invalid date format in specification: "2011-10-21 00:00:00.000000000Z"
Invalid gemspec in [/var/lib/gems/1.8/specifications/json-1.5.4.gemspec]: invalid date format in specification: "2011-08-31 00:00:00.000000000Z"
Invalid gemspec in [/var/lib/gems/1.8/specifications/childprocess-0.2.1.gemspec]: invalid date format in specification: "2011-08-11 00:00:00.000000000Z"
Invalid gemspec in [/var/lib/gems/1.8/specifications/paperclip-2.4.5.gemspec]: invalid date format in specification: "2011-10-21 00:00:00.000000000Z"
rake aborted!
Bundler couldn't find some gems. Did you run <code>bundle install</code>?
/var/www/gitorious/Rakefile:4
(See full trace by running task with --trace)

Change the “s.date” in the following spec files:

/var/lib/gems/1.8/specifications/json-1.5.4.gemspec
/var/lib/gems/1.8/specifications/childprocess-0.2.1.gemspec
/var/lib/gems/1.8/specifications/paperclip-2.4.5.gemspec

from:

s.date = %q{2011-08-31 00:00:00.000000000Z}

in:

s.date = %q{2012-08-31}

fix git-ultrasphinx

You have to fix the /var/www/gitorious/config/boot.rb file,
if you get the following error message after starting /etc/init.d/git-ultrasphinx.

root@ubuntu03:/var/www/gitorious# /etc/init.d/git-ultrasphinx start
(in /var/www/gitorious)
rake aborted!
uninitialized constant ActiveSupport::Dependencies::Mutex
/var/www/gitorious/Rakefile:10:in `require'
(See full trace by running task with --trace)

Add the following line in /var/www/gitorious/config/boot.rb.


# Don't change this file!
# Configure your app in config/environment.rb and config/environments/*.rb

require 'thread'

Apache configuration

Disable the default vhost configuration

a2dissite default
a2dissite default-ssl

create a new vhost configuration

cat >> /etc/apache2/sites-available/gitorious <<EOF
<VirtualHost *:80>
    ServerName gitorious.linux-adm.de         # change the hostname!
    DocumentRoot /var/www/gitorious/public
</VirtualHost>
EOF

create a new SSL vhost configuration

cat >> /etc/apache2/sites-available/gitorious-ssl <<EOF
<IfModule mod_ssl.c>
    <VirtualHost _default_:443>
        ServerName gitorious.linux-adm.de   # change the hostname!
        DocumentRoot /var/www/gitorious/public
        SSLEngine on
        SSLCertificateFile    /etc/ssl/certs/ssl-cert-snakeoil.pem
        SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
        BrowserMatch ".*MSIE.*" nokeepalive ssl-unclean-shutdown downgrade-1.0 force-response-1.0
    </VirtualHost>
</IfModule>
EOF

a2ensite gitorious
a2ensite gitorious-ssl
a2enmod ssl   # enable SSL

create MySQL User for gitorious

mysql -u root -p
GRANT ALL PRIVILEGES ON *.* TO 'gitorious'@'localhost' IDENTIFIED BY 'XXXX' WITH GRANT OPTION;
FLUSH PRIVILEGES;

change the password XXXX!

Add the MySQL User to the production section in /var/www/gitorious/config/database.yml


production:
adapter: mysql
database: gitorious_production
username: gitorious
password: XXXX # change the password!
host: localhost
encoding: utf8

Starting gitorious

Run the following commands as user git.

export RAILS_ENV=production && 
bundle exec rake db:create && 
bundle exec rake db:migrate && 
bundle exec rake ultrasphinx:bootstrap

fix ultrasphinx/production.conf

If you get the following error message, you have to fix the /var/www/gitorious/ultrasphinx/production.conf file.

export RAILS_ENV=production && 
bundle exec rake db:create && 
bundle exec rake db:migrate && 
bundle exec rake ultrasphinx:bootstrap

Sphinx 0.9.9-release (r2117)
Copyright (c) 2001-2009, Andrew Aksyonoff

using config file '/var/www/gitorious/config/ultrasphinx/production.conf'...
WARNING: key 'address' is deprecated in /var/www/gitorious/config/ultrasphinx/production.conf line 11; use 'listen' instead.
listening on all interfaces, port=3312
WARNING: index 'main': preload: failed to open /var/www/gitorious/db/sphinx//sphinx_index_main.sph: No such file or directory; NOT SERVING
FATAL: no valid indexes to serve
Failed to start
Done
Please restart your application containers

Change in /var/www/gitorious/config/ultrasphinx/production.conf the word “base_tags” to “tags”.

git-ultrasphinx cronjob

Add the following cronjob for the git user.

crontab -e -u git


* * * * * cd /var/www/gitorious && /usr/local/bin/bundle exec rake ultrasphinx:index RAILS_ENV=production

create an admin user

git@ubuntu03:/var/www/gitorious$ env RAILS_ENV=production ruby1.8 script/create_admin
Type in Administrator's e-mail: 
admin@gitorious.linux-adm.de
Type in Administrator's password: 
test
Admin user created successfully.

After a reboot of your system the gitorious web front-end should be available.


PlanetMySQL Voting: Vote UP / Vote DOWN

how to setup gitorious on ubuntu server 11.10

Апрель 4th, 2012

In this tutorial I will describe how to setup gitorious on Ubuntu 11.10. Gitorious – a Ruby on Rails web application – can be used to conclude git projects in an easy to manage user interface. In the README of the gitorious repository I found the evidence “One of the main challenges in Gitorious is its installation process. It is anything but trivial.” – It appears correct ;) During the installation I got some errors by the sphinx search engine and some ruby gems. You will find the workarounds in this tutorial.

This tutorial is based on the gitorious – UbuntuInstallation tutorial.

In this setup I will use the hostname gitorious.linux-adm.de – change this hostname in your installation.

software installation

Required packages

apt-get install build-essential zlib1g-dev tcl-dev libexpat-dev \
    libcurl4-openssl-dev postfix apache2 mysql-server mysql-client \
    apg geoip-bin libgeoip1 libgeoip-dev sqlite3 libsqlite3-dev \
    imagemagick libpcre3 libpcre3-dev zlib1g zlib1g-dev libyaml-dev \
    libmysqlclient15-dev apache2-dev libonig-dev ruby-dev rubygems \
    libopenssl-ruby phpmyadmin libdbd-mysql-ruby libmysql-ruby \
    libmagick++-dev  zip unzip memcached git-core git-svn git-doc \
    git-cvs irb libxslt1.1 libxslt1-dev

Ruby, sphinxsearch, stompserver

apt-get install libchronic-ruby librmagick-ruby libdaemons-ruby \
ruby-hoe ruby-echoe libyadis-ruby ruby-mime-types libopenid-ruby \
libdifflcs-ruby libjson-ruby librack-ruby libhmac-ruby rake stompserver \
libapache2-mod-passenger rails sphinxsearch sphinxbase-utils

fetch gitorious

git clone git://gitorious.org/gitorious/mainline.git /var/www/gitorious

install missing ruby gems

cd /var/www/gitorious/
gem install bundler
bundle install

git user

Create a git user with the homedir in /var/www/gitorious/ and set the correct ownership.

adduser --system --home /var/www/gitorious/ --no-create-home --group --shell /bin/bash git
chown git:git -R /var/www/gitorious/

Create a directory for the repositories and a cache directory. Gitorious will add the SSH public keys of the registered users to the authorized_keys file, therefore we have to create a .ssh file and an empty authorized_keys file with the correct permissions.

su - git
git submodule init && \
git submodule update
mkdir -p tmp/pids
mkdir repositories
mkdir tarballs
mkdir .ssh
touch .ssh/authorized_keys
chmod 700 .ssh/
chmod 600 .ssh/authorized_keys

Put gitorious binary on path:

cat >> /var/www/gitorious/.bash_profile <<EOF
export PATH="$PATH:/var/www/gitorious/script/"
EOF

gitorious configuration

prepare configuration files

cd /var/www/gitorious
cp config/database.sample.yml config/database.yml
cp config/gitorious.sample.yml config/gitorious.yml
cp config/broker.yml.example config/broker.yml

create a secret key

Create a secret key for the session cookie. (we need this key later in the next step.

apg -m 32

setup config/gitorious.yml

Setup gitorious in /var/www/gitorious/config/gitorious.yml:


repository_base_path: "/var/www/gitorious/repositories"

cookie_secret: xxx # add a secret key here the command "apg -m 32"

gitorious_client_port should be 80

gitorious_host: gitorious.linux-adm.de # change to your hostname

archive_cache_dir: "/var/www/gitorious/tarballs"

archive_work_dir: "/tmp/tarballs-work"

hide_http_clone_urls: true

is_gitorious_dot_org: false

Create the logroate configuration and the init scripts.

cp /var/www/gitorious/doc/templates/ubuntu/gitorious-logrotate /etc/logrotate.d/

cp /var/www/gitorious/doc/templates/ubuntu/git-ultrasphinx /etc/init.d/
cp /var/www/gitorious/doc/templates/ubuntu/git-daemon /etc/init.d/
cp /var/www/gitorious/doc/templates/ubuntu/git-poller /etc/init.d/

Set in /etc/init.d/git-daemon and in /etc/init.d/git-poller the RUBY_HOME variable to “/usr”

#RUBY_HOME="/opt/ruby-enterprise"
RUBY_HOME="/usr"

Activate the new gitorious init scripts

update-rc.d git-daemon defaults && \
    update-rc.d git-poller defaults && \
    update-rc.d git-ultrasphinx defaults

fix gem spec files

Try to start /etc/init.d/git-ultrasphinx
If you get the following error message you have to fix the spec files:

root@ubuntu03:/var/www/gitorious# /etc/init.d/git-ultrasphinx start
Invalid gemspec in [/var/lib/gems/1.8/specifications/json-1.5.4.gemspec]: invalid date format in specification: "2011-08-31 00:00:00.000000000Z"
Invalid gemspec in [/var/lib/gems/1.8/specifications/childprocess-0.2.1.gemspec]: invalid date format in specification: "2011-08-11 00:00:00.000000000Z"
Invalid gemspec in [/var/lib/gems/1.8/specifications/paperclip-2.4.5.gemspec]: invalid date format in specification: "2011-10-21 00:00:00.000000000Z"
(in /var/www/gitorious)
Invalid gemspec in [/var/lib/gems/1.8/specifications/json-1.5.4.gemspec]: invalid date format in specification: "2011-08-31 00:00:00.000000000Z"
Invalid gemspec in [/var/lib/gems/1.8/specifications/childprocess-0.2.1.gemspec]: invalid date format in specification: "2011-08-11 00:00:00.000000000Z"
Invalid gemspec in [/var/lib/gems/1.8/specifications/paperclip-2.4.5.gemspec]: invalid date format in specification: "2011-10-21 00:00:00.000000000Z"
Invalid gemspec in [/var/lib/gems/1.8/specifications/json-1.5.4.gemspec]: invalid date format in specification: "2011-08-31 00:00:00.000000000Z"
Invalid gemspec in [/var/lib/gems/1.8/specifications/childprocess-0.2.1.gemspec]: invalid date format in specification: "2011-08-11 00:00:00.000000000Z"
Invalid gemspec in [/var/lib/gems/1.8/specifications/paperclip-2.4.5.gemspec]: invalid date format in specification: "2011-10-21 00:00:00.000000000Z"
rake aborted!
Bundler couldn't find some gems. Did you run <code>bundle install</code>?
/var/www/gitorious/Rakefile:4
(See full trace by running task with --trace)

Change the “s.date” in the following spec files:

/var/lib/gems/1.8/specifications/json-1.5.4.gemspec
/var/lib/gems/1.8/specifications/childprocess-0.2.1.gemspec
/var/lib/gems/1.8/specifications/paperclip-2.4.5.gemspec

from:

s.date = %q{2011-08-31 00:00:00.000000000Z}

in:

s.date = %q{2012-08-31}

fix git-ultrasphinx

You have to fix the /var/www/gitorious/config/boot.rb file,
if you get the following error message after starting /etc/init.d/git-ultrasphinx.

root@ubuntu03:/var/www/gitorious# /etc/init.d/git-ultrasphinx start
(in /var/www/gitorious)
rake aborted!
uninitialized constant ActiveSupport::Dependencies::Mutex
/var/www/gitorious/Rakefile:10:in `require'
(See full trace by running task with --trace)

Add the following line in /var/www/gitorious/config/boot.rb.


# Don't change this file!
# Configure your app in config/environment.rb and config/environments/*.rb

require 'thread'

Apache configuration

Disable the default vhost configuration

a2dissite default
a2dissite default-ssl

create a new vhost configuration

cat >> /etc/apache2/sites-available/gitorious <<EOF
<VirtualHost *:80>
    ServerName gitorious.linux-adm.de         # change the hostname!
    DocumentRoot /var/www/gitorious/public
</VirtualHost>
EOF

create a new SSL vhost configuration

cat >> /etc/apache2/sites-available/gitorious-ssl <<EOF
<IfModule mod_ssl.c>
    <VirtualHost _default_:443>
        ServerName gitorious.linux-adm.de   # change the hostname!
        DocumentRoot /var/www/gitorious/public
        SSLEngine on
        SSLCertificateFile    /etc/ssl/certs/ssl-cert-snakeoil.pem
        SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
        BrowserMatch ".*MSIE.*" nokeepalive ssl-unclean-shutdown downgrade-1.0 force-response-1.0
    </VirtualHost>
</IfModule>
EOF

a2ensite gitorious
a2ensite gitorious-ssl
a2enmod ssl   # enable SSL

create MySQL User for gitorious

mysql -u root -p
GRANT ALL PRIVILEGES ON *.* TO 'gitorious'@'localhost' IDENTIFIED BY 'XXXX' WITH GRANT OPTION;
FLUSH PRIVILEGES;

change the password XXXX!

Add the MySQL User to the production section in /var/www/gitorious/config/database.yml


production:
adapter: mysql
database: gitorious_production
username: gitorious
password: XXXX # change the password!
host: localhost
encoding: utf8

Starting gitorious

Run the following commands as user git.

export RAILS_ENV=production && 
bundle exec rake db:create && 
bundle exec rake db:migrate && 
bundle exec rake ultrasphinx:bootstrap

fix ultrasphinx/production.conf

If you get the following error message, you have to fix the /var/www/gitorious/ultrasphinx/production.conf file.

export RAILS_ENV=production && 
bundle exec rake db:create && 
bundle exec rake db:migrate && 
bundle exec rake ultrasphinx:bootstrap

Sphinx 0.9.9-release (r2117)
Copyright (c) 2001-2009, Andrew Aksyonoff

using config file '/var/www/gitorious/config/ultrasphinx/production.conf'...
WARNING: key 'address' is deprecated in /var/www/gitorious/config/ultrasphinx/production.conf line 11; use 'listen' instead.
listening on all interfaces, port=3312
WARNING: index 'main': preload: failed to open /var/www/gitorious/db/sphinx//sphinx_index_main.sph: No such file or directory; NOT SERVING
FATAL: no valid indexes to serve
Failed to start
Done
Please restart your application containers

Change in /var/www/gitorious/config/ultrasphinx/production.conf the word “base_tags” to “tags”.

git-ultrasphinx cronjob

Add the following cronjob for the git user.

crontab -e -u git


* * * * * cd /var/www/gitorious && /usr/local/bin/bundle exec rake ultrasphinx:index RAILS_ENV=production

create an admin user

git@ubuntu03:/var/www/gitorious$ env RAILS_ENV=production ruby1.8 script/create_admin
Type in Administrator's e-mail: 
admin@gitorious.linux-adm.de
Type in Administrator's password: 
test
Admin user created successfully.

After a reboot of your system the gitorious web front-end should be available.


PlanetMySQL Voting: Vote UP / Vote DOWN

MySQL 5.6 too verbose when creating data directory

Март 18th, 2012

When I install a MySQL package using MySQL Sandbox, if everything goes smoothly, I get an informative message on standard output, and I keep working.

This is OK


$HOME/opt/mysql/5.5.15/scripts/mysql_install_db --no-defaults \
--user=$USER --basedir=$HOME/opt/mysql/5.5.15 \
--datadir=$HOME/sandboxes/msb_5_5_15/data \
--lower_case_table_names=2
Installing MySQL system tables...
OK
Filling help tables...
OK

To start mysqld at boot time you have to copy
support-files/mysql.server to the right place for your system

PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER !
To do so, start the server, then issue the following commands:

/Users/gmax/opt/mysql/5.5.15/bin/mysqladmin -u root password 'new-password'
/Users/gmax/opt/mysql/5.5.15/bin/mysqladmin -u root -h gmac4.local password 'new-password'

Alternatively you can run:
/Users/gmax/opt/mysql/5.5.15/bin/mysql_secure_installation

which will also give you the option of removing the test
databases and anonymous user created by default. This is
strongly recommended for production servers.

See the manual for more instructions.

You can start the MySQL daemon with:
cd /Users/gmax/opt/mysql/5.5.15 ; /Users/gmax/opt/mysql/5.5.15/bin/mysqld_safe &

You can test the MySQL daemon with mysql-test-run.pl
cd /Users/gmax/opt/mysql/5.5.15/mysql-test ; perl mysql-test-run.pl

Please report any problems with the /Users/gmax/opt/mysql/5.5.15/scripts/mysqlbug script!
I can actually suppress this output, confident that, if something goes wrong, the error comes to my screen loud and clear. For example, if I try to install to a data directory that is write protected, I get this:

chmod 444 $HOME/sandboxes/msb_5_5_15/data
$HOME/opt/mysql/5.5.15/scripts/mysql_install_db --no-defaults \
--user=$USER --basedir=$HOME/opt/mysql/5.5.15 \
--datadir=$HOME/sandboxes/msb_5_5_15/data \
--lower_case_table_names=2 > /dev/null

mkdir: /Users/gmax/sandboxes/msb_5_5_15/data: Permission denied
chmod: /Users/gmax/sandboxes/msb_5_5_15/data: Permission denied
chown: /Users/gmax/sandboxes/msb_5_5_15/data: Permission denied
This, way, I know that there was an error, it is very clear and readable. I don't need to hunt it down within the regular messages. The standard error is a separate file descriptor, which can be read independently from the standard input.

After fixing permissions:


chmod 755 ~/sandboxes/msb_5_5_15/
$HOME/opt/mysql/5.5.15/scripts/mysql_install_db --no-defaults \
--user=$USER --basedir=$HOME/opt/mysql/5.5.15 \
--datadir=$HOME/sandboxes/msb_5_5_15/data \
--lower_case_table_names=2 > /dev/null
# empty line: means all OK

This is very convenient, and it is the Unix way.

This is not OK

Now, let's try the same with MySQL 5.6


$BASEDIR/scripts/mysql_install_db --no-defaults --user=tungsten \
--basedir=$BASEDIR --datadir=/home/tungsten/sandboxes/msb_5_6_4/data \
--tmpdir=/home/tungsten/sandboxes/msb_5_6_4/tmp > /dev/null
120318 10:10:44 InnoDB: The InnoDB memory heap is disabled
120318 10:10:44 InnoDB: Mutexes and rw_locks use GCC atomic builtins
120318 10:10:44 InnoDB: Compressed tables use zlib 1.2.3
120318 10:10:44 InnoDB: Using Linux native AIO
120318 10:10:44 InnoDB: CPU supports crc32 instructions
120318 10:10:44 InnoDB: Initializing buffer pool, size = 128.0M
120318 10:10:44 InnoDB: Completed initialization of buffer pool
InnoDB: The first specified data file ./ibdata1 did not exist:
InnoDB: a new database to be created!
120318 10:10:44 InnoDB: Setting file ./ibdata1 size to 10 MB
InnoDB: Database physically writes the file full: wait...
120318 10:10:44 InnoDB: Log file ./ib_logfile0 did not exist: new to be created
InnoDB: Setting log file ./ib_logfile0 size to 5 MB
InnoDB: Database physically writes the file full: wait...
120318 10:10:44 InnoDB: Log file ./ib_logfile1 did not exist: new to be created
InnoDB: Setting log file ./ib_logfile1 size to 5 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Doublewrite buffer not found: creating new
InnoDB: Doublewrite buffer created
120318 10:10:44 InnoDB: 128 rollback segment(s) are active.
InnoDB: Creating foreign key constraint system tables
InnoDB: Foreign key constraint system tables created
120318 10:10:44 InnoDB: 1.2.4 started; log sequence number 0
120318 10:10:44 [Warning] Info table is not ready to be used. Table 'mysql.slave_master_info' cannot be opened.
120318 10:10:44 [Warning] Error while checking replication metadata. Setting the requested repository in order to give users the chance to fix the problem and restart the server. If this is a live upgrade please consider using mysql_upgrade to fix the problem.
120318 10:10:44 [Warning] Info table is not ready to be used. Table 'mysql.slave_relay_log_info' cannot be opened.
120318 10:10:44 [Warning] Error while checking replication metadata. Setting the requested repository in order to give users the chance to fix the problem and restart the server. If this is a live upgrade please consider using mysql_upgrade to fix the problem.
120318 10:10:44 [Note] Binlog end
120318 10:10:44 [Note] Shutting down plugin 'partition'
120318 10:10:44 [Note] Shutting down plugin 'ARCHIVE'
120318 10:10:44 [Note] Shutting down plugin 'BLACKHOLE'
120318 10:10:44 [Note] Shutting down plugin 'PERFORMANCE_SCHEMA'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN_COLS'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_SYS_FIELDS'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_SYS_COLUMNS'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_SYS_INDEXES'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_SYS_TABLESTATS'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_SYS_TABLES'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_FT_INDEX_TABLE'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_FT_INDEX_CACHE'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_FT_CONFIG'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_FT_BEING_DELETED'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_FT_DELETED'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_FT_INSERTED'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_FT_DEFAULT_STOPWORD'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_METRICS'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_BUFFER_POOL_STATS'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE_LRU'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_CMPMEM_RESET'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_CMPMEM'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_CMP_RESET'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_CMP'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_LOCK_WAITS'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_LOCKS'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_TRX'
120318 10:10:44 [Note] Shutting down plugin 'InnoDB'
120318 10:10:44 InnoDB: FTS optimize thread exiting.
120318 10:10:44 InnoDB: Starting shutdown...
120318 10:10:46 InnoDB: Shutdown completed; log sequence number 1602841
120318 10:10:46 [Note] Shutting down plugin 'CSV'
120318 10:10:46 [Note] Shutting down plugin 'MEMORY'
120318 10:10:46 [Note] Shutting down plugin 'MyISAM'
120318 10:10:46 [Note] Shutting down plugin 'MRG_MYISAM'
120318 10:10:46 [Note] Shutting down plugin 'mysql_old_password'
120318 10:10:46 [Note] Shutting down plugin 'mysql_native_password'
120318 10:10:46 [Note] Shutting down plugin 'binlog'
120318 10:10:46 InnoDB: The InnoDB memory heap is disabled
120318 10:10:46 InnoDB: Mutexes and rw_locks use GCC atomic builtins
120318 10:10:46 InnoDB: Compressed tables use zlib 1.2.3
120318 10:10:46 InnoDB: Using Linux native AIO
120318 10:10:46 InnoDB: CPU supports crc32 instructions
120318 10:10:46 InnoDB: Initializing buffer pool, size = 128.0M
120318 10:10:46 InnoDB: Completed initialization of buffer pool
120318 10:10:46 InnoDB: highest supported file format is Barracuda.
120318 10:10:46 InnoDB: 128 rollback segment(s) are active.
120318 10:10:46 InnoDB: Waiting for the background threads to start
120318 10:10:47 InnoDB: 1.2.4 started; log sequence number 1602841
120318 10:10:47 [Note] Binlog end
120318 10:10:47 [Note] Shutting down plugin 'partition'
120318 10:10:47 [Note] Shutting down plugin 'ARCHIVE'
120318 10:10:47 [Note] Shutting down plugin 'BLACKHOLE'
120318 10:10:47 [Note] Shutting down plugin 'PERFORMANCE_SCHEMA'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN_COLS'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_SYS_FIELDS'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_SYS_COLUMNS'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_SYS_INDEXES'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_SYS_TABLESTATS'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_SYS_TABLES'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_FT_INDEX_TABLE'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_FT_INDEX_CACHE'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_FT_CONFIG'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_FT_BEING_DELETED'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_FT_DELETED'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_FT_INSERTED'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_FT_DEFAULT_STOPWORD'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_METRICS'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_BUFFER_POOL_STATS'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE_LRU'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_CMPMEM_RESET'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_CMPMEM'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_CMP_RESET'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_CMP'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_LOCK_WAITS'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_LOCKS'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_TRX'
120318 10:10:47 [Note] Shutting down plugin 'InnoDB'
120318 10:10:47 InnoDB: FTS optimize thread exiting.
120318 10:10:47 InnoDB: Starting shutdown...
120318 10:10:49 InnoDB: Shutdown completed; log sequence number 1602851
120318 10:10:49 [Note] Shutting down plugin 'CSV'
120318 10:10:49 [Note] Shutting down plugin 'MEMORY'
120318 10:10:49 [Note] Shutting down plugin 'MyISAM'
120318 10:10:49 [Note] Shutting down plugin 'MRG_MYISAM'
120318 10:10:49 [Note] Shutting down plugin 'mysql_old_password'
120318 10:10:49 [Note] Shutting down plugin 'mysql_native_password'
120318 10:10:49 [Note] Shutting down plugin 'binlog'
Why is this bad? Because you don't see at a glance what is right and what is wrong. All the above messages are printed to the standard error, the kind of output that should be reserved for, well, errors! If the standard error is used for regular messages, you may miss the important error messages that are instead mixed with the "all is OK" messages.

There is Bug#60934 filed about this issue, but it has been considered a feature request, and as such unlikely to be fixed.

In the above text there is something more. There are warnings, mixed with the standard text, telling me of errors that the bootstrap operation is not in a position to fix, like replication metadata and slave_master_info.

MySQL developers, please fix this issue. Users need error messages when there is something wrong, and warning or error messages about something that can actually be fixed. When MySQL 5.6 goes GA, this issue will hit most everybody.

PlanetMySQL Voting: Vote UP / Vote DOWN

MySQL 5.6 too verbose when creating data directory

Март 18th, 2012
When I install a MySQL package using MySQL Sandbox, if everything goes smoothly, I get an informative message on standard output, and I keep working.

This is OK


$HOME/opt/mysql/5.5.15/scripts/mysql_install_db --no-defaults \
--user=$USER --basedir=$HOME/opt/mysql/5.5.15 \
--datadir=$HOME/sandboxes/msb_5_5_15/data \
--lower_case_table_names=2
Installing MySQL system tables...
OK
Filling help tables...
OK

To start mysqld at boot time you have to copy
support-files/mysql.server to the right place for your system

PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER !
To do so, start the server, then issue the following commands:

/Users/gmax/opt/mysql/5.5.15/bin/mysqladmin -u root password 'new-password'
/Users/gmax/opt/mysql/5.5.15/bin/mysqladmin -u root -h gmac4.local password 'new-password'

Alternatively you can run:
/Users/gmax/opt/mysql/5.5.15/bin/mysql_secure_installation

which will also give you the option of removing the test
databases and anonymous user created by default. This is
strongly recommended for production servers.

See the manual for more instructions.

You can start the MySQL daemon with:
cd /Users/gmax/opt/mysql/5.5.15 ; /Users/gmax/opt/mysql/5.5.15/bin/mysqld_safe &

You can test the MySQL daemon with mysql-test-run.pl
cd /Users/gmax/opt/mysql/5.5.15/mysql-test ; perl mysql-test-run.pl

Please report any problems with the /Users/gmax/opt/mysql/5.5.15/scripts/mysqlbug script!
I can actually suppress this output, confident that, if something goes wrong, the error comes to my screen loud and clear. For example, if I try to install to a data directory that is write protected, I get this:

chmod 444 $HOME/sandboxes/msb_5_5_15/data
$HOME/opt/mysql/5.5.15/scripts/mysql_install_db --no-defaults \
--user=$USER --basedir=$HOME/opt/mysql/5.5.15 \
--datadir=$HOME/sandboxes/msb_5_5_15/data \
--lower_case_table_names=2 > /dev/null

mkdir: /Users/gmax/sandboxes/msb_5_5_15/data: Permission denied
chmod: /Users/gmax/sandboxes/msb_5_5_15/data: Permission denied
chown: /Users/gmax/sandboxes/msb_5_5_15/data: Permission denied
This, way, I know that there was an error, it is very clear and readable. I don't need to hunt it down within the regular messages. The standard error is a separate file descriptor, which can be read independently from the standard input.After fixing permissions:

chmod 755 ~/sandboxes/msb_5_5_15/
$HOME/opt/mysql/5.5.15/scripts/mysql_install_db --no-defaults \
--user=$USER --basedir=$HOME/opt/mysql/5.5.15 \
--datadir=$HOME/sandboxes/msb_5_5_15/data \
--lower_case_table_names=2 > /dev/null
# empty line: means all OK
This is very convenient, and it is the Unix way.

This is not OK

Now, let's try the same with MySQL 5.6

$BASEDIR/scripts/mysql_install_db --no-defaults --user=tungsten \
--basedir=$BASEDIR --datadir=/home/tungsten/sandboxes/msb_5_6_4/data \
--tmpdir=/home/tungsten/sandboxes/msb_5_6_4/tmp > /dev/null
120318 10:10:44 InnoDB: The InnoDB memory heap is disabled
120318 10:10:44 InnoDB: Mutexes and rw_locks use GCC atomic builtins
120318 10:10:44 InnoDB: Compressed tables use zlib 1.2.3
120318 10:10:44 InnoDB: Using Linux native AIO
120318 10:10:44 InnoDB: CPU supports crc32 instructions
120318 10:10:44 InnoDB: Initializing buffer pool, size = 128.0M
120318 10:10:44 InnoDB: Completed initialization of buffer pool
InnoDB: The first specified data file ./ibdata1 did not exist:
InnoDB: a new database to be created!
120318 10:10:44 InnoDB: Setting file ./ibdata1 size to 10 MB
InnoDB: Database physically writes the file full: wait...
120318 10:10:44 InnoDB: Log file ./ib_logfile0 did not exist: new to be created
InnoDB: Setting log file ./ib_logfile0 size to 5 MB
InnoDB: Database physically writes the file full: wait...
120318 10:10:44 InnoDB: Log file ./ib_logfile1 did not exist: new to be created
InnoDB: Setting log file ./ib_logfile1 size to 5 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Doublewrite buffer not found: creating new
InnoDB: Doublewrite buffer created
120318 10:10:44 InnoDB: 128 rollback segment(s) are active.
InnoDB: Creating foreign key constraint system tables
InnoDB: Foreign key constraint system tables created
120318 10:10:44 InnoDB: 1.2.4 started; log sequence number 0
120318 10:10:44 [Warning] Info table is not ready to be used. Table 'mysql.slave_master_info' cannot be opened.
120318 10:10:44 [Warning] Error while checking replication metadata. Setting the requested repository in order to give users the chance to fix the problem and restart the server. If this is a live upgrade please consider using mysql_upgrade to fix the problem.
120318 10:10:44 [Warning] Info table is not ready to be used. Table 'mysql.slave_relay_log_info' cannot be opened.
120318 10:10:44 [Warning] Error while checking replication metadata. Setting the requested repository in order to give users the chance to fix the problem and restart the server. If this is a live upgrade please consider using mysql_upgrade to fix the problem.
120318 10:10:44 [Note] Binlog end
120318 10:10:44 [Note] Shutting down plugin 'partition'
120318 10:10:44 [Note] Shutting down plugin 'ARCHIVE'
120318 10:10:44 [Note] Shutting down plugin 'BLACKHOLE'
120318 10:10:44 [Note] Shutting down plugin 'PERFORMANCE_SCHEMA'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN_COLS'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_SYS_FIELDS'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_SYS_COLUMNS'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_SYS_INDEXES'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_SYS_TABLESTATS'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_SYS_TABLES'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_FT_INDEX_TABLE'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_FT_INDEX_CACHE'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_FT_CONFIG'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_FT_BEING_DELETED'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_FT_DELETED'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_FT_INSERTED'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_FT_DEFAULT_STOPWORD'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_METRICS'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_BUFFER_POOL_STATS'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE_LRU'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_CMPMEM_RESET'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_CMPMEM'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_CMP_RESET'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_CMP'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_LOCK_WAITS'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_LOCKS'
120318 10:10:44 [Note] Shutting down plugin 'INNODB_TRX'
120318 10:10:44 [Note] Shutting down plugin 'InnoDB'
120318 10:10:44 InnoDB: FTS optimize thread exiting.
120318 10:10:44 InnoDB: Starting shutdown...
120318 10:10:46 InnoDB: Shutdown completed; log sequence number 1602841
120318 10:10:46 [Note] Shutting down plugin 'CSV'
120318 10:10:46 [Note] Shutting down plugin 'MEMORY'
120318 10:10:46 [Note] Shutting down plugin 'MyISAM'
120318 10:10:46 [Note] Shutting down plugin 'MRG_MYISAM'
120318 10:10:46 [Note] Shutting down plugin 'mysql_old_password'
120318 10:10:46 [Note] Shutting down plugin 'mysql_native_password'
120318 10:10:46 [Note] Shutting down plugin 'binlog'
120318 10:10:46 InnoDB: The InnoDB memory heap is disabled
120318 10:10:46 InnoDB: Mutexes and rw_locks use GCC atomic builtins
120318 10:10:46 InnoDB: Compressed tables use zlib 1.2.3
120318 10:10:46 InnoDB: Using Linux native AIO
120318 10:10:46 InnoDB: CPU supports crc32 instructions
120318 10:10:46 InnoDB: Initializing buffer pool, size = 128.0M
120318 10:10:46 InnoDB: Completed initialization of buffer pool
120318 10:10:46 InnoDB: highest supported file format is Barracuda.
120318 10:10:46 InnoDB: 128 rollback segment(s) are active.
120318 10:10:46 InnoDB: Waiting for the background threads to start
120318 10:10:47 InnoDB: 1.2.4 started; log sequence number 1602841
120318 10:10:47 [Note] Binlog end
120318 10:10:47 [Note] Shutting down plugin 'partition'
120318 10:10:47 [Note] Shutting down plugin 'ARCHIVE'
120318 10:10:47 [Note] Shutting down plugin 'BLACKHOLE'
120318 10:10:47 [Note] Shutting down plugin 'PERFORMANCE_SCHEMA'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN_COLS'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_SYS_FIELDS'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_SYS_COLUMNS'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_SYS_INDEXES'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_SYS_TABLESTATS'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_SYS_TABLES'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_FT_INDEX_TABLE'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_FT_INDEX_CACHE'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_FT_CONFIG'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_FT_BEING_DELETED'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_FT_DELETED'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_FT_INSERTED'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_FT_DEFAULT_STOPWORD'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_METRICS'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_BUFFER_POOL_STATS'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE_LRU'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_CMPMEM_RESET'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_CMPMEM'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_CMP_RESET'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_CMP'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_LOCK_WAITS'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_LOCKS'
120318 10:10:47 [Note] Shutting down plugin 'INNODB_TRX'
120318 10:10:47 [Note] Shutting down plugin 'InnoDB'
120318 10:10:47 InnoDB: FTS optimize thread exiting.
120318 10:10:47 InnoDB: Starting shutdown...
120318 10:10:49 InnoDB: Shutdown completed; log sequence number 1602851
120318 10:10:49 [Note] Shutting down plugin 'CSV'
120318 10:10:49 [Note] Shutting down plugin 'MEMORY'
120318 10:10:49 [Note] Shutting down plugin 'MyISAM'
120318 10:10:49 [Note] Shutting down plugin 'MRG_MYISAM'
120318 10:10:49 [Note] Shutting down plugin 'mysql_old_password'
120318 10:10:49 [Note] Shutting down plugin 'mysql_native_password'
120318 10:10:49 [Note] Shutting down plugin 'binlog'
Why is this bad? Because you don't see at a glance what is right and what is wrong. All the above messages are printed to the standard error, the kind of output that should be reserved for, well, errors! If the standard error is used for regular messages, you may miss the important error messages that are instead mixed with the "all is OK" messages.There is Bug#60934 filed about this issue, but it has been considered a feature request, and as such unlikely to be fixed.In the above text there is something more. THere are warnings, mixed with the standard text, telling me of errors that the bootstrap operation is not in a position to fix, like replication metadata and slave_master_info.MySQL developers, please fix this issue. Users need error messages when there is something wrong, and warning or error messages about something that can actually be fixed. When MySQL 5.6 goes GA, this issue will hit most everybody.

PlanetMySQL Voting: Vote UP / Vote DOWN

Tungsten Replicator 2.0.4 released: usability and power

Сентябрь 7th, 2011
TR 2 0 4It has been a bumpy ride, with dozens of issues opened and resolved, but we finally feel that Tungsten Replicator 2.0.4 is ready for prime time.There have been quite a lot of changes. Most notably, the replicator is much faster, especially when it comes to parallel replication, and it is much easier to install, thanks to its new integrated installer, which can validate all the requirements to install the replicator, and suggest remedies when the requirements aren't met. This new installer is so good, in fact, that calling it installer is an insult. It is a legitimate cluster builder, able to install a full fledged cluster from a central location.
Probably equally important, we have caught up with the documentation, and now you can install several replication topologies following detailed instructions from the docs. You will find both HTML and PDF guides, with the steps to install straight master/slave systems, or direct slave takeover, or bi-directional replication.The binaries are available in the project's Downloads page. Later on, you will find the most updated (and possibly less bug-infested) binaries in our build server list.The Release_Notes list all the issues that have been closed since we released 2.0.3. The advanced users will especially appreciate an innovation introduced in the installer, which now allows users to define one or more of --property=key=value. Using this option wisely, you can now customize the replication properties straight at the start. What used to require several commands and a restart of the replicator right after the installation, now flows smoothly and quickly with one single command.With this release, Tungsten Replicator is closer to become a tool for mass consumption. The old installation method (which we have deprecated and renamed, to discourage anyone from using it) required time, constant attention, and it was unforgiving. The new one will let you make your mistakes freely. If something is amiss anywhere in all the servers where you are installing, it won't install and it will tell you what went wrong. This is probably my favorite feature, because it allows Tungsten to be used by less experienced users.Now it's up to the users. We have no illusion that the product is bug free, and we want to hear from users who try it and report on Issues.

PlanetMySQL Voting: Vote UP / Vote DOWN

Usability improvements in Tungsten Replicator 2.0.4

Август 11th, 2011
If you love a software product, you should try to improve it, and not be afraid of criticizing it. This principle has guided me with MySQL (where I have submitted many usability bugs, and discussed interface with developers for years), and it proves true for Tungsten Replicator as well. When I started working at Continuent, while I was impressed by the technology, I found the installation procedure and the product logs quite discouraging. I would almost say disturbing. Fortunately, my colleagues have agreed on my usability focus, and we can enjoy some tangible improvements. I have already mentioned the new installation procedure, which requires just one command to install a full master/slave cluster. I would like to show how you can use the new installer to deploy a multiple source replication topology like the following: The first step is to install one master in each node. I can run the commands from node #4, which is the one that will eventually receive the updates from the remote masters, and where I need to install the slave services:
TUNGSTEN_BASE=$HOME/newinst

SERVICES=(alpha bravo charlie delta)
REPLICATOR=$TUNGSTEN_BASE/tungsten/tungsten-replicator/bin/replicator

for N in 1 2 3 4
do
INDEX=$(($N-1))

./tools/tungsten-installer \
--master-slave \
--master-host=qa.r$N.continuent.com \
--datasource-user=tungsten \
--datasource-password=secret \
--service-name=${SERVICES[$INDEX]} \
--home-directory=$TUNGSTEN_BASE \
--cluster-hosts=qa.r$N.continuent.com \
--start-and-report
done
The above loop will install a master (remotely or locally) in the four servers. Then I need to create the slave services. To do it, I use the updated configure-service in the tools directory.
TUNGSTEN_TOOLS=$TUNGSTEN_BASE/tungsten/tools

COMMON_OPTIONS='-C -q
--local-service-name=delta
--role=slave
--service-type=remote
--allow-bidi-unsafe=true
--datasource=qa_r4_continuent_com'

$TUNGSTEN_TOOLS/configure-service $COMMON_OPTIONS --master-host=qa.r1.continuent.com alpha
$TUNGSTEN_TOOLS/configure-service $COMMON_OPTIONS --master-host=qa.r2.continuent.com bravo
$TUNGSTEN_TOOLS/configure-service $COMMON_OPTIONS --master-host=qa.r3.continuent.com charlie

$TUNGSTEN_BASE/tungsten/tungsten-replicator/bin/replicator restart
$TUNGSTEN_BASE/tungsten/tungsten-replicator/bin/trepctl services
These commands create the slave services locally in Delta. After restarting the replicator, a simple test will be creating something different in each master, and check that the data has replicated to the single slave. The latest improvement in matter of usability is the simplification of the replicator logs. Until a few days ago, if you had an error in the replicator, you would get a long list of not exactly helpful stuff. For example, if I create a table in a slave, and then create the same table in the master, I will break replication. The extended log would produce something like this:
INFO   | jvm 1    | 2011/08/11 18:10:52 | 2011-08-11 18:10:52,216 [tsandbox - q-to-dbms-0] ERROR pipeline.SingleThreadStageTask Event application failed: seqno=1 fragno=0 message=java.sql.SQLException: Statement failed on slave but succeeded on master

INFO | jvm 1 | 2011/08/11 18:10:52 | 2011-08-11 18:10:52,217 [tsandbox - Event dispatcher thread] ERROR management.OpenReplicatorManager Received error notification, shutting down services: Event application failed: seqno=1 fragno=0 message=java.sql.SQLException: Statement failed on slave but succeeded on master
INFO | jvm 1 | 2011/08/11 18:10:52 | com.continuent.tungsten.replicator.applier.ApplierException: java.sql.SQLException: Statement failed on slave but succeeded on master
INFO | jvm 1 | 2011/08/11 18:10:52 | at com.continuent.tungsten.replicator.applier.MySQLDrizzleApplier.applyStatementData(MySQLDrizzleApplier.java:183)
INFO | jvm 1 | 2011/08/11 18:10:52 | at com.continuent.tungsten.replicator.applier.JdbcApplier.apply(JdbcApplier.java:1233)
INFO | jvm 1 | 2011/08/11 18:10:52 | at com.continuent.tungsten.replicator.applier.ApplierWrapper.apply(ApplierWrapper.java:101)
INFO | jvm 1 | 2011/08/11 18:10:52 | at com.continuent.tungsten.replicator.pipeline.SingleThreadStageTask.runTask(SingleThreadStageTask.java:498)
INFO | jvm 1 | 2011/08/11 18:10:52 | at com.continuent.tungsten.replicator.pipeline.SingleThreadStageTask.run(SingleThreadStageTask.java:155)
INFO | jvm 1 | 2011/08/11 18:10:52 | at java.lang.Thread.run(Unknown Source)
INFO | jvm 1 | 2011/08/11 18:10:52 | Caused by: java.sql.SQLException: Statement failed on slave but succeeded on master
INFO | jvm 1 | 2011/08/11 18:10:52 | at com.continuent.tungsten.replicator.applier.MySQLDrizzleApplier.applyStatementData(MySQLDrizzleApplier.java:139)
INFO | jvm 1 | 2011/08/11 18:10:52 | ... 5 more
INFO | jvm 1 | 2011/08/11 18:10:52 | Caused by: java.sql.SQLSyntaxErrorException: Table 't1' already exists
INFO | jvm 1 | 2011/08/11 18:10:52 | at org.drizzle.jdbc.internal.SQLExceptionMapper.get(SQLExceptionMapper.java:78)
INFO | jvm 1 | 2011/08/11 18:10:52 | at org.drizzle.jdbc.DrizzleStatement.executeBatch(DrizzleStatement.java:930)
INFO | jvm 1 | 2011/08/11 18:10:52 | at com.continuent.tungsten.replicator.applier.MySQLDrizzleApplier.applyStatementData(MySQLDrizzleApplier.java:125)
INFO | jvm 1 | 2011/08/11 18:10:52 | ... 5 more
INFO | jvm 1 | 2011/08/11 18:10:52 | Caused by: org.drizzle.jdbc.internal.common.QueryException: Table 't1' already exists
INFO | jvm 1 | 2011/08/11 18:10:52 | at org.drizzle.jdbc.internal.mysql.MySQLProtocol.executeQuery(MySQLProtocol.java:500)
INFO | jvm 1 | 2011/08/11 18:10:52 | at org.drizzle.jdbc.internal.mysql.MySQLProtocol.executeBatch(MySQLProtocol.java:546)
INFO | jvm 1 | 2011/08/11 18:10:52 | at org.drizzle.jdbc.DrizzleStatement.executeBatch(DrizzleStatement.java:917)
INFO | jvm 1 | 2011/08/11 18:10:52 | ... 6 more
INFO | jvm 1 | 2011/08/11 18:10:52 | 2011-08-11 18:10:52,218 [tsandbox - Event dispatcher thread] WARN management.OpenReplicatorManager Performing emergency service shutdown
INFO | jvm 1 | 2011/08/11 18:10:52 | 2011-08-11 18:10:52,219 [tsandbox - Event dispatcher thread] INFO pipeline.Pipeline Shutting down pipeline: slave
INFO | jvm 1 | 2011/08/11 18:10:52 | 2011-08-11 18:10:52,219 [tsandbox - q-to-dbms-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread
INFO | jvm 1 | 2011/08/11 18:10:52 | 2011-08-11 18:10:52,219 [tsandbox - q-to-dbms-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=0 eventid=mysql-bin.000002:0000000000000426;20
Did you see the reason for the error? No? Neither did I. I would need to open the THL, look for event #1, and determine what it was. Instead, the new user.log looks like this:
2011-08-11 18:10:52,216 ERROR Received error notification: Event application failed: seqno=1 fragno=0 message=java.sql.SQLException: Statement failed on slave but succeeded on master

Caused by : java.sql.SQLException: Statement failed on slave but succeeded on master
Caused by : Statement failed on slave but succeeded on master
Caused by : Table 't1' already exists
Caused by : Table 't1' already exists
2011-08-11 18:10:54,721 INFO State changed ONLINE -> OFFLINE:ERROR
2011-08-11 18:10:54,721 WARN Received irrelevant event for current state: state=OFFLINE:ERROR event=OfflineNotification
That's much better. It is not perfect yet, but it will be soon. Right now, it tells me what is wrong without forcing me to go hunting for it amid hundreds of stack trace lines. Give it a try, using the latest replicator build.

PlanetMySQL Voting: Vote UP / Vote DOWN

Guide to MySQL installation files

Декабрь 1st, 2010

Even for DBAs already familiar with MySQL, the choice of installation methods and the variety of install/package files is overwhelming and confusing.

I’ll make a (very!) brief introduction to the various installation options, concentrating on the Linux operating system, and provide with a simple shopping list.

For illustration, I’ll refer to the MySQL 5.1.52 community edition, the latest at the time of this writing. Downloads are available at http://dev.mysql.com/downloads/mysql/5.1.html.

I will then refer to alternate distributions.

A note for Windows users

You are lucky: your choice is very clear. Download the Windows MSI package. Choose 32 or 64 bit architecture according to your OS. If you know about the other ways to install and setup MySQL under Windows, you don’t need this post.

Linux: repositories

Easiest way to install MySQL on linux would be to use your distro’s repository. Just:

sudo apt-get install mysql-server-5.1

or

sudo yum install MySQL-Server-5.1

Your distro should resolve any package dependencies.

I’m in the opinion that if MySQL is the main application to be used on a server, distribution’s default repositories are not the way to go. Reasons include MySQL outdated version, incapability of installing multiple instance, danger of automatic upgrades or downgrades. Read this post and discussion that follows for more.

Linux: packages

If you’re a RedHat/CentOS/SuSE user, you’re in luck: MySQL provides pre-built RPM packages for your system. I vaguely remember MySQL announcing that Ubuntu is to be supported. That was a couple years ago; there is still no package for Debian/Ubuntu.

So, assuming you’re a RedHat/CentOs user, which packages should you download?

There’s over 60 different RPMs available for download. I always need to stress my eyes to get it right. Here’s the simple answer (again, if you know better, you don’t need this post). Choose “RedHat & Oracle Enterprise Linux”; download:

  • MySQL-client-community-5.1.52-1.rhel5.x86_64.rpm
  • MySQL-server-community-5.1.52-1.rhel5.x86_64.rpm
  • MySQL-shared-community-5.1.52-1.rhel5.x86_64.rpm
  • MySQL-shared-compat-5.1.52-1.rhel5.x86_64.rpm

The above assumes a RedHat/CentOS 5.x and a 64 bit Intel/AMD processor.

The aforementioned post relays my opinion of using RPMs; these are still susceptible to yum‘s whims. Be careful.

Linux: binary

Not afraid to install by hand? Want to avoid limitations introduced by pre-built packages? Download a binary distribution:

  • Choose “Linux – Generic”. Downlaod mysql-5.1.52-linux-x86_64-glibc23.tar.gz

This tar.gz distribution includes server & client. It includes glibc so it does not depend on your OS installed glibc version (a pain to upgrade/downgrade as it is used by so many packages).

You may still want to download and install the shared-compat RPM package (see previous section) to have all possible libmysqlclientX.X packages installed.

Linux: source

If you got here, then you either know your way around (why do you keep reading?) or you have a good reason to use a source distribution.

What good reason could that be?

A more and more common reason is that you want to add something to MySQL. Sphinx search storage engine is such a common addition.

  • Choose “Source Code”: download mysql-5.1.52.tar.gz (Architecture Independent).

Non MySQL downloads

You don’t have to download the official MySQL distribution. Two good alternatives are:


PlanetMySQL Voting: Vote UP / Vote DOWN

Installing MySQL On Mac OS X (Darwin Kernel)

Октябрь 28th, 2010
Recently I happen to install MySQL on Mac OS X (Darvin Kernel).  Below are the quick 5 steps to accomplish the task. Step 1: Check Mac Version Very first step is to verify the Mac OS X’s current version to decide MySQL Installation file to be downloaded. For example for Mac version: 10.4 you should [...] Related posts:
  1. Quick Multi MySQL Server Installation with Master-Master Replication on Same Windows Box This article is a brief step-by-step tutorial on the subject...
  2. 5 useful MySQL Command Options-pager-prompt-rehash-tee-system There are set of commands that MySQL itself interprets. You...
Related posts brought to you by Yet Another Related Posts Plugin.
PlanetMySQL Voting: Vote UP / Vote DOWN

Drupal 7 test drive appliance updated to 7.0-beta2, now with GUI option

Октябрь 26th, 2010

Drupal logoOver the weekend I updated my Drupal 7 test appliance in SUSE Studio to the Drupal 7.0-beta2 release, which was released on Oct. 23rd. I also added phpMyAdmin upon a user request, to provide a web-based method to work with the MySQL instance, if needed.

In addition to the lightweight "headless" appliance (which can only be accessed and configured via a remote network connection), I've now also created a GUI-based version. This appliance starts a minimal GNOME desktop and a Mozilla Firefox browser, which in turn opens the Drupal installation page by default. I hope you will find this useful if you want to toy around and test Drupal 7 without having to go through the entire OS and LAMP stack configuration yourself. In fact, you can even test this appliance via the recently added test drive option from right out of your web browser!

The appliance is now also available in OVF format. SuSE Studio now also builds Amazon EC2 images, which don't seem to be available for download from the SUSE Gallery yet. I assume this is a recent addition to the continuously improving SUSE Studio functionality, hopefully these images will be made available soon.


PlanetMySQL Voting: Vote UP / Vote DOWN