Archive for the ‘setup’ Category

how to setup gitorious on ubuntu server 11.10

Апрель 4th, 2012

In this tutorial I will describe how to setup gitorious on Ubuntu 11.10. Gitorious – a Ruby on Rails web application – can be used to conclude git projects in an easy to manage user interface. In the README of the gitorious repository I found the evidence “One of the main challenges in Gitorious is its installation process. It is anything but trivial.” – It appears correct ;) During the installation I got some errors by the sphinx search engine and some ruby gems. You will find the workarounds in this tutorial.

This tutorial is based on the gitorious – UbuntuInstallation tutorial.

In this setup I will use the hostname gitorious.linux-adm.de – change this hostname in your installation.

software installation

Required packages

apt-get install build-essential zlib1g-dev tcl-dev libexpat-dev \
    libcurl4-openssl-dev postfix apache2 mysql-server mysql-client \
    apg geoip-bin libgeoip1 libgeoip-dev sqlite3 libsqlite3-dev \
    imagemagick libpcre3 libpcre3-dev zlib1g zlib1g-dev libyaml-dev \
    libmysqlclient15-dev apache2-dev libonig-dev ruby-dev rubygems \
    libopenssl-ruby phpmyadmin libdbd-mysql-ruby libmysql-ruby \
    libmagick++-dev  zip unzip memcached git-core git-svn git-doc \
    git-cvs irb libxslt1.1 libxslt1-dev

Ruby, sphinxsearch, stompserver

apt-get install libchronic-ruby librmagick-ruby libdaemons-ruby \
ruby-hoe ruby-echoe libyadis-ruby ruby-mime-types libopenid-ruby \
libdifflcs-ruby libjson-ruby librack-ruby libhmac-ruby rake stompserver \
libapache2-mod-passenger rails sphinxsearch sphinxbase-utils

fetch gitorious

git clone git://gitorious.org/gitorious/mainline.git /var/www/gitorious

install missing ruby gems

cd /var/www/gitorious/
gem install bundler
bundle install

git user

Create a git user with the homedir in /var/www/gitorious/ and set the correct ownership.

adduser --system --home /var/www/gitorious/ --no-create-home --group --shell /bin/bash git
chown git:git -R /var/www/gitorious/

Create a directory for the repositories and a cache directory. Gitorious will add the SSH public keys of the registered users to the authorized_keys file, therefore we have to create a .ssh file and an empty authorized_keys file with the correct permissions.

su - git
git submodule init && \
git submodule update
mkdir -p tmp/pids
mkdir repositories
mkdir tarballs
mkdir .ssh
touch .ssh/authorized_keys
chmod 700 .ssh/
chmod 600 .ssh/authorized_keys

Put gitorious binary on path:

cat >> /var/www/gitorious/.bash_profile <<EOF
export PATH="$PATH:/var/www/gitorious/script/"
EOF

gitorious configuration

prepare configuration files

cd /var/www/gitorious
cp config/database.sample.yml config/database.yml
cp config/gitorious.sample.yml config/gitorious.yml
cp config/broker.yml.example config/broker.yml

create a secret key

Create a secret key for the session cookie. (we need this key later in the next step.

apg -m 32

setup config/gitorious.yml

Setup gitorious in /var/www/gitorious/config/gitorious.yml:


repository_base_path: "/var/www/gitorious/repositories"

cookie_secret: xxx # add a secret key here the command "apg -m 32"

gitorious_client_port should be 80

gitorious_host: gitorious.linux-adm.de # change to your hostname

archive_cache_dir: "/var/www/gitorious/tarballs"

archive_work_dir: "/tmp/tarballs-work"

hide_http_clone_urls: true

is_gitorious_dot_org: false

Create the logroate configuration and the init scripts.

cp /var/www/gitorious/doc/templates/ubuntu/gitorious-logrotate /etc/logrotate.d/

cp /var/www/gitorious/doc/templates/ubuntu/git-ultrasphinx /etc/init.d/
cp /var/www/gitorious/doc/templates/ubuntu/git-daemon /etc/init.d/
cp /var/www/gitorious/doc/templates/ubuntu/git-poller /etc/init.d/

Set in /etc/init.d/git-daemon and in /etc/init.d/git-poller the RUBY_HOME variable to “/usr”

#RUBY_HOME="/opt/ruby-enterprise"
RUBY_HOME="/usr"

Activate the new gitorious init scripts

update-rc.d git-daemon defaults && \
    update-rc.d git-poller defaults && \
    update-rc.d git-ultrasphinx defaults

fix gem spec files

Try to start /etc/init.d/git-ultrasphinx
If you get the following error message you have to fix the spec files:

root@ubuntu03:/var/www/gitorious# /etc/init.d/git-ultrasphinx start
Invalid gemspec in [/var/lib/gems/1.8/specifications/json-1.5.4.gemspec]: invalid date format in specification: "2011-08-31 00:00:00.000000000Z"
Invalid gemspec in [/var/lib/gems/1.8/specifications/childprocess-0.2.1.gemspec]: invalid date format in specification: "2011-08-11 00:00:00.000000000Z"
Invalid gemspec in [/var/lib/gems/1.8/specifications/paperclip-2.4.5.gemspec]: invalid date format in specification: "2011-10-21 00:00:00.000000000Z"
(in /var/www/gitorious)
Invalid gemspec in [/var/lib/gems/1.8/specifications/json-1.5.4.gemspec]: invalid date format in specification: "2011-08-31 00:00:00.000000000Z"
Invalid gemspec in [/var/lib/gems/1.8/specifications/childprocess-0.2.1.gemspec]: invalid date format in specification: "2011-08-11 00:00:00.000000000Z"
Invalid gemspec in [/var/lib/gems/1.8/specifications/paperclip-2.4.5.gemspec]: invalid date format in specification: "2011-10-21 00:00:00.000000000Z"
Invalid gemspec in [/var/lib/gems/1.8/specifications/json-1.5.4.gemspec]: invalid date format in specification: "2011-08-31 00:00:00.000000000Z"
Invalid gemspec in [/var/lib/gems/1.8/specifications/childprocess-0.2.1.gemspec]: invalid date format in specification: "2011-08-11 00:00:00.000000000Z"
Invalid gemspec in [/var/lib/gems/1.8/specifications/paperclip-2.4.5.gemspec]: invalid date format in specification: "2011-10-21 00:00:00.000000000Z"
rake aborted!
Bundler couldn't find some gems. Did you run <code>bundle install</code>?
/var/www/gitorious/Rakefile:4
(See full trace by running task with --trace)

Change the “s.date” in the following spec files:

/var/lib/gems/1.8/specifications/json-1.5.4.gemspec
/var/lib/gems/1.8/specifications/childprocess-0.2.1.gemspec
/var/lib/gems/1.8/specifications/paperclip-2.4.5.gemspec

from:

s.date = %q{2011-08-31 00:00:00.000000000Z}

in:

s.date = %q{2012-08-31}

fix git-ultrasphinx

You have to fix the /var/www/gitorious/config/boot.rb file,
if you get the following error message after starting /etc/init.d/git-ultrasphinx.

root@ubuntu03:/var/www/gitorious# /etc/init.d/git-ultrasphinx start
(in /var/www/gitorious)
rake aborted!
uninitialized constant ActiveSupport::Dependencies::Mutex
/var/www/gitorious/Rakefile:10:in `require'
(See full trace by running task with --trace)

Add the following line in /var/www/gitorious/config/boot.rb.


# Don't change this file!
# Configure your app in config/environment.rb and config/environments/*.rb

require 'thread'

Apache configuration

Disable the default vhost configuration

a2dissite default
a2dissite default-ssl

create a new vhost configuration

cat >> /etc/apache2/sites-available/gitorious <<EOF
<VirtualHost *:80>
    ServerName gitorious.linux-adm.de         # change the hostname!
    DocumentRoot /var/www/gitorious/public
</VirtualHost>
EOF

create a new SSL vhost configuration

cat >> /etc/apache2/sites-available/gitorious-ssl <<EOF
<IfModule mod_ssl.c>
    <VirtualHost _default_:443>
        ServerName gitorious.linux-adm.de   # change the hostname!
        DocumentRoot /var/www/gitorious/public
        SSLEngine on
        SSLCertificateFile    /etc/ssl/certs/ssl-cert-snakeoil.pem
        SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
        BrowserMatch ".*MSIE.*" nokeepalive ssl-unclean-shutdown downgrade-1.0 force-response-1.0
    </VirtualHost>
</IfModule>
EOF

a2ensite gitorious
a2ensite gitorious-ssl
a2enmod ssl   # enable SSL

create MySQL User for gitorious

mysql -u root -p
GRANT ALL PRIVILEGES ON *.* TO 'gitorious'@'localhost' IDENTIFIED BY 'XXXX' WITH GRANT OPTION;
FLUSH PRIVILEGES;

change the password XXXX!

Add the MySQL User to the production section in /var/www/gitorious/config/database.yml


production:
adapter: mysql
database: gitorious_production
username: gitorious
password: XXXX # change the password!
host: localhost
encoding: utf8

Starting gitorious

Run the following commands as user git.

export RAILS_ENV=production && 
bundle exec rake db:create && 
bundle exec rake db:migrate && 
bundle exec rake ultrasphinx:bootstrap

fix ultrasphinx/production.conf

If you get the following error message, you have to fix the /var/www/gitorious/ultrasphinx/production.conf file.

export RAILS_ENV=production && 
bundle exec rake db:create && 
bundle exec rake db:migrate && 
bundle exec rake ultrasphinx:bootstrap

Sphinx 0.9.9-release (r2117)
Copyright (c) 2001-2009, Andrew Aksyonoff

using config file '/var/www/gitorious/config/ultrasphinx/production.conf'...
WARNING: key 'address' is deprecated in /var/www/gitorious/config/ultrasphinx/production.conf line 11; use 'listen' instead.
listening on all interfaces, port=3312
WARNING: index 'main': preload: failed to open /var/www/gitorious/db/sphinx//sphinx_index_main.sph: No such file or directory; NOT SERVING
FATAL: no valid indexes to serve
Failed to start
Done
Please restart your application containers

Change in /var/www/gitorious/config/ultrasphinx/production.conf the word “base_tags” to “tags”.

git-ultrasphinx cronjob

Add the following cronjob for the git user.

crontab -e -u git


* * * * * cd /var/www/gitorious && /usr/local/bin/bundle exec rake ultrasphinx:index RAILS_ENV=production

create an admin user

git@ubuntu03:/var/www/gitorious$ env RAILS_ENV=production ruby1.8 script/create_admin
Type in Administrator's e-mail: 
admin@gitorious.linux-adm.de
Type in Administrator's password: 
test
Admin user created successfully.

After a reboot of your system the gitorious web front-end should be available.


PlanetMySQL Voting: Vote UP / Vote DOWN

how to setup gitorious on ubuntu server 11.10

Апрель 4th, 2012

In this tutorial I will describe how to setup gitorious on Ubuntu 11.10. Gitorious – a Ruby on Rails web application – can be used to conclude git projects in an easy to manage user interface. In the README of the gitorious repository I found the evidence “One of the main challenges in Gitorious is its installation process. It is anything but trivial.” – It appears correct ;) During the installation I got some errors by the sphinx search engine and some ruby gems. You will find the workarounds in this tutorial.

This tutorial is based on the gitorious – UbuntuInstallation tutorial.

In this setup I will use the hostname gitorious.linux-adm.de – change this hostname in your installation.

software installation

Required packages

apt-get install build-essential zlib1g-dev tcl-dev libexpat-dev \
    libcurl4-openssl-dev postfix apache2 mysql-server mysql-client \
    apg geoip-bin libgeoip1 libgeoip-dev sqlite3 libsqlite3-dev \
    imagemagick libpcre3 libpcre3-dev zlib1g zlib1g-dev libyaml-dev \
    libmysqlclient15-dev apache2-dev libonig-dev ruby-dev rubygems \
    libopenssl-ruby phpmyadmin libdbd-mysql-ruby libmysql-ruby \
    libmagick++-dev  zip unzip memcached git-core git-svn git-doc \
    git-cvs irb libxslt1.1 libxslt1-dev

Ruby, sphinxsearch, stompserver

apt-get install libchronic-ruby librmagick-ruby libdaemons-ruby \
ruby-hoe ruby-echoe libyadis-ruby ruby-mime-types libopenid-ruby \
libdifflcs-ruby libjson-ruby librack-ruby libhmac-ruby rake stompserver \
libapache2-mod-passenger rails sphinxsearch sphinxbase-utils

fetch gitorious

git clone git://gitorious.org/gitorious/mainline.git /var/www/gitorious

install missing ruby gems

cd /var/www/gitorious/
gem install bundler
bundle install

git user

Create a git user with the homedir in /var/www/gitorious/ and set the correct ownership.

adduser --system --home /var/www/gitorious/ --no-create-home --group --shell /bin/bash git
chown git:git -R /var/www/gitorious/

Create a directory for the repositories and a cache directory. Gitorious will add the SSH public keys of the registered users to the authorized_keys file, therefore we have to create a .ssh file and an empty authorized_keys file with the correct permissions.

su - git
git submodule init && \
git submodule update
mkdir -p tmp/pids
mkdir repositories
mkdir tarballs
mkdir .ssh
touch .ssh/authorized_keys
chmod 700 .ssh/
chmod 600 .ssh/authorized_keys

Put gitorious binary on path:

cat >> /var/www/gitorious/.bash_profile <<EOF
export PATH="$PATH:/var/www/gitorious/script/"
EOF

gitorious configuration

prepare configuration files

cd /var/www/gitorious
cp config/database.sample.yml config/database.yml
cp config/gitorious.sample.yml config/gitorious.yml
cp config/broker.yml.example config/broker.yml

create a secret key

Create a secret key for the session cookie. (we need this key later in the next step.

apg -m 32

setup config/gitorious.yml

Setup gitorious in /var/www/gitorious/config/gitorious.yml:


repository_base_path: "/var/www/gitorious/repositories"

cookie_secret: xxx # add a secret key here the command "apg -m 32"

gitorious_client_port should be 80

gitorious_host: gitorious.linux-adm.de # change to your hostname

archive_cache_dir: "/var/www/gitorious/tarballs"

archive_work_dir: "/tmp/tarballs-work"

hide_http_clone_urls: true

is_gitorious_dot_org: false

Create the logroate configuration and the init scripts.

cp /var/www/gitorious/doc/templates/ubuntu/gitorious-logrotate /etc/logrotate.d/

cp /var/www/gitorious/doc/templates/ubuntu/git-ultrasphinx /etc/init.d/
cp /var/www/gitorious/doc/templates/ubuntu/git-daemon /etc/init.d/
cp /var/www/gitorious/doc/templates/ubuntu/git-poller /etc/init.d/

Set in /etc/init.d/git-daemon and in /etc/init.d/git-poller the RUBY_HOME variable to “/usr”

#RUBY_HOME="/opt/ruby-enterprise"
RUBY_HOME="/usr"

Activate the new gitorious init scripts

update-rc.d git-daemon defaults && \
    update-rc.d git-poller defaults && \
    update-rc.d git-ultrasphinx defaults

fix gem spec files

Try to start /etc/init.d/git-ultrasphinx
If you get the following error message you have to fix the spec files:

root@ubuntu03:/var/www/gitorious# /etc/init.d/git-ultrasphinx start
Invalid gemspec in [/var/lib/gems/1.8/specifications/json-1.5.4.gemspec]: invalid date format in specification: "2011-08-31 00:00:00.000000000Z"
Invalid gemspec in [/var/lib/gems/1.8/specifications/childprocess-0.2.1.gemspec]: invalid date format in specification: "2011-08-11 00:00:00.000000000Z"
Invalid gemspec in [/var/lib/gems/1.8/specifications/paperclip-2.4.5.gemspec]: invalid date format in specification: "2011-10-21 00:00:00.000000000Z"
(in /var/www/gitorious)
Invalid gemspec in [/var/lib/gems/1.8/specifications/json-1.5.4.gemspec]: invalid date format in specification: "2011-08-31 00:00:00.000000000Z"
Invalid gemspec in [/var/lib/gems/1.8/specifications/childprocess-0.2.1.gemspec]: invalid date format in specification: "2011-08-11 00:00:00.000000000Z"
Invalid gemspec in [/var/lib/gems/1.8/specifications/paperclip-2.4.5.gemspec]: invalid date format in specification: "2011-10-21 00:00:00.000000000Z"
Invalid gemspec in [/var/lib/gems/1.8/specifications/json-1.5.4.gemspec]: invalid date format in specification: "2011-08-31 00:00:00.000000000Z"
Invalid gemspec in [/var/lib/gems/1.8/specifications/childprocess-0.2.1.gemspec]: invalid date format in specification: "2011-08-11 00:00:00.000000000Z"
Invalid gemspec in [/var/lib/gems/1.8/specifications/paperclip-2.4.5.gemspec]: invalid date format in specification: "2011-10-21 00:00:00.000000000Z"
rake aborted!
Bundler couldn't find some gems. Did you run <code>bundle install</code>?
/var/www/gitorious/Rakefile:4
(See full trace by running task with --trace)

Change the “s.date” in the following spec files:

/var/lib/gems/1.8/specifications/json-1.5.4.gemspec
/var/lib/gems/1.8/specifications/childprocess-0.2.1.gemspec
/var/lib/gems/1.8/specifications/paperclip-2.4.5.gemspec

from:

s.date = %q{2011-08-31 00:00:00.000000000Z}

in:

s.date = %q{2012-08-31}

fix git-ultrasphinx

You have to fix the /var/www/gitorious/config/boot.rb file,
if you get the following error message after starting /etc/init.d/git-ultrasphinx.

root@ubuntu03:/var/www/gitorious# /etc/init.d/git-ultrasphinx start
(in /var/www/gitorious)
rake aborted!
uninitialized constant ActiveSupport::Dependencies::Mutex
/var/www/gitorious/Rakefile:10:in `require'
(See full trace by running task with --trace)

Add the following line in /var/www/gitorious/config/boot.rb.


# Don't change this file!
# Configure your app in config/environment.rb and config/environments/*.rb

require 'thread'

Apache configuration

Disable the default vhost configuration

a2dissite default
a2dissite default-ssl

create a new vhost configuration

cat >> /etc/apache2/sites-available/gitorious <<EOF
<VirtualHost *:80>
    ServerName gitorious.linux-adm.de         # change the hostname!
    DocumentRoot /var/www/gitorious/public
</VirtualHost>
EOF

create a new SSL vhost configuration

cat >> /etc/apache2/sites-available/gitorious-ssl <<EOF
<IfModule mod_ssl.c>
    <VirtualHost _default_:443>
        ServerName gitorious.linux-adm.de   # change the hostname!
        DocumentRoot /var/www/gitorious/public
        SSLEngine on
        SSLCertificateFile    /etc/ssl/certs/ssl-cert-snakeoil.pem
        SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
        BrowserMatch ".*MSIE.*" nokeepalive ssl-unclean-shutdown downgrade-1.0 force-response-1.0
    </VirtualHost>
</IfModule>
EOF

a2ensite gitorious
a2ensite gitorious-ssl
a2enmod ssl   # enable SSL

create MySQL User for gitorious

mysql -u root -p
GRANT ALL PRIVILEGES ON *.* TO 'gitorious'@'localhost' IDENTIFIED BY 'XXXX' WITH GRANT OPTION;
FLUSH PRIVILEGES;

change the password XXXX!

Add the MySQL User to the production section in /var/www/gitorious/config/database.yml


production:
adapter: mysql
database: gitorious_production
username: gitorious
password: XXXX # change the password!
host: localhost
encoding: utf8

Starting gitorious

Run the following commands as user git.

export RAILS_ENV=production && 
bundle exec rake db:create && 
bundle exec rake db:migrate && 
bundle exec rake ultrasphinx:bootstrap

fix ultrasphinx/production.conf

If you get the following error message, you have to fix the /var/www/gitorious/ultrasphinx/production.conf file.

export RAILS_ENV=production && 
bundle exec rake db:create && 
bundle exec rake db:migrate && 
bundle exec rake ultrasphinx:bootstrap

Sphinx 0.9.9-release (r2117)
Copyright (c) 2001-2009, Andrew Aksyonoff

using config file '/var/www/gitorious/config/ultrasphinx/production.conf'...
WARNING: key 'address' is deprecated in /var/www/gitorious/config/ultrasphinx/production.conf line 11; use 'listen' instead.
listening on all interfaces, port=3312
WARNING: index 'main': preload: failed to open /var/www/gitorious/db/sphinx//sphinx_index_main.sph: No such file or directory; NOT SERVING
FATAL: no valid indexes to serve
Failed to start
Done
Please restart your application containers

Change in /var/www/gitorious/config/ultrasphinx/production.conf the word “base_tags” to “tags”.

git-ultrasphinx cronjob

Add the following cronjob for the git user.

crontab -e -u git


* * * * * cd /var/www/gitorious && /usr/local/bin/bundle exec rake ultrasphinx:index RAILS_ENV=production

create an admin user

git@ubuntu03:/var/www/gitorious$ env RAILS_ENV=production ruby1.8 script/create_admin
Type in Administrator's e-mail: 
admin@gitorious.linux-adm.de
Type in Administrator's password: 
test
Admin user created successfully.

After a reboot of your system the gitorious web front-end should be available.


PlanetMySQL Voting: Vote UP / Vote DOWN

MySQL University: Dual Master Setups With MMM

Октябрь 21st, 2009

This Thursday (October 22nd, 13:00 UTC), Walter Heck will present Dual Master Setups With MMM. MMM (Multi-Master Replication Manager for MySQL) is a set of flexible scripts to perform monitoring/failover and management of MySQL master-master replication configurations (with only one node writable at any time). The toolset also has the ability to read balance standard master/slave configurations with any number of slaves, so you can use it to move virtual IP addresses around a group of servers depending on whether they are behind in replication. For more information, see http://mysql-mmm.org/.

For MySQL University sessions, point your browser to this page. You need a browser with a working Flash plugin. You may register for a Dimdim account, but you don't have to. (Dimdim is the conferencing system we're using for MySQL University sessions. It provides integrated voice streaming, chat, whiteboard, session recording, and more.) All MySQL University sessions are recorded, that is, slides and voice can be viewed as a Flash movie (.flv). You can find those recordings on the respective MySQL University session pages which are listed on the MySQL University home page.

MySQL University is a free educational online program for engineers/developers. MySQL University sessions are open to anyone, not just Sun employees. Sessions are recorded (slides and audio), so if you can't attend the live session you can look at the recording anytime after the session.

Here's the schedule for the rest of this year:

  • October 29: MySQL scalability on SPARC & INTEL X5500 (Nehalem) (Benoit Chaffanjon)
  • November 12: Gearman for MySQL (Giuseppe Maxia)
  • November 19: memcached Functions for MySQL (UDFs) (Patrick Galbraith)
  • December 3: Practical Full-Text Search in MySQL (Bill Karwin)

The schedule is not engraved in stone at this point. Please visit http://forge.mysql.com/wiki/MySQL_University#Upcoming_Sessions for the up-to-date list. On that page, you can also find the starting time for many time zones.


PlanetMySQL Voting: Vote UP / Vote DOWN