See part 1 for the introduction and talking about power and hardware. This part will go over the 2nd “P”, partitioning. Learning about Oracle’s partitioning has gotten me more interested in how MySQL’s partitioning works, and I do hope that MySQL partitioning will develop to the level that Oracle partitioning does, because Oracle’s partitioning looks very nice (then again, that’s why it costs so much I guess).

Partition – Larger tables or fact tables can benefit from partitioning because it makes data load easier and can increase join performance and use data elimination. Parallel execution can be done with partitioning due to partition pruning. The degree of parallelism should be a power of 2, because of hash-based algorithm in hash partitioning. To translate this to the MySQL world, if you are using LINEAR HASH partitioning, then you should use a degree of parallelism that is a power of 2 (I checked, and indeed. Otherwise, use a degree of parallelism that makes sense given the number of partitions you have.

One important note that during Pythian’s testing of MySQL partitioning, we found that all partitions were locked when an INSERT occurs, for the duration of the INSERT. Bulk-loading with MySQL partitioning is not as fast as it would be if MySQL allowed partition pruning for INSERTs.

So, what should be partitioned? For the first level of partitioning, the goal is to enable partitioning pruning and simplify data management. The most typical partitioning is range or interval partitioning on a date column. Interval partitioning is you say what the partition is (date, month) and partition is automatically created. MySQL does not have interval partitioning, and I have seen typical first-level partitioning be range or list based on a date or timestamp column. Note that if you use a timestamp field, the partitioning expression is optimized if you use TO_DAYS(timestamp_field) or YEAR(timestamp_field). In my experience, using anything else (such as DATE(timestamp_field)) actually makes partitioning slower than not using partitioning at all. Note that this is based on tests I did a few months ago, and your mileage may vary.

So — how do you decide partitioning strategy? Ask yourself:

• What range of data do the queries touch – a quarter, a year?
  
• What is the data loading frequency?
  
• Is an incremental load required?
  
• How much data is involved, a day, a week, a month?

The answers to the above questions will tell you about how big your interval needs to be. The best scenario is that all answers are the same, “we load every day, and people query by day.” If the answers are different weight access a higher priority than loading, because most people care more about query performance than performance of ETL.

This is true even if your intervals have different sizes — ie sales per day are much bigger in Dec but that’s OK. However, Maria recommends that the subpartition be as evenly divided as possible.

Easier to look at more partitions than to look at a partition that’s too big. But you don’t want too many partitions, max Oracle allows partitions is 1 million partitions, prior to 11g it was 64,000. “Stick closer to 64,000 than 1 million”. MySQL’s limitation is 1024 per table.

For the second level of partitioning, also called subpartitioning, the goal is to allow for multi-level pruning and improve join performance. In Oracle, the most typical subpartition is hash or list – in MySQL, you can only subpartition by hash or key.

How do you decide subpartitioning strategy?

• Select the dimension queried most frequently on the fact table OR
  
• Pick the common join column

For example, if you want to look at sales per day, per store, you would choose “per day” as the partition and “per store” as the subpartition.

If you do not have a good partition on logical elements (like grouping), then you can subpartition using hash partitioning on common joins — perhaps surrogate keys, or using join key of the largest table involved in the join.

For example, if the sales table is partitioned and another big table is product, you can hash subpartition product_id.

Because there’s overhead in partitions (loading metadata, reading metadata), make sure size of partitions and subpartitions is >20 Mb. So better to have a 30 Mb subpartition than a 15 Mb subpartition. [I have no idea if this is true in MySQL or not -- I think the general concept is true, because there is some overhead, but I have no idea about the 20 Mb figure and why that's true for Oracle, nor do I know what is true in MySQL.]

One easy calculation is double the # of CPUs, round up to nearest power of 2. If you’re executing in parallel, Oracle will use 2x CPUs. (all this advice, by the way, follows 80/20 rule, this is probably good for about 80% of the environments out there). Of course, MySQL does not do parallel execution very well, so this probably does not apply.

Oracle knows it can get partition elimination while it does a join.

If 2 tables have the same degree of parallelism (same # of buckets) and are partitioned in the same way on the join column (say, customer_id in a subpartition of sales and a partition of customer), Oracle will match the partitions when joining:

sales table joined with customer table can change into 4 small joins:
sales sub part 1 joins with customer part 1
sales sub part 2 joins with customer part 2
sales sub part 3 joins with customer part 3
sales sub part 4 joins with customer part 4

And with parallelism, the total time is now reduced to the time it takes to do one of those smaller joins.

This is also why you want to have a power of 2 for buckets – because cores/processors come in powers of 2. Partition-wise joins like this can also be done with range or list, assuming both tables in the join have the same buckets.

I have no idea if MySQL partitioning works this way, but it’s certainly a functionality that makes sense to me.

These are my notes from the session, which include comparisons of how Oracle works (which Maria gave) and how MySQL works (which I researched to figure out the difference, which is why this blog post took a month after the conference to write). Note that I am not an expert on data warehousing in either Oracle or MySQL, so these are more concepts to think about than hard-and-fast advice. In some places, I still have questions, and I am happy to have folks comment and contribute what they know.

One interesting point brought up:
Maria quoted someone (she said the name but I did not grab it) from Forrester saying, “3NF is typically a selfless model used by Enterprise data warehouse, which is used by the whole company. A star schema is a selfish model, used by a department, because it’s already got aggregation in it.”

I thought that was an interesting way of pointing that out — most people do not understand why 3NF is not good enough for data warehousing, and I have had a hard time explaining why a star or snowflake schema should be used. Another schema-related topic I had a hard time putting into words before this workshop was the difference between a star and a snowflake schema: compared to a star schema, in a snowflake schema, you have more than one fact table and maybe some dimensions that are not used often.

From Maria and the slides:
“Oracle says model what will suit your business best. Don’t get lost in academia. Most schemas are not 100% according to the theoretical
models. Some examples: 3NF schema with denormalized attributes to avoid costly joins, Star schema with multiple hierarchies in same fact table.”

Data warehousing has a 3-step approach —

1) data sources -> staging layer (temp loading layer)
2) staging layer (temp loading layer)-> foundation (logical, data store) layer
3) foundation (logical, data store) layer -> access and performance layer

The foundation layer is usually 3NF the access layer is usually a star or snowflake schema. As for the data sources, they can be varied, you would hope that they are in 3NF (and if they are you can skip the first 2 steps) but they are not always that way.

The 3 P’s of best practice for data warehousing (on Oracle) are power, partitioning, parallelism. The goal of the data warehousing environment is to minimize the amount of data accessed and use the most efficient joins – so it is not so index focused. This may be based on Oracle’s way of doing joins, I am not so sure if it applies to MySQL as well.

Power The weakest link in the chain (the 3 steps above) will define the throughput, so make sure your hardware configuration is balanced. Maria mentioned that as DBAs, “most of the time we don’t have control over this, but we’re still bound to the SLAs.”

This includes hardware that immediately comes to mind such as # of CPUs/cores, speed of CPU, amount of RAM, speed of disk as well as what we may not think of immediately: speed of network switches, speed of disk controllers, number and speed of host BUS adapters. Notes on host BUS adapters (HBAs): Know the # of HBA ports you have. 4 Gb HBA does 400 Mb/sec. 2 Gb HBA does 200 Mb/sec. Make sure there’s enough HBA capacity to sustain the CPU throughput (ie, make sure HBA isn’t the bottleneck). Also the speed at which it all talks. If you have a 4 Gb machine but a 2 Gb switch, you end up having 2 Gb throughput. Upgrade the network at the same time you upgrade machines.

Because we are talking about data warehousing, it is often not possible to eliminate disk I/O, so the goal is to have the fastest I/O throughput possible. Data warehouses need to be sized on I/O throughput not number of I/O’s.

I made a post earlier about how to determine I/O throughput for a system, which used information from this session. Justin Swanhart already pointed out that this is based on the fact that Oracle can do hash joins and MySQL can only do nested loop joins. I wonder, though, if there is indeed no case when using MySQL for which I/O throughput is a more useful metric than iops.

Disk arrays that are expensive are usually sized for iops, not throughput, and because they’re expensive the disk array is shared throughout the company. A DBA needs to ask ‘how many connections into the storage array do I have? How many disk controllers do I have? Where are my physical disks, and which controllers are they hanging off of?’

Typical 15k rpm disk can do about 25-35 Mb/sec (per disk) random i/o’s. Disk manufacturers will throw out numbers like 200-300 Mb/sec but that’s sequential I/O and leading edge of the drive. Make sure all your LUNs are not coming off the same set of disks, so that you’re not conflicting on disk seeks.

Continue to part 2, partitioning.

The section contained an Oracle-specific in-database tool, and a standalone tool that can be used on many operating systems, regardless of whether or not a database exists:

If Oracle is installed, run DBMS_RESOURCE_MANAGER.CALIBRATE_IO:

SET SERVEROUTPUT ON
DECLARE
lat INTEGER;
iops INTEGER;
mbps INTEGER;
BEGIN
-- DBMS_RESOURCE_MANAGER.CALIBRATE_IO(<DISKS>, <MAX_LATENCY>,iops,mbps,lat);
DBMS_RESOURCE_MANAGER.CALIBRATE_IO (2, 10, iops, mbps, lat);
dbms_output.put_line('max_mbps = ' || mbps);
end;

For us MySQL folks, or even the Drizzle or NoSQL folks, Oracle offers a free standalone tool called Orion. The example given in the slides was:

./orion –run advanced –testname mytest –num_small 0 –size_large 1024 –type rand –simulate contact –write 0 –duration 60 –matrix column

-num_small is 0 because you don’t usually do small transactions in a dw.
-type rand for random I/O’s because data warehouse queries usually don’t do sequential reads
-write 0 – no writes, because you do not write often to the dw, that is what the ETL is for.
-duration is in seconds
-matrix column shows you how much you can sustain

I would be interested to see how other folks measure I/O throughput, and maybe even do a side-by-side comparison of different tools. Orion is available for:

Linux (x86, x86-64, Itanium, Power)
Solaris (SPARC64)
AIX (PPC64)
zLinux
HPUX (PA RISC, Itanium)
Windows

I am working on a larger write-up of the session itself, which had many concise descriptions of data warehousing issues, but I thought that this merited its own post.

The short answer is No.

There is clear evidence that in the short to medium term Oracle will continue to promote and enhance MySQL. Some of these indicators include:

• EU 10 point commitment in December 2009 – See Oracle Makes Commitments to Customers, Developers and Users of MySQL
• MySQL Conference April 2010 – Opening keynote by Edward Screven State of the Dolphin
• Oracle Magazine Jul/Aug 2010 – Interview with Edward Screven Open for Business.

It is clear from these sources that Oracle intends to incorporate MySQL into Oracle Backup and Security Vault products. Both a practical and necessary step. There is also a clear mention of focusing on the Microsoft platform, a clear indicator that SQL Server is in their sights without actually saying it.

What is unknown is exact how and when features will be implemented. Also important is how much these may cost the end user. Oracle is in the business of selling, now an entire H/W and S/W stack. They also have a complicated pricing model of different components with product offerings. I assume this will continue. There are already two indications, InnoDBbackup included for Enterprise Backup (from April Keynote) and 5.1 enterprise split. (Note: while this split may have existed prior to Oracle, it is now more clearly obvious).

MySQL can never be seen as drawing away from any Oracle sales of the core entry level database product. It is likely Oracle will provide a SQL Syntax compatibility layer for MySQL within 2 years, however it will I’m sure be a commercial add-on. Likewise, I would suspect a PL/SQL lite layer within 5 years, but again at a significant cost to offset the potential loss of sales in the low end of the server market. There continues to be active development in the MySQL Enterprise Monitor, MySQL Workbench and MySQL Connectors which is all excellent news for users.

Moving forward, how long will this ancillary development of free tools continue? What will happen to the commercial storage engine, OEM and licensing model after the 5 year commitment? How will the MySQL ecosystem survive.? There is active development in Percona, MariaDB and Drizzle forks, however unless all players that want to provide a close MySQL compatible solution work together, progress will continue to be a disappointing disjointed approach. The 2011 conference season will also see a clear line with competing MySQL conferences in April scheduled at the same time, the O’Reilly MySQL conference in Santa Clara California and the Oracle supported^(*) Collaborate 2011 in Orlando, Florida.

I have a number of predictions on what Oracle ME MySQL may look like in 5 years however this is a topic for a personal discussion.

For your reading pleasure this week we have Log Buffer #196:

Charles Hooper blogs about an in-depth investigation on what can cause Oracle to ignore a hint.

Doug Burns reminds his readers that there are only two weeks left to submit papers for UKOUG. The deadline is Aug. 2.

A while back Greg Rahn at Structured Data blog thought that the best way to get results out of Exadata is by changing your application to get the most out of Exadata. He was very happy to see that Pythian thinks the same. On the subject of Exadata and data warehousing, Greg posts this week on the core performance fundamentals of Oracle Data Warehousing – set processing vs row processing.

Jonathan Lewis links to instructions and explanations on how to switch to a different UNDO tablespace. It is trickier than it sounds and Jonathan provided additional traps to watch out for.

Jonathan also continues his fragmentation series with an explanation of table fragmentation and its causes.

Alex Fatkulin explains about ASM mirroring and disk partnership and why you may have less redundancy than you thought.

On the same subject, Jeremy Schneider of Ardent Performance blog explains about hot disks, raid and what it means for ASM mirroring.

Back to blogging after a recent trip to TechInsights 2010, Edwin Sarmiento answers questions on what needs to be done as part of the installation of a SQL Server 2008 R2 Failover Cluster on Windows Server 2008 R2. A number of things related to Windows Clustering need to be considered.

Willie Favero introduces the “IBM zEnterprise System”, on his blog Getting the Most out of DB2 for z/OS and System z.

On Join-fu! the Art of SQL blog, Jay Pipes talks about getting started developing Nova on Linux, as he’s involved in a new OpenStack project.

Peter Zaitsev, on MySQL Performance Blog, posts about estimating replication capacity so that replication load can be dealt with before slave is unable to catch up.

Paul Randal publishes his survey results around the purchase and use of SSIDs.

And, if you happen to be attending Oracle OpenWorld, register before July 30 to take advantage of early bird rates.

Remember if you find a link or interesting blog post that you think Log Buffer should mention, send a note to the editor at Log Buffer and be sure to include the link, and a short note on why you think that others will want to read it too.

Now on to Log Buffer #195. Alex Gorbachev starts us off with his suggested readings and funnily enough, Chen Shapira had many of the same favorites this week.

Jonathan Lewis introducing a new series about fragmentation. In this post he defines what he means by fragmentation. Alex is looking forward to reading the next bits.

Charles Hooper shows how to present performance data in visually pleasing charts.

Mark Calahan, on his High Availability MySQL blog announces a new book on MySQL replication and high availability from Charles, Mat and Lars, stating it’s a good read to save people from some inevitable failures when distributed systems are deployed for enought time over many servers.

Tanel Poder, back blogging full force shows what really happens when you drop a table, and why you can drop tables from a read-only tablespace.

Maria Colgan from Oracle Optimizer Team drills into details of CURSOR_SHARING. By the way, Oracle Optimizer blog has moved to the official Oracle blogs as Doug Burns noted. And he couldn’t resist mentioning another post from Maria Colgan about moving from the old Rule-Based Optimizer to the Cost-based Optimizer. Alex saw Maria speak at ODTUG/Kaleidoscope and since then follows her blog all the time. There are lots of other recently posted materials in the new location so read on.

Randolf Geist, Oracle ACE, and member of the OakTable Network, posted a monstrous (size-wise) article about limitations of index and table compression covering Oracle release from 9i up to the latest 11.2. Very thorough!

Michael Janke complains about the number of severe bugs in Oracle and tells some amusing anecdotes about defective parts in the industry.

Michelle Gutzait suggests a new free eBook available now on how to do performance tuning with Data Management Views in SQL Server.

Joshua Drake notes the call for papers for PgWest 2010 is now open and provides the link to submit your talk. Deadline for submission is September 5, 2010 and speakers will be notified the following week. They’re looking for general PostgreSQL and stack topics and tutorials.

Until next week.

We had speculated on the fate of Sun open source software under Oracle and while we acknowledged Oracle’s participation in, contribution and commitment to and opportunity from open source software, we questioned its appreciation of open source software communities beyond code and customers. It appears the OpenSolaris community and thus the OS itself, which we believe is key to advancing development of the more popular, proprietary cousin Solaris — are not a priority for Oracle.

The same cannot be said for all open source from Sun, and there’s a lot of it, now at Oracle. Amid the struggles of the OpenSolaris community, one of the other open source keystones from Sun, MySQL, seems to be doing well, despite persisting claims Oracle purchased Sun and MySQL simply to keep it from competing with Oracle database products. According to a Jaspersoft survey of customers/developers, there is a lack of awareness or concern of Oracle’s involvement in MySQL (59 percent were not aware Oracle reorganized and established a separate MySQL business unit apart from Oracle’s traditional RDBMS business …). Another 43% of Jaspersoft’s respondents said MySQL development and innovation would improve under Oracle.

The Jaspersoft survey found even more love for Java under Oracle, with 80 percent of respondents indicating they believe the Java process will improve or stay the same under Oracle. The related GlassFish application server also appears to be healthy with both community and commercial versions recently released.

The OpenOffice community appears also to be continuing forward supported and unfettered by Oracle (perhaps because it was typically fettered by Sun?), but it may also me failing to fully seize the opportunity.

It has also been interesting to see how Sun’s cloud computing technology has helped give Oracle new love for the term and the market.

There are a number of key open source projects and pieces from Sun, those listed above as well as many others, that may be on the line right now (or may have already been branded ’stay’ or ’stop’). We will be watching to see how Sun’s open source continues to shine or to set at Oracle.

The first tool in the dbForge for Oracle product line will be Data Compare. For the first release we decided to make a free tool with basic functionality:

• comparison and synchronization of tables only (views are not supported)
• table and column mapping (by name)
• type conversion is not supported (column types must match)
• only simple types will be supported

Tool intended to help accomplishing simple database synchronization tasks and we plan to keep it free while adding new features to the commercial version.

Release is scheduled at the first decade of August 2010.

After gaining some experience in SQL Server data compare tools development we decided to release

User Management

By default there will be a new mysql user and group created. This user is used to run the mysqld process is generally not used for any other purpose.

$ grep mysql /etc/{passwd,shadow,group}
/etc/passwd:mysql:x:27:27:MySQL Server:/var/lib/mysql:/bin/bash
/etc/shadow:mysql:!!:14796::::::
/etc/group:mysql:x:27:

Binaries

MySQL binaries are found in /usr/bin.

$ ls -l /usr/bin/mysql*
-rwxr-xr-x 1 root root  314568 Feb 16 17:45 /usr/bin/mysql
-rwxr-xr-x 1 root root  110776 Feb 16 14:39 /usr/bin/mysqlaccess
-rwxr-xr-x 1 root root   35144 Feb 16 17:45 /usr/bin/mysqladmin
-rwxr-xr-x 1 root root  112944 Feb 16 17:45 /usr/bin/mysqlbinlog
-rwxr-xr-x 1 root root    7632 Feb 16 17:45 /usr/bin/mysqlbug
-rwxr-xr-x 1 root root   30576 Feb 16 17:45 /usr/bin/mysqlcheck
-rwxr-xr-x 1 root root    7632 Feb 16 17:45 /usr/bin/mysql_config
-rwxr-xr-x 1 root root    3670 Feb 16 17:44 /usr/bin/mysql_convert_table_format
-rwxr-xr-x 1 root root   22522 Feb 16 17:44 /usr/bin/mysqld_multi
-rwxr-xr-x 1 root root   13073 Feb 16 17:44 /usr/bin/mysqld_safe
-rwxr-xr-x 1 root root   75184 Feb 16 17:45 /usr/bin/mysqldump
-rwxr-xr-x 1 root root    6356 Feb 16 17:44 /usr/bin/mysqldumpslow
-rwxr-xr-x 1 root root   11648 Feb 16 17:44 /usr/bin/mysql_explain_log
-rwxr-xr-x 1 root root    3245 Feb 16 14:39 /usr/bin/mysql_find_rows
-rwxr-xr-x 1 root root     483 Feb 16 17:44 /usr/bin/mysql_fix_extensions
-rwxr-xr-x 1 root root    5834 Feb 16 17:44 /usr/bin/mysql_fix_privilege_tables
-rwxr-xr-x 1 root root   31431 Feb 16 17:44 /usr/bin/mysqlhotcopy
-rwxr-xr-x 1 root root   26160 Feb 16 17:45 /usr/bin/mysqlimport
-rwxr-xr-x 1 root root   13659 Feb 16 17:44 /usr/bin/mysql_install_db
-rwxr-xr-x 1 root root    6586 Feb 16 17:44 /usr/bin/mysql_secure_installation
-rwxr-xr-x 1 root root   16687 Feb 16 17:44 /usr/bin/mysql_setpermission
-rwxr-xr-x 1 root root   28224 Feb 16 17:45 /usr/bin/mysqlshow
-rwxr-xr-x 1 root root   14473 Feb 16 14:39 /usr/bin/mysql_tableinfo
-rwxr-xr-x 1 root root  158192 Feb 16 17:45 /usr/bin/mysqltest
-rwxr-xr-x 1 root root   42360 Feb 16 17:45 /usr/bin/mysqltestmanager
-rwxr-xr-x 1 root root   15464 Feb 16 17:45 /usr/bin/mysqltestmanagerc
-rwxr-xr-x 1 root root   13448 Feb 16 17:45 /usr/bin/mysqltestmanager-pwgen
-rwxr-xr-x 1 root root 1312064 Feb 16 17:45 /usr/bin/mysql_tzinfo_to_sql
-rwxr-xr-x 1 root root   54160 Feb 16 17:45 /usr/bin/mysql_upgrade
-rwxr-xr-x 1 root root    5753 Feb 16 17:44 /usr/bin/mysql_upgrade_shell
-rwxr-xr-x 1 root root  112136 Feb 16 17:45 /usr/bin/mysql_waitpid
-rwxr-xr-x 1 root root    3818 Feb 16 17:44 /usr/bin/mysql_zap

The mysqld binary is found in /usr/libexec

Error Log

The MySQL error log is found in /var/log/mysqld.log

The content after an initial start of MySQL will look similar to:

cat /var/log/mysqld.log
100705 22:09:03  mysqld started
InnoDB: The first specified data file ./ibdata1 did not exist:
InnoDB: a new database to be created!
100705 22:09:03  InnoDB: Setting file ./ibdata1 size to 10 MB
InnoDB: Database physically writes the file full: wait...
100705 22:09:03  InnoDB: Log file ./ib_logfile0 did not exist: new to be created
InnoDB: Setting log file ./ib_logfile0 size to 5 MB
InnoDB: Database physically writes the file full: wait...
100705 22:09:03  InnoDB: Log file ./ib_logfile1 did not exist: new to be created
InnoDB: Setting log file ./ib_logfile1 size to 5 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Doublewrite buffer not found: creating new
InnoDB: Doublewrite buffer created
InnoDB: Creating foreign key constraint system tables
InnoDB: Foreign key constraint system tables created
100705 22:09:03  InnoDB: Started; log sequence number 0 0
100705 22:09:03 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.0.77'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  Source distribution

On the first invocation of MySQL, the InnoDB storage engine will create a default tablespace and redo logs. This is the majority of messages in the above log.

Processes

MySQL is a multi-threaded single process called mysqld. A second wrapper process mysqld_safe is generally found. This process logs stderr and also will restart the mysqld process if not found.

ps -ef | grep mysql
root     14733     1  0 Jul05 pts/1    00:00:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --socket=/var/lib/mysql/mysql.sock --log-error=/var/log/mysqld.log --pid-file=/var/run/mysqld/mysqld.pid --user=mysql
mysql    14783 14733  0 Jul05 pts/1    00:00:10 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-external-locking --socket=/var/lib/mysql/mysql.sock

Memory Usage

MySQL can have a very low memory footprint. By default the mysqld process has a 175M virtual size.

$ ps -eopid,fname,rss,vsz,user,command | grep -e "RSS" -e "mysql"
  PID COMMAND    RSS    VSZ USER     COMMAND
14275 grep       720  61136 root     grep -e RSS -e mysql
14733 mysqld_s  1192  63820 root     /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --socket=/var/lib/mysql/mysql.sock --log-error=/var/log/mysqld.log --pid-file=/var/run/mysqld/mysqld.pid --user=mysql
14783 mysqld   27004 179496 mysql    /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-external-locking --socket=/var/lib/mysql/mysql.sock

Disk Usage

The MySQL data files will be stored on a default installation in /var/lib/mysql

$ du -sh /var/lib/mysql
22M     /var/lib/mysql

$ ls -ld /var/lib/mysql
drwxr-xr-x 4 mysql mysql 4096 Jul 13 11:50 /var/lib/mysql

$ ls -l /var/lib/mysql
total 20552
-rw-rw---- 1 mysql mysql 10485760 Jul  5 22:09 ibdata1
-rw-rw---- 1 mysql mysql  5242880 Jul  5 22:09 ib_logfile0
-rw-rw---- 1 mysql mysql  5242880 Jul  5 22:09 ib_logfile1
drwx------ 2 mysql mysql     4096 Jul  5 22:09 mysql
srwxrwxrwx 1 mysql mysql        0 Jul  5 22:09 mysql.sock

The MySQL data directory includes the InnoDB tablespace datafile (ibdata1), redo logs (ib_logfile?), and the mysql directory corresponding to the mysql schema containing instance meta data.

This directory also contains the socket file, which is actually a poor location as this opens the security of this directory for world access. This will be discussed later in securing your installation.

Running MySQL

The best means of controlling the starting and stopping of mysql is to use the provided service init script mysqld

$ ls -l /etc/init.d/mysqld
-rwxr-xr-x 1 root root 4286 Feb 16 17:45 /etc/init.d/mysqld

Configuration

For OEL the MySQL configuration can be found in /etc.
NOTE: MySQL can use multiple configuration files.

$ ls -l /etc/my.cnf
-rw-r--r-- 1 root root 441 Feb 16 14:39 /etc/my.cnf

MySQL includes a minimalistic configuration file by default. The configuration file format is variable=value pairs for a given number of different sections, in this file [mysqld] and [mysqld_safe].

$ cat /etc/my.cnf

[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
# Default to using old password format for compatibility with mysql 3.x
# clients (those using the mysqlclient10 compatibility package).
old_passwords=1

# Disabling symbolic-links is recommended to prevent assorted security risks;
# to do so, uncomment this line:
# symbolic-links=0

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

Audit

A full audit of all MySQL related files.

find / -name "*mysql*"
/etc/rc.d/rc3.d/S64mysqld
/etc/rc.d/rc5.d/S64mysqld
/etc/rc.d/rc6.d/K36mysqld
/etc/rc.d/init.d/mysqld
/etc/rc.d/rc0.d/K36mysqld
/etc/rc.d/rc4.d/S64mysqld
/etc/rc.d/rc1.d/K36mysqld
/etc/rc.d/rc2.d/S64mysqld
/etc/php.d/pdo_mysql.ini
/etc/php.d/mysql.ini
/etc/php.d/mysqli.ini
/etc/ld.so.conf.d/mysql-x86_64.conf
/etc/ld.so.conf.d/mysql-i386.conf
/usr/lib64/mysql
/usr/lib64/mysql/mysqlbug
/usr/lib64/mysql/libmysqlclient_r.so.15.0.0
/usr/lib64/mysql/libmysqlclient.so.15
/usr/lib64/mysql/libmysqlclient_r.so.15
/usr/lib64/mysql/mysql_config
/usr/lib64/mysql/libmysqlclient.so.15.0.0
/usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi/Bundle/DBD/mysql.pm
/usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi/auto/DBD/mysql
/usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi/auto/DBD/mysql/mysql.so
/usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi/DBD/mysql
/usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi/DBD/mysql.pm
/usr/lib64/php/modules/mysql.so
/usr/lib64/php/modules/pdo_mysql.so
/usr/lib64/php/modules/mysqli.so
/usr/libexec/mysqld
/usr/libexec/mysqlmanager
/usr/share/mysql
/usr/share/mysql/mysql_system_tables.sql
/usr/share/mysql/mysql_system_tables_data.sql
/usr/share/mysql/mysql_fix_privilege_tables.sql
/usr/share/mysql/mysql_test_data_timezone.sql
/usr/share/vim/vim70/syntax/mysql.vim
/usr/share/man/man8/mysqld.8.gz
/usr/share/man/man8/mysqlmanager.8.gz
/usr/share/man/man1/mysql.1.gz
/usr/share/man/man1/mysql.server.1.gz
/usr/share/man/man1/mysql_tableinfo.1.gz
/usr/share/man/man1/mysql_upgrade.1.gz
/usr/share/man/man1/mysqlaccess.1.gz
/usr/share/man/man1/mysql_waitpid.1.gz
/usr/share/man/man1/mysql_fix_extensions.1.gz
/usr/share/man/man1/mysqlman.1.gz
/usr/share/man/man1/mysqlbinlog.1.gz
/usr/share/man/man1/mysql_install_db.1.gz
/usr/share/man/man1/mysql_tzinfo_to_sql.1.gz
/usr/share/man/man1/mysql_secure_installation.1.gz
/usr/share/man/man1/mysqld_safe.1.gz
/usr/share/man/man1/mysqladmin.1.gz
/usr/share/man/man1/mysqlimport.1.gz
/usr/share/man/man1/mysql_zap.1.gz
/usr/share/man/man1/msql2mysql.1.gz
/usr/share/man/man1/mysqlshow.1.gz
/usr/share/man/man1/mysqldump.1.gz
/usr/share/man/man1/safe_mysqld.1.gz
/usr/share/man/man1/mysql_explain_log.1.gz
/usr/share/man/man1/mysql_config.1.gz
/usr/share/man/man1/mysqlbug.1.gz
/usr/share/man/man1/mysqld_multi.1.gz
/usr/share/man/man1/mysql_setpermission.1.gz
/usr/share/man/man1/mysqlhotcopy.1.gz
/usr/share/man/man1/mysql_find_rows.1.gz
/usr/share/man/man1/mysql_convert_table_format.1.gz
/usr/share/man/man1/mysql_fix_privilege_tables.1.gz
/usr/share/man/man1/mysqldumpslow.1.gz
/usr/share/man/man1/mysqltest.1.gz
/usr/share/man/man1/mysqlcheck.1.gz
/usr/share/man/man3/Bundle::DBD::mysql.3pm.gz
/usr/share/man/man3/DBD::mysql.3pm.gz
/usr/share/man/man3/DBD::mysql::INSTALL.3pm.gz
/usr/share/doc/mysql-server-5.0.77
/usr/share/doc/mysql-5.0.77
/usr/share/doc/selinux-policy-2.4.6/html/services_mysql.html
/usr/share/pixmaps/comps/mysql.png
/usr/share/info/mysql.info.gz
/usr/share/selinux/devel/include/services/mysql.if
/usr/bin/mysql_fix_extensions
/usr/bin/mysql
/usr/bin/mysqltestmanager
/usr/bin/mysqldumpslow
/usr/bin/mysql_upgrade_shell
/usr/bin/mysql_convert_table_format
/usr/bin/mysqlimport
/usr/bin/mysqldump
/usr/bin/mysqltestmanager-pwgen
/usr/bin/mysql_tzinfo_to_sql
/usr/bin/mysqlbug
/usr/bin/mysqlhotcopy
/usr/bin/mysqlaccess
/usr/bin/mysqltest
/usr/bin/mysqladmin
/usr/bin/mysql_upgrade
/usr/bin/mysqltestmanagerc
/usr/bin/mysqld_safe
/usr/bin/mysql_zap
/usr/bin/mysql_waitpid
/usr/bin/msql2mysql
/usr/bin/mysql_secure_installation
/usr/bin/mysql_fix_privilege_tables
/usr/bin/mysqlshow
/usr/bin/mysql_config
/usr/bin/mysql_setpermission
/usr/bin/mysql_tableinfo
/usr/bin/mysql_find_rows
/usr/bin/mysqld_multi
/usr/bin/mysqlcheck
/usr/bin/mysqlbinlog
/usr/bin/mysql_install_db
/usr/bin/mysql_explain_log
/usr/lib/mysql
/usr/lib/mysql/mysqlbug
/usr/lib/mysql/libmysqlclient_r.so.15.0.0
/usr/lib/mysql/libmysqlclient.so.15
/usr/lib/mysql/libmysqlclient_r.so.15
/usr/lib/mysql/mysql_config
/usr/lib/mysql/libmysqlclient.so.15.0.0
/usr/lib/python2.4/site-packages/sos/plugins/mysql.pyo
/usr/lib/python2.4/site-packages/sos/plugins/mysql.pyc
/usr/lib/python2.4/site-packages/sos/plugins/mysql.py
/var/log/mysqld.log
/var/run/mysqld
/var/run/mysqld/mysqld.pid
/var/lock/subsys/mysqld
/var/lib/mysql
/var/lib/mysql/mysql
/var/lib/mysql/mysql.sock
/root/.mysql_history
/selinux/booleans/mysqld_disable_trans
/selinux/booleans/allow_user_mysql_connect

$ yum install -y httpd php php-mysql
# Start the Apache Httpd Process
$ /etc/init.d/httpd start

To test and confirm Apache Httpd and PHP, we can use the CLI browser lynx:

$ yum install -y lynx
$ echo "<? phpinfo() ?>" > /var/www/html/phpinfo.php
$ lynx http://localhost/phpinfo.php

If successful, you will find a web page that contains the following.

phpinfo() (p1 of 31)

   PHP Logo 

PHP Version 5.1.6

   System Linux localhost.localdomain 2.6.18-164.el5 #1 SMP Thu Sep 3 04:15:13
   EDT 2009 x86_64
   Build Date Feb 11 2010 19:07:36
   Configure   Command   './configure'  '--build=x86_64-redhat-linux-gnu'
   '--host=x86_64-redhat-linux-gnu'    '--target=x86_64-redhat-linux-gnu'
   '--program-prefix=''--prefix=/usr''--exec-prefix=/usr''--bindir=/usr/bin'
   '--sbindir=/usr/sbin'    '--sysconfdir=/etc'    '--datadir=/usr/share'
   '--includedir=/usr/include'                      '--libdir=/usr/lib64'
   '--libexecdir=/usr/libexec'                     '--localstatedir=/var'
   '--sharedstatedir=/usr/com'                  '--mandir=/usr/share/man'
   '--infodir=/usr/share/info'             '--cache-file=../config.cache'
   '--with-libdir=lib64'                   '--with-config-file-path=/etc'

It is important to note that PHP is also a standalone scripting language that doesn’t require a web browser. You can use PHP on the command line, for example:

$ php --version
PHP 5.1.6 (cli) (built: Feb 11 2010 19:06:40)
Copyright (c) 1997-2006 The PHP Group
Zend Engine v2.1.0, Copyright (c) 1998-2006 Zend Technologies

$ echo "<?phpinfo()?>" | php | grep -i mysql
Configure Command =>  './configure' '--build=x86_64-redhat-linux-gnu' '--host=x86_64-redhat-linux-gnu' '--target=x86_64-redhat-linux-gnu' '--program-prefix=' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--localstatedir=/var' '--sharedstatedir=/usr/com' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--cache-file=../config.cache' '--with-libdir=lib64' '--with-config-file-path=/etc' '--with-config-file-scan-dir=/etc/php.d' '--disable-debug' '--with-pic' '--disable-rpath' '--without-pear' '--with-bz2' '--with-curl' '--with-exec-dir=/usr/bin' '--with-freetype-dir=/usr' '--with-png-dir=/usr' '--enable-gd-native-ttf' '--without-gdbm' '--with-gettext' '--with-gmp' '--with-iconv' '--with-jpeg-dir=/usr' '--with-openssl' '--with-png' '--with-pspell' '--with-expat-dir=/usr' '--with-pcre-regex=/usr' '--with-zlib' '--with-layout=GNU' '--enable-exif' '--enable-ftp' '--enable-magic-quotes' '--enable-sockets' '--enable-sysvsem' '--enable-sysvshm' '--enable-sysvmsg' '--enable-track-vars' '--enable-trans-sid' '--enable-yp' '--enable-wddx' '--with-kerberos' '--enable-ucd-snmp-hack' '--with-unixODBC=shared,/usr' '--enable-memory-limit' '--enable-shmop' '--enable-calendar' '--enable-dbx' '--enable-dio' '--with-mime-magic=/usr/share/file/magic.mime' '--without-sqlite' '--with-libxml-dir=/usr' '--with-xml' '--with-system-tzdata' '--enable-force-cgi-redirect' '--enable-pcntl' '--with-imap=shared' '--with-imap-ssl' '--enable-mbstring=shared' '--enable-mbstr-enc-trans' '--enable-mbregex' '--with-ncurses=shared' '--with-gd=shared' '--enable-bcmath=shared' '--enable-dba=shared' '--with-db4=/usr' '--with-xmlrpc=shared' '--with-ldap=shared' '--with-ldap-sasl' '--with-mysql=shared,/usr' '--with-mysqli=shared,/usr/lib64/mysql/mysql_config' '--enable-dom=shared' '--with-dom-xslt=/usr' '--with-dom-exslt=/usr' '--with-pgsql=shared' '--with-snmp=shared,/usr' '--enable-soap=shared' '--with-xsl=shared,/usr' '--enable-xmlreader=shared' '--enable-xmlwriter=shared' '--enable-fastcgi' '--enable-pdo=shared' '--with-pdo-odbc=shared,unixODBC,/usr' '--with-pdo-mysql=shared,/usr/lib64/mysql/mysql_config' '--with-pdo-pgsql=shared,/usr' '--with-pdo-sqlite=shared,/usr' '--enable-dbase=shared'
/etc/php.d/mysql.ini,
/etc/php.d/mysqli.ini,
/etc/php.d/pdo_mysql.ini,
mysql
MySQL Support => enabled
MYSQL_MODULE_TYPE => external
MYSQL_SOCKET => /var/lib/mysql/mysql.sock
MYSQL_INCLUDE => -I/usr/include/mysql
MYSQL_LIBS => -L/usr/lib64/mysql -lmysqlclient
mysql.allow_persistent => On => On
mysql.connect_timeout => 60 => 60
mysql.default_host => no value => no value
mysql.default_password => no value => no value
mysql.default_port => no value => no value
mysql.default_socket => no value => no value
mysql.default_user => no value => no value
mysql.max_links => Unlimited => Unlimited
mysql.max_persistent => Unlimited => Unlimited
mysql.trace_mode => Off => Off
mysqli
MysqlI Support => enabled
MYSQLI_SOCKET => /var/lib/mysql/mysql.sock
mysqli.default_host => no value => no value
mysqli.default_port => 3306 => 3306
mysqli.default_pw => no value => no value
mysqli.default_socket => no value => no value
mysqli.default_user => no value => no value
mysqli.max_links => Unlimited => Unlimited
mysqli.reconnect => Off => Off
PDO drivers => mysql, sqlite
pdo_mysql
PDO Driver for MySQL, client library version => 5.0.77

Short URL: rb42.com/oel-install-lamp