PlanetMysql.ru — информация о СУБД MySQL

I gave two monitoring related talks at OpenWorld, thanks to all that came along!

Both were monitoring related, the first an introduction to MySQL Enterprise Monitor, and the second a look at some of the new instrumentation that is getting developed, primarily within the MySQL 5.6 release. 

If you'd like to get a quick overview of how MySQL Enterprise Monitor works, then take a look through the "Getting to Know MySQL Enterprise Monitor" talk. This gives you a high level view of how the different pieces fit together, and then each of the important factors within the user interface:

And if you are interested in seeing how the instrumentation and monitoring landscape will look when 5.6 hits the streets, then you can get a sneak peak at the "MySQL Monitoring Mechanisms" talk, which runs through the new instrumentation in Performance Schema, the new INFORMATION_SCHEMA tables for monitoring InnoDB, and the new ways to trace how a statement executes with the enhanced EXPLAIN plan, and the new Optimizer Trace:

PlanetMySQL Voting: Vote UP / Vote DOWN

In 2007, Etsy made a big bet on homegrown middleware to help with the site’s scalability. A half-year after it was taken live, the company decided to abandon it. As a senior software engineer at Etsy put it, “if you’re doing something ‘clever,” you’re probably doing it wrong.”

Read the full article at Arstechnica.com

I want to focus on the important lessons from this article, about middleware and using stored procedures in this fashion for a public web application, creating unscalable design complexity (smart and “proper” according to the old enterprise design teachings…) – causing infrastructure, development and maintenance hassles.

In the process they did replace PostgreSQL with MySQL but that’s not the critical change that made all the difference. PostgreSQL is a fine database system also.

PlanetMySQL Voting: Vote UP / Vote DOWN

Topics for this podcast:

*Cloud M&A potential around OpenStack
*Oracle’s commercial extensions for MySQL
*Puppet Labs rolls out Enterprise 2.0, hosts PuppetConf
*Basho bolsters Riak distributed data store in NoSQL race
*Our latest special CAOS report, ‘The Changing Linux Landscape’

iTunes or direct download (25:59, 4.4MB)

PlanetMySQL Voting: Vote UP / Vote DOWN

MySQL Cluster is, without doubt, the most interesting product Oracle offers to the people out there. It’s the flagship, the holy grail, based on the knowledge and technology developed doing our well known MySQL Server. I’m not going to go through why MySQL Cluster is so great, that you can find anywhere. I’m going to [...]
PlanetMySQL Voting: Vote UP / Vote DOWN

All DBAs understand the importance and priority of quick, reliable database backup and recovery operations.  In fact, dating back to my early days with MySQL, the most commonly requested product features from the MySQL user base have been around online, non-blocking backup solutions for running MySQL servers.  In response, Oracle now provides MySQL Enterprise Backup ("MEB") which performs high performant, online "hot" backups for MySQL databases.  MEB provides all of the backup/recovery features and functionality DBAs expect, all from a scriptable command line interface.  You can learn all about MEB in the related MySQL docs.

My congratulations and appreciation go out to Lars Thalmann and the MySQL Enterprise Backup engineering team for the recent release of MEB 3.6.  While there are many great improvements in this specific release, as an operational DBA I am most excited about the new support for single file streaming and for the SBT interface features, described here:

Single File Streaming - This allows DBAs to offload the footprint of backup images to a different server or storage device without having to store them locally on the MySQL database server.  This removes storage and related overhead from the server being backed up and speeds up total backup time by removing the need to copy local backup images (which even when compressed can be very large) over the network to their ultimate network destination.  You can learn about this specific MEB option along with a good usage example here.

Support for SBT interface - The "Secure Backup to Tape" interface was originally developed by Oracle as a standard way for third-party backup media providers to easily integrate their solutions with Oracle Recovery Manager ("RMAN").  SBT is now supported in MEB 3.6 so MySQL backup images can now be generated by and streamed directly to advanced enterprise backup media management solutions (Oracle Secure Backup, Symantec Netbackup, most others) that are already deployed within an environment.  This simplifies MySQL administration by enabling DBAs to incorporate MySQL backup/recovery operations and media rotation/retention policies into existing standard operating procedures.  You can learn all about this new option, again with a useful example here.

MySQL Enterprise Backup is part of the commercial MySQL Enterprise Edition but like all Oracle products is free to download and use without obligation for 30 days.  This is a great way to try it out to see if it fits your needs.  

You can download and begin working with MEB 3.6 now:

1. Go to Oracle eDelivery.
2. Enter some basic details and click through the agreement.
3. Select "MySQL Product Pack", then your platform, then Go.


I will keep you posted as new MySQL product features and interesting Oracle integrations become available.  As always, thanks for your continued support of MySQL! 
PlanetMySQL Voting: Vote UP / Vote DOWN

Rapid Application Development (RAD) is a way of developing computer software applications with less effort than the traditional means.

RAD tools focus on providing code generation and automated testing capabilities with the use of convention over configuration to provide a streamlined workflow to create applications.

Even with the most advanced and easiest to use RAD tools, there are times which the traditional enterprise and the business software development vendors which are having their own implementations and in-house built frameworks are continuously refusing to adopt them.

Most of the misconceptions on the RAD are based on FUD (Fear, Uncertainty and Doubt) which has been created around the internal complexity of the RAD tools.

Here we take a look at the biggest common misconceptions on RAD tool adoption in the enterprise.

1. RAD tools are magical

This is a common myth among the developers who insist on writing their own code from scratch.

Due to the elegance and the accuracy of properly generated applications, they will assume that the changing of generated code would cause the application to lose its beauty.

They are reluctant to see patterns in commonly developed applications which could be easily generated by the use of RAD tools.

2. You can’t make changes to the generated code

Most of the code generated by the RAD tools are editable, but takes a developer who has the understanding on the related technologies.

When people do not have the exposure to the technologies beneath the RAD generated code, they will start to spread fear and doubt on the customizability of the generated application.

3. RAD tools are for simple applications

Yes, simple applications can be generated by RAD tools, but it doesn’t limit you from creating a large application with the use of the tool.

As long as your application is properly designed, RAD tools will always have a solution.

Agree or not, most of the current applications with support to the cloud are the ones built with RAD tools.

4. You can’t make mistakes while generating the code

Most RAD tools have the capability of correcting mistakes done while generating the code. In most cases, even if you have made a mistake that the tool cannot reverse, there will be a manual way you can correct it.

5. You can’t make complex user interfaces with RAD tools

First, ask yourself whether you really need a complex user interface. The users are really comfortable using a simple, elegant and consistent user interface, rather than a complex and bloated user interface.

Even if you want a complex user interface, you can always integrate your generated code to your own user interface.

6. RAD is for startups, not for big businesses

If you are a big business, you will benefit more from RAD than a startup, because your development and testing time would be significantly reduced due to the reuse of patterns.

Actually, big businesses use RAD, but will not tell you.

7. You can’t integrate RAD applications with other systems

This again is due to the lack of knowledge about the particular application development framework. Almost all of the RAD tools provide means of exposing services to integrate your generated application to other applications.

8. You can’t work in a team setting or use a Version Control System (VCS)

You can always check-in your generated code to the VCS, and work in collaboration. In most of the cases, you just have to make sure only one developer is responsible for the use of tool for a particular module of the application.

9. RAD tools lock you in for a particular technology

Before asking this, ask yourself how much you is locked into the technology which you are currently using.

Most tools provide a humble way to remove the RAD tool dependencies and continue the project as a handwritten project.

10. People who can code should always code, instead of using tools

People who can cook, should always cook? No. Especially when they are given a machine which can create any delicious meal, once the guidelines are defined by the cook? No, not at all.
People, who can code, are the best people to properly use RAD tools, as they understand the tool better, and are able to extend and make the maximum use of the power of the tool.

Additionally, they can write add-ons, application specific and platform specific components to better match the business needs while increasing productivity to the organization as a whole.

11. RAD tools reduces the job security of the developers

This is largely false, as the tool would only increase productivity. Developers would have more time creating ‘applications’ instead of creating ‘application code’. As the tests and the related scripts are automatically generated they would have more time for manual testing of the application and to cater more requirements instead of battling the complexities arising from day to day changes to the application.

12. You can’t maintain generated applications with RAD tools

Before asking this question, look at your own code and see how much legacy code and technologies are still present. One thing you would understand is that application code never grows old or dies. As long as it works, it will be alive.

Using RAD tools, you will always have the particular version of the tool to make changes to the existing code based on the available capabilities. Even if a new version comes along, it will be mostly backwards compatible and would even have capabilities to support the conversion of the existing generated code to the new code to make it better. This is largely possible due to the proper use of patterns in the generated application.

13. RAD is complete

The perfect RAD tool is not yet complete, but the current tools have been evolved over the years and are usually more than enough for almost all businesses. Most tools would provide you ways to enhance the tool in order to better cater for your individual needs.

14. RAD will take your house, your spouse, everything you have and leave you with nothing.

No. Really?

These are just the biggest misconceptions; the rest will be covered in a next post on this blog.

Additional Resources:

• Wikipedia: Rapid Application Development
• Wikipedia: Fear, Uncertainty and Doubt

Frameworks and Tools

• Microsoft Visual Studio
• Grails
• Spring Roo
• CakePHP

Read more posts on this blog :

• Nokia and Symbian : Business and Software : Lessons To Be Learnt
• What Apple taught other businesses tonight.
• Flash and Runtimes on Mobile Devices, Who is right? Steve Jobs or Adobe?
• MySQL is gone. Here comes MariaDB and Drizzle.
• Is Open Source software a classic example of fail for Design by Committee?
• Why a ‘Single Line’ was added to the Solaris Terms of Use by Oracle?
• Oracle starts to monetize Free Software, is it wrong?

 

PlanetMySQL Voting: Vote UP / Vote DOWN

In my continuing MySQL 5.5 blog series, today I am covering what's new on the replication front.  MySQL replication is my favorite server and what drew me to MySQL during my tenure with Embarcadero Technologies.  Others seem to agree as based on community and customer surveys, MySQL replication is the most popular and widely used database feature.  Mostly because it is easy to set up and ease, it enables scalability and provides a pretty robust solution for data redundancy, backup and overall availability.  In MySQL 5.5 replication has been enhanced in response to user requests that MySQL replication:

• Ensure data consistency between master and slave servers
• Immediately detect if replication is not working
• Allow a crashed slave to automatically recover from the master relay log
• Allow users to filter events for specific servers
• Correctly convert data types between master and slave servers

MySQL 5.5 replication includes the following enhancements that support users in these key areas:
 
Semi-synchronous Replication

MySQL replication is asynchronous by default meaning that a master and its slave/slaves are autonomous when it comes to data consistency.  Asynchronous replication provides optimal performance because a master is free to service other inbound transactional requests after
writing updates to its Binlog without waiting to verify that updates have been replicated to at least one slave in the topology.  While fast, this comes with a high risk of master/slave data inconsistency or even data loss at recovery if there is a failure on either end.  

MySQL 5.5 introduces semi-synchronous replication to ensure data consistency and redundancy between master and at least one slave in the calling chain.  In this configuration, a master and any number of its replicant slaves are configured so that at least one slave in the replication topology must acknowledge updates have been received and written to its relay log before the parent master commits the transaction.  In the event of a time-out, the originating master will temporarily switch to asynchronous replication until at least one of the slaves set up for semi-synchronous replication catches up.

Semi-synchronous replication must be enabled on both the master and slave servers, otherwise the master defaults to asynchronous replication.  MySQL 5.5 uses a new plug-in architecture to enable semi-synchronous replication.  To this end, the following commands and variable settings are used to enable 5.5 masters and slaves.  Static settings can also be added to the my.* configuration files:

To enable the semi-synchronous replicator on the master:

INSTALL PLUGIN 'rpl_semi_sync_master' SONAME 'semisync_master.so';
SET rpl_semi_sync_master_enabled=1;
SET rpl_semi_sync_master_timeout=1000; (1s, default 10s)

To enable the semi-synchronous replicator on a slave or slaves:

INSTALL PLUGIN 'rpl_semi_sync_slave' SONAME 'semisync_slave.so';
SET rpl_semi_sync_slave_enabled=1;
START SLAVE;

Once enabled semi-synchronous replication exposes new system and status variables that can be used to check on configuration and operational status.  The values for each are exposed using SHOW VARIABLES and SHOW STATUS.  These include:
 
On master:

• Rpl_semi_sync_master_status - indicates status of when master is using asynchronous or semi-sycnhronous replication.
• Rpl_semi_sync_master_clients - shows how many slaves are configured for semi-synchronous replication.
• Rpl_semi_sync_master_yes_tx - shows number of successfully acknowledged commits by slaves.
• Rpl_semi_sync_master_no_tx - shows number of unsuccessfully acknowledged commits by slaves.

On Slave:

• Rpl_semi_sync_slave_status - indicates if semi-synchronous replication is enabled on slave.

Replication Heartbeat

MySQL 5.5 provides a new replication heartbeat option that helps users know immediately when replication stops working.  The heartbeat is a message sent at regular intervals from a master node to slave nodes.  The slave can be configured to automatically check connection and message status; if the message is not received by the slave the slave knows that a connection to the master node has failed in some way. 

Replication heartbeat is an optional configuration and is enabled on the 5.5 slave using:

STOP SLAVE;
CHANGE MASTER TO master_heartbeat_period= milliseconds;
START SLAVE;

The following status variables can then be monitored to easily detect when a master is idle and to get a finer-grained estimate on slave seconds behind master for recovery purposes:

SHOW STATUS like 'slave_heartbeat period'
SHOW STATUS like 'slave_received_heartbeats'


Automatic Relay Log Recovery

MySQL 5.5 ensures master/slave consistency on a restart by allowing replication users to optionally configure slaves to automatically discard its own unprocessed relay logs and then recover pending transactions from the originating master.  This can be used after a slave crash to ensure that potentially corrupted relay logs are not processed.   For compatibility the default for this is disabled, but can be set using the new relay_log_recovery=value to 1 on the slave to be recovered.

Replication Per Server Filtering

Circular, or multi-master replication, provides a highly available deployment that ensures redundancy of data in the case any of the servers in a topology ring fails or is removed.  In this configuration master servers are configured so that each is also a slave of another server in the topology.  Updates written to any of the masters are then replicated around the ring until the transaction reaches the originating server which acts as the terminator of its own events.  In the event of a node failure the affected server is removed from the topology and its slave is simply redirected to another master in the ring and processing then continues.

In previous versions when a server is removed from the ring due to failure, maintenance, etc. users needed to manually ensure that all of its updates were terminated from the new calling chain.  MySQL 5.5 provides a new set of time-saving commands that allow users to easily filter out any events related to a removed server.




In the above case, when Server A is removed from the topology, users can now easily filter any Server A related events by entering the following command on the next server in the calling chain:

Server B> CHANGE MASTER TO MASTER_HOST=D ...
IGNORE_SERVER_IDS=(A)
 

Replication Slave Side Data Type Conversions

In MySQL 5.1 precise data type conversions between master and slave are supported for statement-based replication only.  In this configuration column types could be different in the master and slave tables as long as the underlying data had high level compatibility (INT to TINYINT for example).  MySQL 5.5 now provides precise data type conversions between master and slave for both statement-based and row-based operations.   Conversions within integer, decimal, string, binary, BIT, ENUM and SET domains are supported.
 
A new SET variable in 5.5 enables the conversion, and requires that the slave be restarted to take effect.  The settings and what they enable are:

• SET SLAVE_TYPE_CONVERSIONS="ALL_LOSSY' - enables conversions to types with smaller domain (INT to TINY for example)
• SET SLAVE_TYPE_CONVERSION="ALL_NON_LOSSY" - enables conversions to types with larger domain (TINY to INT for example)

The above is not exhaustive so you can learn more about all of the new MySQL 5.5 replication features, including how to enable, monitor and tune them, here.


Thanks for reading!
 
PlanetMySQL Voting: Vote UP / Vote DOWN

A catchy headline, and I believe more accurate than Oracle Puts the Squeeze on SMBs with MySQL Price Hike (Network World) and MySQL price hikes reveal depth of Oracle’s wallet love [MySQL Jacking up MySQL Prices] (The Register). Slightly more realistic is Oracle kills low-priced MySQL support (again The Register).

First, let’s review what Oracle has actually done: they ditched the MySQL enterprise Basic and Silver offerings. For Oracle, that makes sense. Their intended client base is “enterprise” (high end, think big corporates) and their MySQL sales and cost structure reflects this. It’s not a new thing that came with MySQL at Oracle, because MySQL at Sun Microsystems and MySQL AB before it had the same approach.

A company simply cannot operate below its market – that is not simply a matter of choice, instead it is dictated by their processes and cost structure. Smart people like Clayton Christensen at Harvard Business School have done ample research on this, here I’ll just give one simple example:

If you hire a sales person on commission and their quarterly quota is $100k, then they have to talk with clients that have at least a $10k-$20k potential (qualified leads), and they need to close (sign contract) with at least 10 within the period. They simply cannot spend any time on talking with potential $1k customers.

We may lament this state of affairs, but you can see how, given the choices made (sales person hired, commission system, quota), it’s as inevitable as an apple falling when you drop it. The way I describe this at Upstarta: if a company wants different results, they need to make sure that their business processes and cost structure lead them in that direction. But the simple fact is that most companies don’t have an internal feedback cycle that keeps an eye on these things, so they just go with the flow of consequences of common choices: aim for large(r) clients, grow turnover, get higher operational costs along the way – that in itself is a cycle and the only direction this particular one can go is up. As a natural consequence, over time old low-end offerings and clients need to be jettisoned – one way or another.

I say horay for Oracle to finally acknowledge this, since Sun Microsystems and MySQL AB before it did not (for whatever reason). This is years overdue. Whether the original MySQL company should have aimed to also serve smaller clients also is an entirely separate topic – and one which I covered at length previously (including internally in my time at MySQL AB), but it’s very much a station long passed. Once you float upward in the market, you can’t operate or move downward.

Now, are SMBs using MySQL actually getting squeezed by Oracle? They are not. There is no lock-in. This is about service contracts, not licensing. As we all know, MySQL is GPL licensed and internal use (even on a website or SaaS offering) is well within GPL parameters. There are a number of different companies offering service for MySQL, different types of service and delivery models and a corresponding wide range of pricing. So SMBs and anyone else has a choice, each can pick the type of service most suited to their needs. Let us celebrate and promote that freedom within the MySQL ecosystem, rather than being outraged about dropped apples falling!

PlanetMySQL Voting: Vote UP / Vote DOWN

Yes, you read the title correctly — there are three editions of MySQL available, according to http://www.mysql.com/products/enterprise/server.html. Well, that page names two, and then of course there is the community edition….

From the manual page:

MySQL Enterprise Server is available in the following editions:

* MySQL Enterprise Server – Pro is the world’s most popular open source database that enables you to rapidly deliver high performance and scalable Online Transaction Processing (OLTP) applications.
* MySQL Enterprise Server – Advanced is the most comprehensive edition of MySQL. It provides all the benefits of MySQL Enterprise Server Pro and adds horizontal table and index partitioning for improving the performance and management of VLDBs (Very Large Databases).

How is “horizontal table and index partitioning” different from the regular partitioning available in MySQL 5.1?

Those of us that have been around for the past 3 or so years know that there was a point in time where there were two different editions of MySQL available, back when MySQL Enterprise and MySQL Community were actually different. But that experiment was a complete failure, and the code is now the same. MySQL Enterprise does package the software in a way that is not available to the community, specifically the quarterly service pack (QSP) releases. But the actual code….the same.

The pricing page at http://globalspecials.sun.com/store/mysql/ContentTheme/pbPage.categoryEnterprise shows that the Advanced server can be acquired for $3k (Gold) or $5k (Platinum) per year. The fee is worth it for the support MySQL will give you, but why is MySQL muddying the waters by having more “editions”, which very likely are not even different code?

(Special thanks to Aaron Macks for pointing out the existence of mysql-advanced, which was the impetus for this blog post.)

PlanetMySQL Voting: Vote UP / Vote DOWN

The MySQL Enterprise Spring 2010 Release includes key improvements designed to help you proactively optimize MySQL performance and identify potential issues before they can become costly outages or slowdowns. In this webinar we will explore the new features of the MySQL Enterprise Monitor 2.2 that are designed to save DBAs and Developers time and effort in keeping MySQL systems running at the highest levels of security, performance and availability. Join us and learn about:

• How the MySQL Connectors can now be used to collect, aggregate and monitor MySQL query performance data
• How to quickly identify which queries are performing full table scans
• How to isolate your worst performing queries based on total execution counts, time, data returned, date/time, query content, query type, and more
• How to identify and drill down into line-level source code for specific query executions
• New LDAP authentication model
• New Read Only User
• New Auto-Closing Events
• New MySQL Support Diagnostic Reports
• New Custom Rule/Graph Import and Export options
• New Faster Purging of Repository Data
• New Installation Options
• And more!

PlanetMySQL Voting: Vote UP / Vote DOWN