Usually people do this around New Year, I will do it in February. Actually, I was inspired to do this after reviewing all the talks for this year's MySQL Conference - what a snapshot into the state of where we are! It made me realize we've made important progress in the past year, worth taking a moment to celebrate it. So here we go...
Usually people do this around New Year, I will do it in February. Actually, I was inspired to do this after reviewing all the talks for this year's MySQL Conference - what a snapshot into the state of where we are! It made me realize we've made important progress in the past year, worth taking a moment to celebrate it. So here we go...
In the past few years there was a lot of fear and doubt about MySQL due to Oracle taking over the ownership. But if you ask me, I was more worried for MySQL because of MySQL itself. I've often said that if MySQL had been a healthy open source project - like the other 3 components in the LAMP stack - then most of the NoSQL technologies we've seen come about would never have been started as their own projects, because it would have been more natural to build those needs on top of MySQL. You could have had a key-value store (HandlerSocket, Memcache API...), sharding (Spider) and clustering (Galera). You could have had a graph storage engine (OQGraph engine isn't quite it, I understand it is internally an InnoDB table still?). There could even have been MapReduce functionality, although I do think the Hadoop ecosystem targets problems that actually are better solved without MySQL.
Last week I had the opportunity to attend the Surge 2011 conference in Baltimore, MD. I thought it was a great conference, and I’m already looking forward to next year. I’m sure there’s already a plethora of great blog posts on Surge, but here’s just some thoughts based on my experience.
In no particular order:
EC2 has changed the world, everybody hates EC2
I don’t think I heard a presentation where somebody didn’t use EC2 and the other assorted AWS products. Amazon (as far as I know) was not represented at the conference, and it seemed awkward for them to not be there (to me, at least). This conference was full of folks rigging up serious production architectures in EC2, and they all seemed to have their own private horror story about something that EC2 did to them.
I’m really puzzled by why some of EC2s competitors aren’t just as popular, or why with all the talk of HA and redundancy, people seemed to think it a weird idea to run a production stack on EC2 and another provider, as opposed to running Hot/Hot (for the few actually doing so) on multiple EC2 availability zones.
The good part about how transient EC2 is? Everybody cares about service resiliency now.
Technology goes through fads, but good Engineering practices are true regardless
We’re all partial to the technologies we know and love (or know and hate!). I think the biggest reason is inertia: once we get familiar with a tool, it’s natural to see how to apply it in various ways and get comfortable with its implementation.
The diversity of technology being employed by presenters at Surge was impressive. I fully expected the Etsy presentation to talk about how they moved away from the big bad RDBMS (Postgres) and into YourFavoriteNoSQLSolutionGoesHere. However, they migrated from Python and Postgres to the ultra trendy: PHP and MySQL! What is this, 2003?
Seriously, Ross Snyder gave a great presentation and really highlighted the struggles Etsy went through with the traditional silos built between Dev and Ops. The underlying technology played second fiddle to those company dynamics: the end result seemed less tied to those particular technologies than getting the relationships right.
Postgres is a lot more popular than I thought
I have nothing against Postgres, I’ve just never had opportunity to use it. Nor have I ever heard anyone talk about it, much less say they actually used it. I guess I’ve just been living in a MySQL-only bubble.
Folks are a lot more serious about Operations work than in the past
I haven’t been to Velocity yet, so maybe I’m missing out. But it’s refreshing to see so much focus on Operability. No longer are Operations folks the ogres in the closet, or maybe it’s just the case that the ogres are getting better organized. :) In any case, I’m all for it.
MongoDB’s Replica set and sharding configurations is what I always wanted for MySQL
I can’t speak for Mongo’s scalability, reliability, client feature sets, or whatever. But I can say that the talk given by Matthias Stearn about scaling Mongo turned out to be more about High Availability and it was impressive.
Cross-WAN replica sets with client-configurable quorum write requirements, oh la-la! Master election priorities, replica auto-cloning, SSL inter-replica communication, and auto shard re-balancing! Response-time based client -> server and server -> server connection management, so stuff stays local including quorum reads/writes (up to the minimum quorum requirement) and slave recloning from the nearest up-to-date replica. Beautiful.
The replica sets in Mongo seem to operate not unlike MySQL with Tungsten Replicator/Enterprise (with at least a non-empty subset of these features). It’s super refreshing to see this kind of stuff baked in to the core of the datastore and I really want to see how it progresses to more feature-rich RDBMSs.
MySQL Connector/Net 6.4.4, a new version of the all-managed .NET driver for MySQL has been released. This is an update to our latest GA release and is intended for full production deployment.
Version 6.4.4 is intended for use with versions of MySQL from 5.0 – 5.5
It is now available in source and binary form from here and mirror sites (note that not all mirror sites may be up to date at this point of time- if you can’t find this version on some mirror, please try again later or choose another download site.)
The release is also available for download on the My Oracle Support (MOS) and will be available from Oracle eDelivery.
This release includes several bug fixes including a fix to using Windows authentication. Please review the change log and documentation for a review of what changed.
Enjoy and thanks for the support!
Update: I won't be in Moscow after all. I was denied visa on grounds that my passport is beginning to fall apart and there wasn't time to get new passport, invitation and visa. Maybe next year - I was excited to go.
October brings 2 very interesting conferences. I will be speaking first on Oct 3rd at HighLoad++ in Moscow and a few weeks later on Oct Oct 25 at Percona Live in London. I will give a talk called Choosing a MySQL Replication / High Availability Solution which is based on my thinking developed in my recent blog post The ultimate MySQL high availability solution and many benchmarks and functional tests I've done while evaluating these technologies.
At Percona Live I will also give a second talk Fixed in Drizzle: No more GOTCHA's. It looked like none of the Drizzle core team would be able to attend the conference and as I was going to be there I volunteered to cover a Drizzle topic at the same time. This is a talk Stewart Smith has given a few times at earlier conferences which I liked and proposed to Percona. As it turns out, also Stewart will be in London after all, so there will be 2 Drizzle talks, I will still give the one I'm committed to.
A few days ago we announced the availability of our new MySQL Installer for Windows. While it’s a great first edition we know that you are going to have questions and we wanted the community to be able to help. So we have created a new forum for the community to be able to discuss questions and issues with the new installer.
You can access the new forum here.
So, go forth and post!
Well, our baby is born! Some time ago we analyzed the feedback from users and customers and determined that far too many of our server installs were failing. We have the best open-source database on the planet but no one can see that if we can’t get it properly installed. Clearly something had to be done.
So, the new MySQL Installer is born. It’s a Windows application that comes delivered in a bundle along with a version of the database server, applications like Workbench, connectors, samples, and documentation. It includes some customized configuration screens that help with setting up the proper configuration files.
One of the great benefits of using the new MySQL Installer is that it comes with all the products very tightly integrated such as automatically creating connection entries in Workbench for the freshly installed server. Our goal with MySQL is to have you up and running in 15 minutes. With the new MySQL Installer, we’ve got that down to 3 minutes!
The best way to see all the greatness that is MySQL Installer is to download it yourself. You can grab a copy here.
You can also see the Oracle press release on it here.
Thank you for trying out our new MySQL Installer and please let us know what you think!
Most analytical and BI databases have date dimension table(s). One frequently needs to generate and populate such data. I present a solution below for such data generation, written in Python. Please use different database drivers/modules to connect to your specific database server (MySQL, SQL Server, Oracle, etc.) for data population.
Notes:
1. It takes 2 parameters, start date and end date, in YYYYMMDD format, inclusive. Extensive error checking is built in, but let me know if you have comments/suggestions;
2. The script produce a Python dictionary (associated array) and print out its content;
3. The output includes dayNumber: a day’s position in a year. For example, 2011-02-01 is the 32ed day in 2011, therefore its dayNumber is 32;
4. The output includes weekNumber: a week’s position in a year. The week number in year is based on ISO standard. From documentation: the ISO year consists of 52 or 53 full weeks, where a week starts on a Monday and ends on a Sunday. The first week of an ISO year is the first (Gregorian) calendar week of a year containing a Thursday. This is called week number 1, and the ISO year of that Thursday is the same as its Gregorian year.
So, 2011-01-01 has the weekNumber 52, because it falls on a Saturday and belongs to the last week of 2010.
5. The output includes weekday information as well. 4 different variations are included:
Sunday 0, Monday 1, and so on
Sunday 1, Monday 2, and so on
Monday 0, Tuesday 1, and so on
Monday 1, Tuesday 2, and so on
6. The script requires the argparse module. It comes with Python 2.7. Python version prior to 2.7 does not have it by default, therefore you need to install it.
import argparse, sys, time
from datetime import date, timedelta
parser = argparse.ArgumentParser(description="Generating date dimension data")
parser.add_argument('-s', '--startDate', help='Start date in YYYYMMDD format', required=True, dest='startDate')
parser.add_argument('-e', '--endDate', help='end date in YYYYMMDD format', required=True, dest='endDate')
argList = parser.parse_args()
if (((not argList.startDate.isdigit()) or (not (len(argList.startDate) == 8))) or ((not argList.endDate.isdigit()) or (not (len(argList.endDate) == 8))) or (argList.startDate > argList.endDate)):
print "Input(s) must be numeric in YYYYMMDD format and end date must not be earlier than start date"
sys.exit (1)
try:
startDate = date(int(argList.startDate[0:4]), int(argList.startDate[4:6]), int(argList.startDate[6:8]))
endDate = date(int(argList.endDate[0:4]), int(argList.endDate[4:6]), int(argList.endDate[6:8]))
except ValueError:
print "Input(s) must be valid date value in YYYYMMDD format"
sys.exit (1)
start = time.time()
while startDate <= endDate:
dateInfo = {'dateYYYYMMDD': startDate.strftime('%Y%m%d'), 'calDate': startDate.strftime('%Y-%m-%d'), 'calDay': startDate.day, 'calMonth': startDate.month, 'calYear': startDate.year}
dateInfo['dayOfWeekSunday0Monday1'] = startDate.isoweekday() % 7
dateInfo['dayOfWeekSunday1Monday2'] = startDate.isoweekday() % 7 + 1
dateInfo['dayOfWeekSunday6Monday0'] = startDate.weekday()
dateInfo['dayOfWeekSunday7Monday1'] = startDate.isoweekday()
dateInfo['dayNumber'] = startDate.toordinal() - date(startDate.year - 1, 12, 31).toordinal()
dateInfo['weekNumber'] = startDate.isocalendar()[1]
print dateInfo
startDate = startDate + timedelta(1)
MySQL Connector/Net 6.4.3, a new version of the all-managed .NET driver for MySQL has been released. This is a GA release and is intended for full production deployment.
Version 6.4.3 is intended for use with versions of MySQL from 5.0 – 5.5
It is now available in source and binary form from [http://dev.mysql.com/downloads/connector/net/6.4.html] and mirror
sites (note that not all mirror sites may be up to date at this point of time
- if you can’t find this version on some mirror, please try again later or choose another download site.)
The release is also available for download on the My Oracle Support (MOS) and will be available from Oracle eDelivery.
** New features found in 6.4 include (please see release notes for more information) **
* Windows Authentication*
This release includes our new support for Windows authentication when connecting to MySQL Server 5.5.
* Table Caching *
We are also introducing a new feature called table caching. This feature makes it possible to cache the rows of slow changing tables on the client side.
* Simple connection fail-over support *
We are also including some SQL generation improvements related to our entity framework provider. Please review the change log that ships with the product for a complete list of changes and enhancements.
Enjoy and thanks for the support!
MySQL Connector/Net 6.2.5, a update to our all-managed .NET driver for MySQL has been released. This is an update to our legacy 6.2 version line. All new development should be using a more recent product such as 6.4.3.
Version 6.2.5 is intended for use with versions of MySQL from 4.1 – 5.1. It is not suitable for use with MySQL 5.5 or later.
It is now available in source and binary form from [http://dev.mysql.com/downloads/connector/net/] and mirror
sites (note that not all mirror sites may be up to date at this point of time
- if you can’t find this version on some mirror, please try again later or choose another download site.)
This update includes more than 45 fixes from 6.2.4. Please review the change log that is included with the product to determine the exact nature of the changes.
Enjoy and thanks for the support!