Archive for the ‘PostgreSQL’ Category

Continuent Partners With VNC To Collaborate in DACH

Февраль 1st, 2012
Continuent is increasing its presence in Europe to offer our solutions and services for the open source database business sector in the region. VNC is Continuent's new partner in Germany, Austria and Switzerland (DACH). Read the entire press release here.Continuent and VNC are hosting a live webcast demonstrating advanced MySQL and PostgreSQL replication and clustering with Continuent Tungsten
PlanetMySQL Voting: Vote UP / Vote DOWN

MySQL and PostgreSQL Cloud Offerings – linux.conf.au 2012 miniconf talk by myself and Selena

Январь 23rd, 2012

Selena and I gave a talk on the various issues of running databases “in the cloud” at the recent linux.conf.au in Ballarat. Video is up, embedded below:


PlanetMySQL Voting: Vote UP / Vote DOWN

German webinar Feb 7th at 15:00 CET: Hochverfügbarkeit und Performance von MySQL und PostgreSQL

Январь 18th, 2012
Zahlreiche Unternehmen verlassen sich auf MySQL und PostgreSQL für die Datenverwaltung. Aber was passiert, wenn die Datenbank abstürzt oder eine Wartung benötigt? Wie können Sie sicherstellen, dass Ihre Datenbank Höchstleistung bringt mit minimalem Arbeitsaufwand für den Administrator und maximaler Auslastung der vorhandenen Hardware? Erfahren Sie, wie VNC und Continuent Tungsten Enterprise
PlanetMySQL Voting: Vote UP / Vote DOWN

I Really Dislike Anonymous Attacks

Ноябрь 14th, 2011
If you are interested in NoSQL databases (or maybe not) perhaps you have seen the anonymous "warning" about using MongoDB.   It concludes with the following pious request:

  Please take this warning seriously.

Now there are a lot of great resources about data management on the web but the aforementioned rant is not one of them.  If you plan to write technical articles and have people take them seriously, here are a few tips.
  1. Sign your name.  Readers are more impressed when they see you are not afraid to stand behind your words. 
  2. Explain what problem you were trying to solve.  Otherwise uncharitable readers might think you just started pumping information into a new database without thinking about possible consequences and now want to blame somebody else for your bad decision.  
  3. Explain how you could do better.  Not all designs work out, so propose alternatives.  Readers love to see authors demonstrate that they are not discouraged by adversity.  
As for most of the points made by the anonymous author, all I can say is: well, duh! 

MongoDB behavior with respect to global write locking and transaction durability is obvious from the official documentation.  These features are not my cup of tea, but it's also not as if 10gen is hiding them either.  Moreover, most people understand that new DBMS implementations have problems, not least of all losing data now and then.  You usually pick them because they have features that make it worth putting up with the immaturity.  I am not an expert on MongoDB, but I can say from experience it is amazingly easy to load JSON objects into it.  The up-front usability alone demonstrates excellent engineering.  I am sure for this reason that there are many other good features.  


PlanetMySQL Voting: Vote UP / Vote DOWN

MariaDB Statistics and Surveys

Октябрь 19th, 2011

I just finished reading a couple of interesting, and somewhat related, blog posts which I think are worth sharing (apologies to anyone who has already seen them). One is from Jelastic and the other is from Michal Hrušecký.

I’ve written about MariaDB and the Jelastic cloud before (see MariaDB now available as a hosted database via Jelastic cloud platform). Now Jelastic has published statistics on the relative popularity of the various databases they offer. The good news is MariaDB is currently the database of choice for 14% of their customers. The bad news is that we’re in fourth place behind their other three database choices (MySQL, PostgreSQL, and MongoDB). However, MariaDB has only been available on their platform for a little over two months and we’re very happy that so many users of Jelastic’s state-of-the-art Java cloud are choosing to use MariaDB. Thanks!

In the second blog post, Michal Hrušecký shares the results of what he terms his “little MySQL survey”. This time MariaDB comes in second behind MySQL Community Server (and ahead of MySQL Cluster and Percona Server). In case you didn’t know, Michal packages MySQL and MariaDB for openSUSE, so this survey was a good way for him to judge the relative popularity of some of the more popular variants and give him ideas on how to improve things.

BTW: If you know of other interesting/informative/etc… blog posts related to MariaDB, let us know so we can add them to the Blog Posts Relevant to MariaDB page of the AskMonty Knowledgebase!


PlanetMySQL Voting: Vote UP / Vote DOWN

Same query, 3 databases, 3 different results

Октябрь 12th, 2011
The SQL standard leaves a lot of room for different implementations. This is a little demonstration of one of such differences.

SQLite  3.7.4
sqlite> create table t1 (id serial, t time);
sqlite> insert into t1(t) values ('00:05:10');
sqlite> select t,t*1.5 from t1;
00:05:10|0.0

MySQL 5.6.4-m5
mysql> create table t1 (id serial, t time);
Query OK, 0 rows affected (0.01 sec)

mysql> insert into t1(t) values ('00:05:10');
Query OK, 1 row affected (0.00 sec)

mysql> select t,t*1.5 from t1;
+----------+-------+
| t        | t*1.5 |
+----------+-------+
| 00:05:10 |   765 |
+----------+-------+
1 row in set (0.00 sec)

PostgreSQL 9.0.3
test=# create table t1 (id serial, t time);
NOTICE:  CREATE TABLE will create implicit sequence "t1_id_seq" for serial column "t1.id"
CREATE TABLE
test=# insert into t1(t) values ('00:05:10');
INSERT 0 1
test=# select t,t*1.5 from t1;
    t     | ?column?
----------+----------
 00:05:10 | 00:07:45
(1 row)

I think that the behaviour from PostgreSQL is the correct one. MySQL will just remove the :'s to get the string 000510 and then multiplies that value. The behaviour from SQLite is even more strange.

Of course for MySQL you could use the time_to_sec and sec_to_time functions.

mysql> select t,sec_to_time(time_to_sec(t)*1.5) from t1;
+----------+-------------------------------+
| t        | sec_to_time(time_to_sec(t)*2) |
+----------+-------------------------------+
| 00:05:10 | 00:07:45                      |
+----------+-------------------------------+
1 row in set (0.00 sec)

But those functions are not available on SQLite.

sqlite> select t,sec_to_time(time_to_sec(t)*1.5) from t1;
Error: no such function: sec_to_time

PlanetMySQL Voting: Vote UP / Vote DOWN

When systems scale better than linearly

Октябрь 7th, 2011

I’ve been seeing a few occasions where Neil J. Gunther’s Universal Scalability Law doesn’t seem to model all of the important factors in a system as it scales. Models are only models, and they’re not the whole truth, so they never match reality perfectly. But there appear to be a small number of cases where systems can actually scale a bit better than linearly over a portion of the domain, due to what I’ve been calling an “economy of scale.” I believe that the Universal Scalability Law might need a third factor (seriality, coherency, and the new factor, economy of scale). I don’t think that the results I’m seeing can be modeled adequately with only two parameters.

Here are two publicly available cases that appear to demonstrate this phenomenon: Robert Haas’s recent blog post on PostgreSQL, titled Scalability, in Graphical Form, Analyzed and Mikael Ronstrom’s post from May on MySQL (NDB) Cluster, titled Better than Linear Scaling is Possible.

Dr. Ronstrom’s post discusses the mechanics of the phenomenon, and speculates (I’m not sure it’s conclusive) that it is from a combination of partitioning and better use of CPU caches. Now someone needs to do the math to figure out how to include this factor into the equation.

The good thing about the Universal Scalability Law is how simple and applicable it is for many systems. It’s nice that this economy-of-scale factor seems to be unusual and the simpler model remains easy to apply for a large variety of tasks.

Further Reading:


PlanetMySQL Voting: Vote UP / Vote DOWN

Open Source Hardware

Октябрь 1st, 2011
Back in 2010 I stopped buying test servers from Dell and began building them from components using Intel i7 processors, X58-based mother boards, and modular power supplies from Ultra.  It was a good way to learn about hardware.  Besides, it was getting old to pay for Dell desktop systems with Windows, which I would then wipe off when installing Linux.  Between the educational value of understanding the systems better, selecting the exact components I wanted, and being able to fix problems quickly, it has been one of the best investments I have ever made.  And it didn't cost any more than equivalent Dell servers.

For this reason, a couple of recent articles about computer hardware caught my attention.  First, Dell is losing business as companies like Facebook build their own customized servers.  Open source database performance experts like Peter Zaitsev have been talking about full-stack optimization including hardware for years.  Google built their original servers using off-the-shelf parts.  Vertical integration of applications and hardware has since gone mainstream.  If you deploy the same application(s) on many machines, balancing characteristics like cost, performance, and power utilization is no longer a specialist activity but a necessity of business.  It's not just cutting out the Microsoft tax but many other optimizations as well.

Second, developments in hardware itself are making custom systems more attractive to a wide range of users.  A recent blog post by Bunnie Huang describes how decreases in the slope of CPU clock speed increase over time mean you can get better cost/performance by building optimized, application-specific systems now than waiting for across-the-board improvements.  Stable standards also drive down the difficulty of rolling your own.  Components on mid-range servers are sufficiently standardized it is easier to build basic systems from components than to put together a bicycle from scratch.  Try building your own wheels sometime if you don't believe this.

Easily customizable hardware has important consequences.  At a business level, Dell and other mainline hardware vendors will adapt to lower margins, but the market for generic, mid-range appliances has evaporated.  Starting around 2005 there was a wave of companies trying to sell open source databases, memcached, and datamarts on custom hardware.   Most seem to have moved away from hardware, like Schooner,  or folded entirely (like Gear6 and Kickfire).  The long-term market for such appliances, to the extent it exists, is in the cloud.

The other consequence is potentially far more significant.  The traditional walls that encapsulated hardware and software design are breaking down.  Big web properties or large ISPs like Rackspace run lean design teams that integrate hardware with open source software deployment.  This not just a matter of software engineers learning about hardware or vice-versa.  It is the tip of a much bigger iceberg.  Facebook recently started the Open Compute Project, which is a community-based effort to design server infrastructure.   In their own words:
By releasing Open Compute Project technologies as open hardware, our goal is to develop servers and data centers following the model traditionally associated with open source software projects. That’s where you come in.
Facebook and others are opening up data center design.  Gamers have been building their own systems for years.  Assuming Bunnie's logic is correct, open hardware will apply to wide range of devices from phones up to massive clusters.  Community-based, customized system designs are no longer an oddity but part of a broad movement that will change the way all of us think about building and deploying applications on any kind of physical hardware.  It will upset current companies but also create opportunities for new kinds of businesses.  The "cloud" is not the only revolution in computing.  Open source hardware has arrived.  

PlanetMySQL Voting: Vote UP / Vote DOWN

Blog Summary for Week of September 5

Сентябрь 11th, 2011

1. Apache and MySQL Logging with Syslog-ng
This article shows how to use the popular system logging tool Syslog-ng to log Apache and MySQL events. Apache does not log via syslog-ng by default so we go over two methods of easily remedying this. We also show how to use SQL queries to view syslog-ng data.

2. Using M3 to take System Monitors to the Next Level
Monitis provides built in functionality to monitor a wide variety of system statistics as well as the ability to create custom system monitors. Monitis Monitor Manager, or M3 for short, allows you to take these custom monitors even further by providing you with a simple framework to use the incredible power of regular expressions to pull and format literally any kind of data and automatically send it over the wire to your Monitis dashboard.

3. How to Log to PostgreSQL with Syslog-ng
Here we show how to use syslog-ng to log and view PostgreSQL log data. We also include some tips on troubleshooting and avoiding annoyances.

4. 30 VMware vSphere Performance Tips
There’s a multitude of optimizations that can be made to make your vSphere setup perform at optimal levels . This article covers 30 tips and tricks that will ensure the best performance from VMware’s vSphere and some even apply to other virtualization products. Some of the highlights are to Use only VMware-compatible hardware and Run your system through a burn-in/stress test.

Share Now:
  • del.icio.us
  • Digg
  • Facebook
  • LinkedIn
  • BlinkList
  • DZone
  • Google Bookmarks
  • Reddit
  • StumbleUpon
  • Twitter
  • RSS

PlanetMySQL Voting: Vote UP / Vote DOWN

What’s Next for Tungsten Replicator

Сентябрь 8th, 2011
As Giuseppe Maxia recently posted we released Tungsten Replicator 2.0.4 this week.  It has a raft of bug fixes and new features of which one-line installations are the single biggest improvement.  I set up replicators dozens of times a day and having a single command for standard cluster topologies is a huge step forward.  Kudos to Jeff Mace for getting this nailed down.

So what's next?  You can get see what we are up to in general by looking at our issues list.  We cannot do everything at once, but here are the current priorities for Tungsten Replicator 2.0.5.
  • Parallel replication speed and robustness.  I'm currently working on eliminating choke points in performance (like this one) as well as eliminating corner cases that cause the replicator to require manual intervention, such as aging out logs that are still needed by slaves.  
  • Multi-master replication.  This includes better support for system of record architectures, many masters to one slave, and replication between the same databases on different sites.  Stephane Giron nailed a key MyISAM multi-master bug for the last release.  We will continue to polish this as we work through our current projects.   
  • Better installations for more types of databases.  Jeff recently hacked in support for PostgreSQL as well as Oracle slaves, and we are contemplating addition of MongoDB support.  Heterogeneous replication is getting simpler to set up.  
  • Filter usability.  Giuseppe has a list of improvements for filters, which are one of the most powerful Tungsten Replicator features but not as easy for non-developers to use as we would like.  Better installation support is first on the list followed by ability to load and unload dynamically.  
  • Data warehouse loading.  We have a design for fast data warehouse loading that I hope we'll be able to implement in the next few weeks.  Linas Virbalas has also been working on this problem along with a number of other heterogeneous projects for customers.  
This is a lot of work and not everything will necessarily be finished when 2.0.5 goes out.  However, I hope we'll make progress on all of them.  In case you are wondering how we pick things, replicator development is largely driven by customer projects.   If you have something you need in the replicator, please contact Continuent.

After this build we will... Er, let's get 2.0.5 done first.  Suffice it to say we have a long list of useful and interesting features to discuss in future blog articles.

PlanetMySQL Voting: Vote UP / Vote DOWN