PlanetMysql.ru - информация о СУБД MySQL » PostgreSQL

On datatypes, domains and why I think it’s time we reconsidered

Anders Karlsson — Mon, 06 Feb 2012 10:34:00 +0000

What's in a datatype then? A MySQL SMALLINT? A C int? An Oracle BLOB? One thing is for sure, they are not very well standardized, not even within the same environment, and much less so across them. And what does it means, really? When should I use a BLOB, when a BINARY VARCHAR and when to use a long long?

A datatype defines many attributes:

What data I can store in it: Only numbers? Printable alaphanumeric characters? Unicode? Binary data? An object?
What I can do with a value of that particular type and how does it behave? Can I concatenate two values? Add them (that is NOT the same as concatenate! The same operator (+) may be used, but it's not the same thing!)? Save it to a file?
How is it stored and represented in technical terms. And this can be argued, a C int doesn't define how it is stored on disk, but a MySQL INT does! And a BLOB is a type that is largely there just because it defines how to handle things on disk!

To link this up with MySQL, let's look at the built-in MySQL datatypes. Some of them are rarely used, and for all intents and purposes, many of them are overlapping and in some cases they are there JUST BECAUSE they have different semantics and behaviour! I am, for example, not a big fan of all the different INT types that MySQL supports; tinyint, smallint mediuming, bigint all in signed and unsigned shape. Why is this useful? Yes, I know a smallint is smaller on disk so there is 2 bytes less to read from disk compared to an int. Yeah right, big deal: Look now, if that is a big deal for your application, then you have bigger problems. Which doesn't mean that using a smallint isn't a means of, in some cases, improve performance a small bit. And that this is sometimes useful. But really, this should be hidden somewhere, we just have too many datatypes. The same goes for all the different BLOB datatypes! The BLOB datatypes has another issue by the way, the names of them a a bit silly: "TINYBLOB" is a contradiction, as BLOB stands for Binary LARGE Object! A Large object that is Tiny? And a something that is at the most 255 bytes isn't really "Large" anymore, that was way back in the 1970's or so.

No, I'm not a big fan of determining which datatype to use by considering storage requirements to optimize performance. That is something you might do as a performance enhancing measure after the job is done, possibly. In C, which is a language I have used longer than I care to remember, int seems to serve the role of being the default numeric integer datatype, which is not a bad thing, in particular not in C as C is a language used to write low-level stuff in, stuff that is performance sensitive, and a C int is linked to hardware specific integers.

FLOAT is another issue altogether. In very few business applications is there a need for FLOAT or DOUBLE in terms of IEEE floating point values. Really! This is a scientific number format, that is also supported by the Floating point unit in most computers of today, so operations on them are pretty fast these days. But I have seen too many business applications where the developer uses a FLOAT in the database for any kind of numeric value that is not an integer, which is a distinctly bad idea! It works as long as the numbers aren't too big, and when the operations on them are reasonably simple. This is not to say a FLOAT or DOUBLE isn’t useful, just that they are more targeted for scientific programming (Hey, they are after all defined by IEEE!). What most people really should use is DECIMAL (I'm using MySQL datatypes for reference, but you know what I mean). This is a datatype that accepts (hey, big surprise coming up here) decimal numbers, fixed point decimal numbers! Operations on DECIMAL are slower though, as they aren't really supported directly by either C (which is the language that MySQL is largely written in) or by the CPU / FPU. But they are better for business applications (for scientific applications, use FLOAT or DOUBLE. And continue to write your code in Fortran by all means!)

But all these datatype are increasingly used also to define the domain of the data in them! The SQL DOMAIN feature is in the SQL Standard but is largely forgotten. It is there in PostgreSQL for you to try. The advantage of this, in my mind, isn't just that you can define in more exact terms what data should go into a database column, without writing code to do that, and to make it apparent in the database schema what data is expected, it also means that we can get away from the wide array of different built-in datatypes in, say, MySQL. Instead we can stick with a few optimized ones, let them have a simple inherent domain and then we can define the application specific domains in application code. Anyone for a IP_ADDRESS datatype? A ZIPCODE datatype? An URL type. All those a reasonable requests from the application POV, but it really shouldn't (in my mind) be defined by the database server (As: What are the semantics of a ZIP-code? They are different in different countries, and the post-office can change their minds (although that is something that takes a long time I guess)). Why? As this is application specific domain! And application specifics should be in application code! Simple as that!

So what datatypes are useful then, the way I look at it:

RAW Data - This should be a basic type. As long as can be, or as short. Any data can be stored in it, and nearly as much as you would like.
Integers - Although an integer can be considered to be a special case of a more generic number, the integer domain is so generic, and the integer is such a common and basic type that is deserves a place here.
Decimal – Fixed point decimal numbers.
Float - Now we are getting close to the application domain here, but these guys deserve a place here anyway, as they are so common and hooked up to the programming languages and hardware.
DATETIME - Only one of them is really needed. One of DATETIME, DATE, YEAR, MONTH, TIMESTAMP, TIME etc etc. that is. Just one, very generic base type, where the specific domain is, you've heard it before, application specific!
Text - Reluctantly I add this to the list. Although this is just a RAW with some limitations (usually only UTF8 or ISO8859-15 or something characters), there are some operations and attributes that are so tied up with text strings that a TEXT type is reasonable to include.

And that's about it! Yes! We could add a few standard domains of top of all this, like FLOAT and DOUBLE, SMALLINT, TINYINT etc etc. But as for the basic types, this is what I would like to see. As for the weirdo TIMESTAMP semantics in MySQL, add them as triggers on your table, in application code or, best, as a specific domain (I do know that not all weirdiness of TIMESTAMP can be represented by a standard SQL DOMAIN, which is both a sign of just how weird TIMESTAMP is, and of some of the limitation of the SQL DOMAIN).

I have now mentioned SQL DOMAINs a few times, so lets spend some quality time with them right now before closing up, here is a simple session using PostgreSQL 8.4:

First we create a domain:
CREATE DOMAIN yesno AS CHAR(1)
DEFAULT 'N'
NOT NULL CHECK (VALUE = 'Y' OR VALUE = 'N');
Then we create a table that uses that domain:
CREATE TABLE user_active(username CHAR(10), is_active YESNO);
And then we check if it works using a few SQL statements:
INSERT INTO user_active VALUES('foo', NULL);
ERROR: domain yesno does not allow null values
INSERT INTO user_active VALUES('foo', 'Y');
INSERT 0 1
INSERT INTO user_active VALUES('foo', 'N');
INSERT 0 1
INSERT INTO user_active VALUES('foo', 'A');
ERROR: value for domain yesno violates check constraint "yesno_check"

Before I wrap this up, I want to say one more thing on the issue of SQL Domains: There is an alternative n PostgreSQL, which is user defined datatypes using a CREATE TYPE SQL command. To begin with, I like domains much better as they are declarative, which means I don’t have to read a whole bunch of (usually uncommented and undocumented) code to understand what they type does. Secondly, this little warning in the PostgreSQL 9.1 manual isn’t too encouraging (page 1289): “To create a new base type, you must be a superuser. (This restriction is made because an erroneous type definition could confuse or even crash the server.)”

I have more to say on types, and I will follow up with a new post on this subject eventually, but this is enough for now.

/Karlsson

PlanetMySQL Voting: Vote UP / Vote DOWN

Continuent Partners With VNC To Collaborate in DACH

Petri Virsunen — Wed, 01 Feb 2012 16:22:00 +0000

Continuent is increasing its presence in Europe to offer our solutions and services for the open source database business sector in the region. VNC is Continuent's new partner in Germany, Austria and Switzerland (DACH). Read the entire press release here.Continuent and VNC are hosting a live webcast demonstrating advanced MySQL and PostgreSQL replication and clustering with Continuent Tungsten
PlanetMySQL Voting: Vote UP / Vote DOWN

MySQL and PostgreSQL Cloud Offerings – linux.conf.au 2012 miniconf talk by myself and Selena

Stewart Smith — Mon, 23 Jan 2012 02:52:06 +0000

Selena and I gave a talk on the various issues of running databases “in the cloud” at the recent linux.conf.au in Ballarat. Video is up, embedded below:

PlanetMySQL Voting: Vote UP / Vote DOWN

German webinar Feb 7th at 15:00 CET: Hochverfügbarkeit und Performance von MySQL und PostgreSQL

Petri Virsunen — Tue, 17 Jan 2012 22:14:00 +0000

Zahlreiche Unternehmen verlassen sich auf MySQL und PostgreSQL für die Datenverwaltung. Aber was passiert, wenn die Datenbank abstürzt oder eine Wartung benötigt? Wie können Sie sicherstellen, dass Ihre Datenbank Höchstleistung bringt mit minimalem Arbeitsaufwand für den Administrator und maximaler Auslastung der vorhandenen Hardware? Erfahren Sie, wie VNC und Continuent Tungsten Enterprise
PlanetMySQL Voting: Vote UP / Vote DOWN

I Really Dislike Anonymous Attacks

Robert Hodges — Mon, 14 Nov 2011 04:33:00 +0000

If you are interested in NoSQL databases (or maybe not) perhaps you have seen the anonymous "warning" about using MongoDB. It concludes with the following pious request:

Please take this warning seriously.

Now there are a lot of great resources about data management on the web but the aforementioned rant is not one of them. If you plan to write technical articles and have people take them seriously, here are a few tips.

Sign your name. Readers are more impressed when they see you are not afraid to stand behind your words.
Explain what problem you were trying to solve. Otherwise uncharitable readers might think you just started pumping information into a new database without thinking about possible consequences and now want to blame somebody else for your bad decision.
Explain how you could do better. Not all designs work out, so propose alternatives. Readers love to see authors demonstrate that they are not discouraged by adversity.

As for most of the points made by the anonymous author, all I can say is: well, duh!

MongoDB behavior with respect to global write locking and transaction durability is obvious from the official documentation. These features are not my cup of tea, but it's also not as if 10gen is hiding them either. Moreover, most people understand that new DBMS implementations have problems, not least of all losing data now and then. You usually pick them because they have features that make it worth putting up with the immaturity. I am not an expert on MongoDB, but I can say from experience it is amazingly easy to load JSON objects into it. The up-front usability alone demonstrates excellent engineering. I am sure for this reason that there are many other good features.

p.s., Here is a point-by-point response from 10gen, helpfully pointed out by Alex Popescu.

PlanetMySQL Voting: Vote UP / Vote DOWN

MariaDB Statistics and Surveys

Monty Program Group Blog — Wed, 19 Oct 2011 19:06:17 +0000

I just finished reading a couple of interesting, and somewhat related, blog posts which I think are worth sharing (apologies to anyone who has already seen them). One is from Jelastic and the other is from Michal Hrušecký.

I’ve written about MariaDB and the Jelastic cloud before (see MariaDB now available as a hosted database via Jelastic cloud platform). Now Jelastic has published statistics on the relative popularity of the various databases they offer. The good news is MariaDB is currently the database of choice for 14% of their customers. The bad news is that we’re in fourth place behind their other three database choices (MySQL, PostgreSQL, and MongoDB). However, MariaDB has only been available on their platform for a little over two months and we’re very happy that so many users of Jelastic’s state-of-the-art Java cloud are choosing to use MariaDB. Thanks!

In the second blog post, Michal Hrušecký shares the results of what he terms his “little MySQL survey”. This time MariaDB comes in second behind MySQL Community Server (and ahead of MySQL Cluster and Percona Server). In case you didn’t know, Michal packages MySQL and MariaDB for openSUSE, so this survey was a good way for him to judge the relative popularity of some of the more popular variants and give him ideas on how to improve things.

BTW: If you know of other interesting/informative/etc… blog posts related to MariaDB, let us know so we can add them to the Blog Posts Relevant to MariaDB page of the AskMonty Knowledgebase!

PlanetMySQL Voting: Vote UP / Vote DOWN

Same query, 3 databases, 3 different results

Daniel van Eeden — Wed, 12 Oct 2011 06:35:24 +0000

The SQL standard leaves a lot of room for different implementations. This is a little demonstration of one of such differences.

SQLite 3.7.4

sqlite> create table t1 (id serial, t time);
sqlite> insert into t1(t) values ('00:05:10');
sqlite> select t,t*1.5 from t1;
00:05:10|0.0

MySQL 5.6.4-m5
mysql> create table t1 (id serial, t time);
Query OK, 0 rows affected (0.01 sec)

mysql> insert into t1(t) values ('00:05:10');
Query OK, 1 row affected (0.00 sec)

mysql> select t,t*1.5 from t1;
+----------+-------+
| t        | t*1.5 |
+----------+-------+
| 00:05:10 |   765 |
+----------+-------+
1 row in set (0.00 sec)

PostgreSQL 9.0.3
test=# create table t1 (id serial, t time);
NOTICE: CREATE TABLE will create implicit sequence "t1_id_seq" for serial column "t1.id"
CREATE TABLE
test=# insert into t1(t) values ('00:05:10');
INSERT 0 1
test=# select t,t*1.5 from t1;
    t     | ?column?
----------+----------
00:05:10 | 00:07:45
(1 row)

I think that the behaviour from PostgreSQL is the correct one. MySQL will just remove the :'s to get the string 000510 and then multiplies that value. The behaviour from SQLite is even more strange.

Of course for MySQL you could use the time_to_sec and sec_to_time functions.

mysql> select t,sec_to_time(time_to_sec(t)*1.5) from t1;
+----------+-------------------------------+
| t        | sec_to_time(time_to_sec(t)*2) |
+----------+-------------------------------+
| 00:05:10 | 00:07:45                      |
+----------+-------------------------------+
1 row in set (0.00 sec)

But those functions are not available on SQLite.

sqlite> select t,sec_to_time(time_to_sec(t)*1.5) from t1;
Error: no such function: sec_to_time

PlanetMySQL Voting: Vote UP / Vote DOWN

When systems scale better than linearly

Baron Schwartz (xaprb) — Fri, 07 Oct 2011 02:33:37 +0000

I’ve been seeing a few occasions where Neil J. Gunther’s Universal Scalability Law doesn’t seem to model all of the important factors in a system as it scales. Models are only models, and they’re not the whole truth, so they never match reality perfectly. But there appear to be a small number of cases where systems can actually scale a bit better than linearly over a portion of the domain, due to what I’ve been calling an “economy of scale.” I believe that the Universal Scalability Law might need a third factor (seriality, coherency, and the new factor, economy of scale). I don’t think that the results I’m seeing can be modeled adequately with only two parameters.

Here are two publicly available cases that appear to demonstrate this phenomenon: Robert Haas’s recent blog post on PostgreSQL, titled Scalability, in Graphical Form, Analyzed and Mikael Ronstrom’s post from May on MySQL (NDB) Cluster, titled Better than Linear Scaling is Possible.

Dr. Ronstrom’s post discusses the mechanics of the phenomenon, and speculates (I’m not sure it’s conclusive) that it is from a combination of partitioning and better use of CPU caches. Now someone needs to do the math to figure out how to include this factor into the equation.

The good thing about the Universal Scalability Law is how simple and applicable it is for many systems. It’s nice that this economy-of-scale factor seems to be unusual and the simpler model remains easy to apply for a large variety of tasks.

Further Reading:

PlanetMySQL Voting: Vote UP / Vote DOWN

Open Source Hardware

Robert Hodges — Sat, 01 Oct 2011 19:06:24 +0000

Back in 2010 I stopped buying test servers from Dell and began building them from components using Intel i7 processors, X58-based mother boards, and modular power supplies from Ultra. It was a good way to learn about hardware. Besides, it was getting old to pay for Dell desktop systems with Windows, which I would then wipe off when installing Linux. Between the educational value of understanding the systems better, selecting the exact components I wanted, and being able to fix problems quickly, it has been one of the best investments I have ever made. And it didn't cost any more than equivalent Dell servers.

For this reason, a couple of recent articles about computer hardware caught my attention. First, Dell is losing business as companies like Facebook build their own customized servers. Open source database performance experts like Peter Zaitsev have been talking about full-stack optimization including hardware for years. Google built their original servers using off-the-shelf parts. Vertical integration of applications and hardware has since gone mainstream. If you deploy the same application(s) on many machines, balancing characteristics like cost, performance, and power utilization is no longer a specialist activity but a necessity of business. It's not just cutting out the Microsoft tax but many other optimizations as well.

Second, developments in hardware itself are making custom systems more attractive to a wide range of users. A recent blog post by Bunnie Huang describes how decreases in the slope of CPU clock speed increase over time mean you can get better cost/performance by building optimized, application-specific systems now than waiting for across-the-board improvements. Stable standards also drive down the difficulty of rolling your own. Components on mid-range servers are sufficiently standardized it is easier to build basic systems from components than to put together a bicycle from scratch. Try building your own wheels sometime if you don't believe this.

Easily customizable hardware has important consequences. At a business level, Dell and other mainline hardware vendors will adapt to lower margins, but the market for generic, mid-range appliances has evaporated. Starting around 2005 there was a wave of companies trying to sell open source databases, memcached, and datamarts on custom hardware. Most seem to have moved away from hardware, like Schooner, or folded entirely (like Gear6 and Kickfire). The long-term market for such appliances, to the extent it exists, is in the cloud.

The other consequence is potentially far more significant. The traditional walls that encapsulated hardware and software design are breaking down. Big web properties or large ISPs like Rackspace run lean design teams that integrate hardware with open source software deployment. This not just a matter of software engineers learning about hardware or vice-versa. It is the tip of a much bigger iceberg. Facebook recently started the Open Compute Project, which is a community-based effort to design server infrastructure. In their own words:

By releasing Open Compute Project technologies as open hardware, our goal is to develop servers and data centers following the model traditionally associated with open source software projects. That’s where you come in.

Facebook and others are opening up data center design. Gamers have been building their own systems for years. Assuming Bunnie's logic is correct, open hardware will apply to wide range of devices from phones up to massive clusters. Community-based, customized system designs are no longer an oddity but part of a broad movement that will change the way all of us think about building and deploying applications on any kind of physical hardware. It will upset current companies but also create opportunities for new kinds of businesses. The "cloud" is not the only revolution in computing. Open source hardware has arrived.

PlanetMySQL Voting: Vote UP / Vote DOWN

Blog Summary for Week of September 5

Hovhannes Avoyan — Sat, 10 Sep 2011 23:55:40 +0000

1. Apache and MySQL Logging with Syslog-ng
This article shows how to use the popular system logging tool Syslog-ng to log Apache and MySQL events. Apache does not log via syslog-ng by default so we go over two methods of easily remedying this. We also show how to use SQL queries to view syslog-ng data.

2. Using M3 to take System Monitors to the Next Level
Monitis provides built in functionality to monitor a wide variety of system statistics as well as the ability to create custom system monitors. Monitis Monitor Manager, or M3 for short, allows you to take these custom monitors even further by providing you with a simple framework to use the incredible power of regular expressions to pull and format literally any kind of data and automatically send it over the wire to your Monitis dashboard.

3. How to Log to PostgreSQL with Syslog-ng
Here we show how to use syslog-ng to log and view PostgreSQL log data. We also include some tips on troubleshooting and avoiding annoyances.

4. 30 VMware vSphere Performance Tips
There’s a multitude of optimizations that can be made to make your vSphere setup perform at optimal levels . This article covers 30 tips and tricks that will ensure the best performance from VMware’s vSphere and some even apply to other virtualization products. Some of the highlights are to Use only VMware-compatible hardware and Run your system through a burn-in/stress test.

Share Now:

PlanetMySQL Voting: Vote UP / Vote DOWN