Archive for the ‘oscon’ Category

My Talks at the MySQL conference 2012

Апрель 3rd, 2012

I was honored to get two talks accepted at the MySQL conference this year, but interestingly both of them are actually only indirectly about MySQL.  

The first is called “Writing non-blocking code for interaction with data systems and web services in Node.js and Perl”.  This sounds fancier than it is, I picked up some non-blocking programming in Perl and a bit of Node.js (which, strangely, is pretty straightforward once you write some Perl AnyEvent) my last year or so at Yahoo.  This talk really is just meant to be a gentle introduction to what this type of coding is, how it works, and some of the things you can do with it.  My goal is to present something I could have used when I was trying to make the jump from straight procedural to event-driven programming a few years ago.

The second is “Running a high performance LAMP stack on a $20 Virtual server”.  This talk is based off of my experience running a small hosting/consulting company that resold cheap VPS servers running my custom tuned Debian Linux to run LAMP stack sites.  This typically ended up being things like Expression Engine and Magento.  I spent a lot of time tuning these servers to maximize the RAM utilization to give my customers the best bang for their buck.  This talk goes through all of the various architecture paths I went down and what I ultimately settled on, including webserver(s) of choice, CDN usage, PHP daemons, and big win MySQL tuning.

I’m working on putting these classes together this week, and I just got some good news that both of them have been selected for OSCON this year as well, so if you miss the MySQL conference, maybe I’ll see you there instead.  


PlanetMySQL Voting: Vote UP / Vote DOWN

More New MySQL 5.6 Early Access Features

Август 1st, 2011

Last week was a banner week for MySQL at OSCON. We had many MySQL developers meeting with the MySQL community, conducting technical sessions, leading BOF sessions, working the exhibit hall, and confirming Oracle's leadership in the technical evolution of MySQL.  The highlight of the week was the unveiling of even more 5.6 early access InnoDB and Replication features that are now available for early adopters to download, evaluate and shape via labs.mysql.com.  

InnoDB is one of MySQL's "crown jewels" and beginning in 5.5 is now the default storage engine.  The following 5.6 feature improvements are in direct response to community and customer feedback and requests.  The new 5.6 early access features include:


  • Full-text search
  • REDO log files max size extended to 2 TB
  • UNDO logs on their own tablespace
  • Buffer Pool options for pre-loading/warming on re-start
  • Improved auto-extension of .ibd files
  • Support for smaller 4k, 8k page sizes

Replication is by far the most popular and widely used MySQL feature. The following feature improvements in 5.6 are also in direct response to community and customer feedback and requests.  The new 5.6 early access features include:
  • New Binlog API
  • Binlog group commit (completes InnoDB group commit implemented in MySQL 5.5)
  • Durable Slave Reads
  • Enhanced multi-threaded slaves
You can read about the details, including development blogs on how to get started with each in this new DevZone article.  My sincere thanks and appreciation to the InnoDB and Replication development teams for their leadership in technical innovation and mind share and for their dedicated work in providing these and other new features "early and often" to the MySQL community.  Stay tuned for more to come!

We can't say this "early and often" enough...thanks for your continued support of MySQL!
PlanetMySQL Voting: Vote UP / Vote DOWN

MariaDB is at OSCON

Июль 27th, 2011

The MariaDB Project is at OSCON 2011. We’ve got a booth, and we plan to also have a BoF session: Wednesday 27 July 2011 at room E142 at 8pm.

There will as usual be lots of black vodka (there was some yesterday at the MySQL BoF as well), and we’re going to talk about and celebrate the release of MariaDB 5.3.0 beta.

Come drop by the booth… we clearly have an interesting booth giveaway. And feel free to say hi to Kurt von Finck, Michael “Monty” Widenius, or Colin Charles who will be present, walking around, etc. Spot them in a MariaDB t-shirt of course!


PlanetMySQL Voting: Vote UP / Vote DOWN

My MySQL SNMP Agent

Июль 22nd, 2011
Back in February I wrote an article titled A Small Fix For mysql-agent. Since then we did a few more fixes to the agent and included a Bytes Behind Master (or BBM) chart. For those who can't wait to get their hands on the code, here's the current version: MySQL SNMP agent RPM. For those who'd like to learn about it's capabilities and issues, keep reading.

What to Expect From this Version


The article I quoted above pretty much describes the main differences with the original project, but we went further with the changes while still relying on Masterzen's code for the data collection piece.

The first big change is that we transformed Masterzen's code into a Perl module, this way we can easily plug in a new version without having to do massive editing to ours.

The 2nd change is that we added the code to calculate how many bytes behind is a slave, which should be cross checked always with seconds behind master to get replication's full picture. When a slave is just a few bytes behind, the script calculates the difference straight out of the SHOW SLAVE STATUS information. If the SQL thread is executing statements that are in a binary log file older than the one being updated by the I/O thread, then the script logs into the master to collect the sizes of the previous binary logs and make an accurate calculation of the delta.

For this change we hit another bug in CentOS 5 SNMP agent, by which 64bit counters were being truncated. The solution is to upgrade to CentOS 6 (not anytime soon, but that's another story) or a work around. We decided for the latter and display a variable flagging this value roll over. This is not needed for non-CentOS 5 platforms as far as we know.

By now I expect that many of you would have a question in your mind:

Why Not Branch / Fork?

Why provide an RPM instead of creating a branch/fork in the original project? There are many reasons, but I'll limit myself to a couple. I trust that before you write an enraged comment you'll keep in mind that this is a personal perception, which might be in disagreement with yours.

This code is different enough from the original that creating a branch to the original project would be too complicated to maintain. For example: we are using a completely different SNMP protocol and created a module out of the original code. We don't have the resources to follow behind all of Masterzen's possible patches and I wouldn't expect him to adopt my changes.

If we would've created a fork (a new project derived from the original), I believe at this point, it would divert the attention from the original project or others like PalominoDB's Nagios plugin.

What's Next

We plan to continue maintaining this RPM driven by our specific needs and keep sharing the results this way. If at some point we see it fit to drive the merge into another project or create a new fork of an existing one, we'll do it.

I will be presenting the project at OSCON next week. If you're going to be around, please come to my talk: Monitoring MySQL through SNMP and we can discuss issues like: why use pass_persist, why not use information schema instead of the current method, why not include your personal MySQL instrumentation pet peeve, I'd be glad to sit down with you and personally chat about it.

In the meantime, enjoy, provide feedback and I hope to get to know you at OSCON next Thursday.

PlanetMySQL Voting: Vote UP / Vote DOWN

OSCON 2011

Июль 1st, 2011

This year I'll attend OSCON for the first time. I'll give two talks:

  • PHP and MySQL - Recent Developments
    PHP’s MySQL support recently received many changes under the hood. PHP 5.3 introduced mysqlnd – the MySQL native driver which is a replacement for libmysql deeply bound into PHP. In this presentation you will learn what the PHP and MySQL development teams were up to. After starting with an introduction of the PHP-stack, demystifying things like mysqli, mysqlnd or PDO, this presentation will show you how to build mysqlnd plugins as PHP C extension and hooking into mysqlnd from PHP userland. It will also discuss existing plugins like a client side query cache or a module for doing read-write-splitting, both working transparently, without changes to your application.
  • PHP Under the hood
    The beauty of PHP is that everybody can read the code and see the inner workings of software. But understanding concepts from reading code isn’t often helpful. Especially if you are no pro in that language. This presentation will take apart many parts of the PHP runtime, describe the concepts behind so attendees understand the inner workings without actually reading C code. Concepts covered include HashTables, the foundation for PHP arrays and many other internal data structures, the reference counting mechanism, which is important for writing efficient code as well as the overall executor.

In case you can't make it to these talks but want to talk to me you'll probably find me at the Oracle booth where I'll also try to give some short talks on some topics to be defined (any wishes?)

 In case you're not interested in me and my talks but MySQL there are a few sessions by other MySQL Engineers:

In case you want to attend you can use the code os11fos which should give you a 20% discount.

PlanetMySQL Voting: Vote UP / Vote DOWN

HOWTO screw up launching a free software project

Июль 27th, 2010

Josh Berkus gave a great talk at linux.conf.au 2010 (the CFP for linux.conf.au 2011 is open until August 7th) entitled “How to destroy your community” (lwn coverage). It was a simple, patented, 10 step program, finely homed over time to have maximum effect. Each step is simple and we can all name a dozen companies that have done at least three of them.

Simon Phipps this past week at OSCON talked about Open Source Continuity in practice – specifically mentioning some open source software projects that were at Sun but have since been abandoned by Oracle and different strategies you can put in place to ensure your software survives, and check lists for software you use to see if it will survive.

So what can you do to not destroy your community, but ensure you never get one to begin with?

Similar to destroying your community, you can just make it hard: “#1 is to make the project depend as much as possible on difficult tools.

#1 A Contributor License Agreement and Copyright Assignment.

If you happen to be in the unfortunate situation of being employed, this means you get to talk to lawyers. While your employer may well have an excellent Open Source Contribution Policy that lets you hack on GPL software on nights and weekends without a problem – if you’re handing over all the rights to another company – there gets to be lawyer time.

Your 1hr of contribution has now just ballooned. You’re going to use up resources of your employer (hey, lawyers are not cheap), it’s going to suck up your work time talking to them, and if you can get away from this in under several hours over a few weeks, you’re doing amazingly well – especially if you work for a large company.

If you are the kind of person with strong moral convictions, this is a non-starter. It is completely valid to not want to waste your employers’ time and money for a weekend project.

People scratching their own itch, however small is how free software gets to be so awesome.

I think we got this almost right with OpenStack. If you compare the agreement to the Apache License, there’s so much common wording it ends up pretty much saying that you agree you are able to submit things to the project under the Apache license.  This (of course) makes the entire thing pretty redundant as if people are going to be dishonest about submitting things under the Apache licnese there’s no reason they’re not going to be dishonest and sign this too.

You could also never make it about people – just make it about your company.

#2 Make it all about the company, and never about the project

People are not going to show up, do free work for you to make your company big, huge and yourself rich.

People are self serving. They see software they want only a few patches away, they see software that serves their company only a few patches away. They see software that is an excellent starting point for something totally different.

I’m not sure why this is down at number three… it’s possibly the biggest one for danger signs that you’re going to destroy something that doesn’t even yet exist…

#3 Open Core

This pretty much automatically means that you’re not going to accept certain patches for reasons of increasing your own company’s short term profit. i.e. software is no longer judged on technical merits, but rather political ones.

There is enough politics in free software as it is, creating more is not a feature.

So when people ask me about how I think the OpenStack launch went, I really want people to know how amazing it can be to just not fuck it up to begin with. Initial damage is very, very hard to ever undo. The number of Open Source software projects originally coming out of a company that are long running, have a wide variety of contributors and survive the original company are much smaller than you think.

PostgreSQL has survived many companies coming and going around it, and is stronger than ever. MySQL only has a developer community around it almost in spite of the companies that have shepherded the project. With Drizzle I think we’ve been doing okay – I think we need to work on some things, but they’re more generic to teams of people working on software in general rather than anything to do with a company.


PlanetMySQL Voting: Vote UP / Vote DOWN

VistA scenarios, and other controversies at the Open Source health care track

Июль 23rd, 2010

The history and accomplishments attributed to VistA, the Veterans
Administration's core administrative software, mark it as one of the
most impressive software projects in history. Still, lots of smart
people in the health care field deprecate VistA and cast doubt that it
could ever be widely adopted. Having spent some time with people on
both sides, I'll look at their arguments in this blog, and then
summarize other talks I heard today at the href="http://www.oscon.com/oscon2010">Open Source Convention
health care track.

Yesterday, as href="http://radar.oreilly.com/2010/07/day-one-of-the-health-care-it.html">I
described in my previous blog, we heard an overview of trends in
health care and its open source side in particular. Two open source
free software projects offering electronic health records were
presented, Tolven and href="http://www.oemr.org/">openEMR. Today was VistA day, and
those who stayed all the way through were entertained by accolades of
increasing fervor from the heads of href="http://www.oscon.com/oscon2010/public/schedule/detail/15291">vxVistA,
href="http://www.oscon.com/oscon2010/public/schedule/detail/15255">Medsphere,
and href="http://www.oscon.com/oscon2010/public/schedule/detail/15255">ClearHealth. (Anyone
who claims that VistA is cumbersome and obsolete will have to explain
why it seems to back up so many successful companies.) In general, a
nice theme to see today was so many open source companies making a go
of it in the health care field.

VistA: historical anomaly or the future of electronic medical systems?

We started our exploration of VistA with a href="http://www.oscon.com/oscon2010/public/schedule/detail/15274p">stirring
overview by Phillip Longman, author of the popular paperback book,
Best Care Anywhere: Why VA Health Care is Better Than
Yours
. The story of VistA's development is a true medical
thriller, with scenes ranging from sudden firings to actual fires
(arson). As several speakers stressed, the story is also about how the
doctors at the VA independently developed the key aspects of open
source development: programming by the users of the software, loose
coordination of independent coders, freedom to fork, and so on.

Longman is convinced that VistA could and should be the basis of
universal health records in the U.S., and rains down omens of doom on
the comprehensive health care bill if it drives physicians to buy
proprietary health record systems.

VistA is much more than an electronic health record system, and even
bigger than a medical system. It is really a constellation of hundreds
of applications, including food preparation, library administration,
policing, and more.

The two main objections to VistA are:


That it is clunky old code based on an obsolete language and database technology

As a project begun by amateurs, VistA probably contains some fearsome
passages. Furthermore, it is written in MUMPS (standardized by ANSI as
simply M), a language that dates from the time of LISP and
COBOL. Predating relational databases, MUMPS contains a hierarchical
database based on a B*-tree data structure.

Supporters of Vista argue that anything qualifying as "legacy code"
can just as well be called "stable." They can also answer each of
these criticisms:

  • The code has been used heavily by the VA long enough to prove that
    it is extendable and maintainable.

  • It is strangely hypocritical to hear VistA's use of MUMPS criticized
    by proprietary vendors when so any of them are equally dependent on
    that language. Indeed, the best-known vendors of proprietary health
    care software, including Epic and InterSystems, use MUMPS. Need I
    remind readers that we put a man on the moon using 1960s-style
    FORTRAN?

    It's interesting to learn, however, that ClearHealth is migrating
    parts of VistA away from MUMPS and does most of its coding in
    higher-level languages (and many modern programmers would hardly offer
    praise for the language chosen for ClearHealth's interface, PHP).

  • Similarly, many current vendors use the Cache hierarchical
    database. Aspersions concerning pre-relational databases sound less
    damning nowadays in an age of burgeoning interest in various NoSQL
    projects. Still, Medsphere and the community-based href="http://www.worldvista.org/">WorldVistA project are
    creating a SPARQL interface and a mechanism for extracting data from
    VistA into a MySQL database.


That it works well only in the unique environment of the Veterans Administration

This critique seems to be easier to validate through experience. The
VA is a monolithic, self-contained environment reflected in VistA. For
instance, the critical task of ordering prescriptions in VistA depends
on the pharmacy also running VistA.

Commercial pharmacies could theoretically interact with VistA, but it
would require effort on the part of those companies, which in turn
would depend on VistA being adopted by a substantial customer base of
private hospitals.

Several successful deployments of VistA by U.S. hospitals, as well as
adoption by whole networks of hospitals in several other countries,
indicate that it's still a viable option. And the presence of several
companies in the space shows that adopters can count on support.

On the other hand, the competing implementations by vxVistA,
Medsphere, and ClearHealth complicate the development landscape. It
might have been easier if a single organization such as WorldVistA
could have unified development as the Apache or GNOME foundation does.

vxVistA has come in for particular criticism among open source
advocates. In fact, the speakers at today's conference started
out defensive, making me feel some sympathy for them.

vxVistA's developers, the company DSS, kept their version of VistA
closed for some time until they had some established customers.
Speaker Deanne Clark argued that they did this to make sure they had
enough control over their product to produce some early successes,
warning that any failure would hurt the image of the whole VistA
community. I don't know why a closed development process is necessary
to ensure quality, but I'll accept her explanation. And DSS seems to
be regarded highly for its quality work by everyone, including those
who embroil

More galling to other open source advocates is that when DSS did
release vxVistA as open source, they did so under an Eclipse license
that is incompatible with the GPL used by WorldVistA.

I wouldn't dare guess whether VistA will continue as a niche product
or will suddenly emerge to eat up the U.S. market for electronic
medical systems. But I think it's definitely something to watch.

The odd position of the VA as the source for new versions of VistA, as
well as its role as VistA's overwhelmingly largest user, could also
introduce distortions into the open source development pattern outside
the VA. For instance, commercial backers of VistA are determined to
get it certified for meaningful use so that their clients can win
financial rewards from the Department of Health and Human
Services. But the VA doesn't have to be certified for meaningful use
and doesn't care about it. (As David Uhlman of ClearHealth pointed
out, nearly everything in the meaningful use criteria was done thirty
years ago by the VA using VistA.)

The VA even goes through periods of refusing bug fixes and
improvements from the outside community. Luckily, the VA lets some of
its programmers participate on WorldVistA forums, and seems interested
in getting more involved.

Other presentations

Attendance varies between 30 and 70 people for today's health care
session. Roni Zeiger of Google brought out a big crowd for his href="http://www.oscon.com/oscon2010/public/schedule/detail/15272">discussion
of Google's interest in health care, with a focus on how its API
accepts data from devices.

Zeiger pointed out that we lead most of our lives outside doctor's
offices (unless we're very unlucky) and that health information should
be drawn from everyday life as well. A wide range of devices can
measure everything from how fast we walk to our glucose levels. Even
if all you have is a smart phone, there are a lot of things you can
record. Collecting this kind of data, called Observations of Daily
Living, is becoming more and more popular.

  • One app uses GPS to show your path during a run.

  • Another app uses the accelerometer to show your elevation during a
    bike ride.

  • One researcher uses a sensor, stuck into an inhaler, to feed data to a
    phone and collect information on where and when people have asthma
    attacks. If we collect a lot of data from a lot of people over time,
    we may learn more about what triggers these attacks.

  • On the fun side, a Google employee figured out how to measure the
    rotation of bike pedals using the magnet in an Android phone. This
    lets employees maintain the right aerobic speed and record what how
    fast and their friends are peddling.

You can set up Google Health to accept data from these
devices. Ultimately, we can also feed the data automatically to our
doctors, but first they'll need to set up systems to accept such
information on a regular basis.

Will Ross href="http://www.oscon.com/oscon2010/public/schedule/detail/14944">described
a project to connect health care providers across a mostly rural
county in California and exchange patient data. The consortium
found that they had barely enough money to pay a proprietary vendor of
Health Information Exchange systems, and no money for maintenance. So
they contracted with Mirth
Corporation
to use an open source solution. Mirth supports
CONNECT, which I described in
href="http://radar.oreilly.com/2010/07/day-one-of-the-health-care-it.html">yesterday's
blog, and provides tools for extracting data from structured
documents as well as exchanging it.

Nagesh Bashyam, who runs the large consulting practice that Harris
Corporation provides to CONNECT, href="http://www.oscon.com/oscon2010/public/schedule/detail/15267">talked
about how it can lead to more than data exchange--it can let a doctor
combine information from many sources and therefore be a platform for
value-added services.

Turning to academic and non-profit research efforts, we also heard
today from href="http://www.oscon.com/oscon2010/public/schedule/detail/15279">
Andrew Hart of NASA's Jet Propulsion Laboratory and some colleagues at
Children's Hospital Los Angeles. Hart described a reference
architecture that has supported the sharing of research data among
institutions on a number of large projects. The system has to be able
to translate between formats seamlessly so that researchers can
quickly query different sites for related data and combine it.

Sam Faus of Sujansky & Associates href="http://www.oscon.com/oscon2010/public/schedule/detail/15275">recounted
a project to create a Common Platform for sharing Observations of
Daily Living between research projects. Sponsored by the Robert Wood
Johnson Foundation to tie together a number of other projects in the
health care space, Sujansky started its work in 2006 before there were
systems such as Google Health and Microsoft Health Vault. Even after
these services were opened, however, the foundation decided to
continue and create its own platform.

Currently, there are several emerging standards for ODL, measuring
different things and organizing them in different ways. Faus said this
is a reasonable state of affairs because we are so early in the
patient-centered movement.

I talked about standards later with David Riley, the government's
CONNECT initiative lead. HHS can influence the adoption of standards
through regulation. But Riley's office has adopted a distributed and
participatory approach to finding new standards. Whenever they see a
need, they can propose an area of standardization to HHS's
specification advisory body. The body can prioritize these
requests and conduct meetings to hammer out a standard. To actually
enter a standard into a regulation, however, HHS has to follow the
federal government's rule-making procedures, which require an
eighteen-month period of releasing draft regulations and accepting
comments.

It's the odd trait of standards that discussions excite violent
emotions among insiders while driving outsiders to desperate
boredom. For participants in this evening's Birds of a Feather
session, the hour passed quickly discussing standards.

The 800-pound gorilla of health care standards is the HL7 series,
which CONNECT supports. Zeiger said that Google (which currently
supports just the CCR, a lighter-weight standard) will have to HL7's
version of the continuity of care record, the CCD. HL7 standards have
undergone massive changes over the decades, though, and are likely to
change again quite soon. From what I hear, this is urgently
necessary. In its current version, the HL7 committee layered a
superficial XML syntax over ill-structured standards.

A major problem with many health care standards, including HL7, is the
business decision by standard-setting bodies to fund their activities
by charging fees that put standards outside the reach of open source
projects, as well as ordinary patients and consumers. Many standards
bodies require $5.00 or $10.00 per seat.

Brian Behlendorf discussed the recent decision of the NHIN Direct
committee to support both SOAP versus SMTP for data exchange. Their
goal was to create a common core that lets proponents of each system
do essentially the same thing--authenticate health care providers and
exchange data securely--while also leaving room for further
development.


PlanetMySQL Voting: Vote UP / Vote DOWN

At OSCON

Июль 20th, 2010

I’m at OSCON this week. Come say hi and talk Drizzle, Rackspace, cloud, photography, vegan food or brewing.


PlanetMySQL Voting: Vote UP / Vote DOWN

MariaDB at OSCON 2010

Июль 18th, 2010

We’re at OSCON and today has been fabulous – I’ve just been connecting with old friends, and making new friends, and all this is what makes the travel experience completely worthwhile. If you’re at OSCON, why not come to a couple of BoF’s:

  1. MariaDB: The Community Fork of MySQL at 8pm on Monday 19/07/2010. Great for a general overview of MariaDB, for beginners to the advanced folk to come to.
  2. MariaDB: Features In-depth at 9pm on Wednesday 21/07/2010. Great if you’re a more intermediate to advanced user of MySQL/MariaDB and want to know more about the additional features MariaDB has to offer, and what else it might offer in the near future (i.e. what are you requesting).

Related posts:

  1. MariaDB 5.1.42 released!
  2. Ubuntu 10.04 LTS released, MariaDB 5.1.44/5.2-BETA VM’s available
  3. MariaDB 5.1.44 released



PlanetMySQL Voting: Vote UP / Vote DOWN

Speaking in the US — San Francisco User Group — Community Summit — OSCON

Июль 5th, 2010
Giuseppe in US On July 15th and 16th I will be in San Francisco for a few meetings, and it will be my pleasure to meet the San Francisco MySQL User Group, where I will talk about MySQL Sandbox.
From there, I will continue to Portland, to attend the Community Leadership Forum and, of course OSCON, where I will have three talks: two in the main event, and one at the Intel booth.

OSCON 2010

PlanetMySQL Voting: Vote UP / Vote DOWN