The latest episode of our “Meet The MySQL Experts” podcast focuses on MySQL & PHP! Andrey Hristov and Johannes Schlüter from the MySQL Connectors Team talk about various ways PHP developers can use MySQL, and also how they can leverage the MySQL Query Analyzer to optimize queries.
Enjoy the podcast!
For additional information about MySQL & PHP, check out our last special edition newsletter.
The latest episode of our “Meet The MySQL Experts” podcast focuses on MySQL & PHP! Andrey Hristov and Johannes Schlüter from the MySQL Connectors Team talk about various ways PHP developers can use MySQL, and also how they can leverage the MySQL Query Analyzer to optimize queries.
Enjoy the podcast!
For additional information about MySQL & PHP, check out our last special edition newsletter.
Over the last few weeks I had been quite silent, but that's about to change: Over the next few weeks I'll give a few presentations. Feel free to join any of those.
Cloud technologies and services increase every day. Global players like Amazon put continuously new features to their cloud and products to provide sufficiently functionality of actually web development activities and requirements.
In fact, Amazon starts with the S3 (Simple Storages Service) to provide cloud storage space since 2006 and only a few months later EC2 (Elastic Compute Cloud) was launched a service to provide computing power to manage capacity as many as we need over time. Now, RDS (Relational Database Service) is available to get elastic database power as much as we need to cover all components that we require to implement the whole stack of current web applications.
Use that services to transform the development and of course every application in a liquid and full dynamic status. Every service increase productivity and can be used to cover architecture requirements in sufficient way. That means, use every service as long as we need and allocate capacity as many as we need in the finest way up to 100% not more, not less to reduce resources and costs.
Ok let us start to introduce each important service below to get more details and show examples what we have to do to use these services. Amazon provides a SDK especially for PHP developers to connect each service. You can download the SDK under the following URL:
http://aws.amazon.com/sdkforphp/
To start, download and install the SDK to your destination directory in your application root folder. Before we can start edit the configuration file and change the privacy settings with your AWS (Amazon Web Services) credentials (see Listing 1). To get credentials please create an AWS account on:
http://aws.amazon.com/de/ and follow the instructions.
S3 is the storages service from Amazon where we can store (Create), retrieve (Read), overwrite (Update) and remove (Delete) data. The logical structure based on buckets that contains items like files e.g. images, videos or documents or any data that we want to store. Buckets must have a unique developer-assigned key across the account each item can be contain 1 byte up to 5 terabytes.
First, to store our objects we create a bucket with the method create_bucket. All what we have to do is specify a name of the bucket testbucket and the region where we want to create it. In our case the region is EU_E1 (European 1- Region). Additionally, a simple if check covers the return value whether the call was correct.
In the next step, we put files to the bucket and store some data to the cloud. Simply, call the create_object method to handle our request.
Listing 3 shows the PHP snippet to create a test file. Call the method with the bucket name testbucket, name of the file testfile.txt and some additional parameters in our case body, acl and contentType to specify the characteristic of our data.
Now we want to get back our stored data. To finalize that request we have to use the method get_object.
Listing 5 covers our example and echoed the string that we have inserted a few minutes before.
To see it is very simple to communicate to our cloud storage with the SDK and execute the proper methods. If you do not know the right method take a look in the library service file where every single method is described well that enclose every required parameter and excepted return values.
EC2 is the next create cloud services that we can use to allocate computing resources for our web environment. Through defined methods from the SDK and the proper service file you can easily start an AMI (Amazon Machine Image) with an operating system that match our requirements. There are images from the market leaders like Linux and Windows systems and you can upload an own image as well whether you need a self defined system e.g. pre-installed web, db or chat server. Please go to the Amazon website:
http://aws.amazon.com/ec2/
to see which kind of images are available that fits your requirements. After we have chosen a suitable images we can start it with a method call from the SDK similar create a bucket in S3.
Listing 6 shows that method call run_instances with the parameter ami-48aa4921 with specifies the machine images that will be loaded and some optional parameters in that case the type of the underlying hardware configuration which is specified by m1.micro instance - currently the smallest type with 2 processor kernels (CPUs) - and round about 613MB RAM. Check the response value with the method isOk. If everything is fine we can connect our recently started machine through a secure shell.
Listing 7 visualize a connection to the virtual machine that we have started. We need only our private key that you can download from the AWS account and the hostname of the machine that you can find on the EC2 overview simply go to the AWS website:
http://aws.amazon.com/ec2/.
After the login (see Listing 8) we see our secure shell of the machine where we can input commands like on our self administrated local systems.
Similar S3, EC2 is also pretty easy to use with some SDK commands like run_instances that helps us to integrate this service into our web application. You need only a valid AWS account (key credentials) to access the service, a machine image like ami-48aa4921 and a proper instance type e.g. m1.micro with sufficient processor and memory power.
The final service that we want to cover is the RDS service the third part to complete our web environment stack. It is also pretty easy to use it with the SDK. Call only a single method to start a new RDS instance and connect to the machine.
See Listing 9 to show an example how to create a database instance. Execute simple the method create_db_instance with the parameter testinstance the name of the instance, 5 specifies the size of the db storages in GB, db.m1.small clarifies the size of the underlying instance type (varies in processor and memory power), MySQL the type of the database system, testuser a user name and testpassword the password to connect the database. Check also with the isOk method the response value to guarantee the availability of the system.
Now we can connect to our new instantiated database through a secure shell. Input only the following known MySQL client command to connect. Specify the user name, password and the database host (see Listing 10).
After we have connected to the database server we will see the start screen where we can input commands (see Listing 11).
Check out which tables are available and execute the show databases command. Additionally, show all tables in the MySQL database with the show tables command (see Listing 12).
In this blog post we have cover common cloud services from the Amazon service. S3, EC2 and the RDS are services to extend and of course to replace current development architectures where normally dedicated machines and systems work. With the SDK it is very simple to connect all the services, request commands and response values e.g. S3 items. These proposed services cover a small overview about available services. There are much more excellent functionalities we can use to integrate.
SimpleDB is an additional database service that based on a key value store without any relation between items unlike relational database systems. Handle data in domains similar buckets in S3 and retrieve values with proper SQL statements.
Last but not least, SQS (Simple Queue Service) integrate a SOA service. Loosely coupled items communicate through defined interfaces. Add values to a queue at one application point and retrieve items on another. Works asynchronous and can be used to handle jobs that converts large data like image or video rendering. Thank you for your attention to read until the end ;-) if you have any comments please add your notes and ideas after that post.
Today's topic is deployment. It's called one-click deployment for a reason: Developers are lazy.
It's hard to do less than clicking on one button, so that's our goal.
With the growing need for lower time-to-market and faster response to user feedback it is inevitable to not be limited by technical factors (there are enough other obstacles already). The focus lies on reproducible results.
So, what do we need? Actually, not much. Disregarding the tools and practices that build the foundation of agile software development, you only need a central build server. But you've already got that one covered, right?
If you don't, you should get one. It's a huge help to discover errors quickly and be alerted instantly. This usually leads to a shorter time frame until a fix is done. Tests are run continuously and new parts are integrated into the whole code base.
Everything written here is based on Apache ant, so the easy choices here are Hudson and Jenkins. We're focusing on Jenkins here, but we've successfully used some variation of this continuous integration/deployment script on both.
We've also used CruiseControl/phpUnderControl and Atlassian Bamboo in the past for continuous integration, so the deployment parts should work as well.
Back to the hard facts. We're using the Template for Jenkins Jobs for PHP Projects here for our basic checks, so let's assume you got it installed and your build server is working.
The typical workflow for a developer now consists of these steps:
There's not much deployment in this, so a few more steps:
There are a few preconditions in our environment that haven't been named yet. Not all of them are hard requirements, but they make some things a lot easier:
The frontend nodes use a setup including varnish, nginx, PHP 5.3+APC, ZF 1.11, Dojo 1.6 and Doctrine 2.1.
The backend nodes use MySQL(Master+Slave), Apache Solr, RabbitMQ, memcached and ejabberd.
It basically executes these steps:
And that's it.
The manually executed task is called deploy-to-staging and consists of these steps:
Here's one caveat: We've set up our staging (and production) nodes to include a directory (like /opt/repository) in /etc/apt/sources.list and puppet periodically checks if there are new packages in this directory, and installs it. Of course you could just call dpkg -i through ssh as well.
So far, so good. After waiting 5mins we've got our "release 1.32c" deployed to all 4 staging nodes and the testers can do their work.
We're not quite done yet though. The testers really liked release 1.32c and we want to deploy to live now!
So this means:
In greater detail:
If everything went well, you just deployed a new release!
The first thing we noticed is the build time. The build used to take 15 minutes because of massive disk I/O. Mostly all the analysis/metrics tools are to blame, if you have a few of those, every single one has to read and check all of your hundreds of files even if you exclude the external dependencies and libraries (which is a good idea, anyway). We put the files on a ram disk and brought the build from 15 minutes down to 4. Good enough.
The setup is quite complex (as you've seen if you're still reading). It takes a bit to get used to and there are some moving parts. But so far it's working nicely.
Jenkins is a single point of failure. If your build server goes the way of the dodo, it's very hard to get a release going. There's an easy fix, however: Have a second machine ready that's running an identical copy of your Jenkins setup. If your main build server is down you can still use that one then. As your developer VMs are using the same ant build.xml for daily work, even using that as a last resort is possible.
There are also some goodies that aren't crucial, but nice:
Having talked with developers from other companies there seem to be quite a few similar setups including chef, rake, phing or even make - so you're surely not forced to use ant or a debian-based distribution.
Good luck simplifying your own deployment!
Object-relational mapping (ORM) frameworks have been around for several years now and for some people, ORM is already outdated by now. As we have seen with other technologies and concepts before, PHP is not exactly what we call an early adopter among the programming languages. Thus it took some time for ORM to grow up in the PHP context.
There have been some frameworks before Doctrine 2 that implement ORM (remember e.g. Propel) specific tasks but most of them lack the required maturity to be used in large projects. With Doctrine 2, PHP takes a huge step into the right direction Doctrine 2 is fast, extensible and easy to use.
This article will take you on a tour through the main concepts of Doctrine 2 in the first part and then explain how to use it in a real world application in the second part. Since at the time of writing Zend Framework 1.11.xx (ZF) is very popular, we will integrate Doctrine 2 into a ZF project.
To understand Doctrine 2, we have to take a look at some relevant terms (or in this case objects), study their behavior and practice their usage. We start with some introductory phrases on ORM systems and then go on to the concepts underlying Doctrine 2: Entity Objects, the Entity Manager, Repositories and Proxies.
Since the beginning of Object-Orientation, people had to manage the persistence of their application's state resp.
their objects. In the context of Web Application Development, this usually involves a Database server which is being
consulted using a Query Language. One example for this pattern is a PHP application that uses some kind of SQL server
by sending SQL queries to it. Another one is an application using a CouchDB server by querying it via its REST API.
Due to the author's laziness, we will talk in terms of relational databases from now on. Keep in mind, that you can
accomplish almost everything mentioned here with NoSQL databases, too.
ORM relates value objects that exist in an application's business logic to database records.
Thus every object that should be persistent is saved in one row of a database table. The most common approach is to
map classes to tables and the classes' objects to rows in the these tables.
Besides writing objects to a database, ORM systems are also intended to ease the process of finding data stored in the
database. When talking in terms of ORM, finding data always means making the framework fetch one or many objects
that meet a certain criteria.
The objects that are being managed by an ORM system are called Entity Objects. Every entity object relates to one entry in a table. In Doctrine 2, the classes that represent entities do not have to fulfill special requirements like inheriting from a certain super class (as you might have seen in other database abstraction frameworks like Zend_Db). When creating a new entity class with Doctrine 2, all you have to do is to write down a regular PHP class with properties. Besides this, you have to provide some hints on how these attributes should be persisted. The information how entity attributes relate to columns in the DB is called Metadata. Metadata can be described in different ways: By default there are metadata drivers for descriptions in XML, YAML and PHP. The fourth and most popular driver is based on DocBlock annotations (since in PHP, annotations aren't a language feature as in Java (see Wikipedia), they are contained by the classes' and attributes' DocBlocks). We will use annotations to describe our entities metadata. To get an impression on how easy this is, take a look at the following example.
This example contains all it needs to tell Doctrine 2 about the new entity User. With this class, you can create,
find, delete and modify user objects and persist their state to the underlying database. But keep in mind: as long
as you don't need any persistence features, you can use your user objects just like any other objects!
The next two objects resp. object types we will describe are responsible for doing the ORM functionality: persisting and finding.
To use ORM functionality, the Entity Manager (Doctrine\ORM\EntityManager) is the main access point to Doctrine 2. The entity manager is
responsible as you might have guessed for managing entities and for building a facade for the whole framework.
To accomplish its tasks, the entity manager uses some helpers. The Unit of Work object for example collects entities
that should be written back to the database and is capable of doing this in batches. This way, database operation
can be executed with almost no overhead and therefore are really fast.
Another dependency of the entity manager is the Event Manager. To be as extensible as possible, Doctrine 2 comes with an event system that publishes all important state changes to the outside as events. You can register for such events and extend the life cycle of your entity objects at one single point.
The entity manager's API combines methods for managing entities (find, persist, contains, copy, detatch, merge,
remove and refresh), methods that control the use of transations (beginTransaction, commit, flush, rollback and
transactional) and some helper methods for creating custom queries and accessing some of the entity manager's dependencies.
The following example shows how to query an object from the entity manager, modify it and write the changes back into the database.
Creating a new persistent object is almost as easy as modifying it:
For finding entities, Repositories are used. Every entity class has its own repository which is responsible for finding entities of that type. By default, repositories have some handy methods for fetching entities that match certain criteria:
find: Finds an entity by its primary key / identifierfindAll: Finds all entities of the repository's entity typefindBy / findOneBy: Finds all resp. one entity that matches the passed criteria:findBy<attribute> / findOneBy<attribute>: Magic methods that ease the filtering by a single attribute:
To access a repository, all you have to do is ask the entity manager for one. If you have implemented your own
repository, it will be returned by Doctrine\ORM\EntityManager::getRepository(). Otherwise, Doctrine 2 will provide
a generic repository. The main reason to implement custom repository classes is to group custom queries for an entity
type to make them reusable. For custom query logic, there are several mechanisms you can use: You can either use
Doctrine's query builder that implements an API similar to Zend_Db_Select or queries written in the Doctrine Query
Language (DQL) or you can even execute plain SQL queries. With these options, it is also possible to migrate old
applications which use complex queries by just wrapping these queries into the methods of custom repositories.
When traversing a graph of entity objects (which is required when entities are having relations to other entities),
it would be very expensive (in the sense of requiring many database queries) to fetch every depending entity with
an additional query. Therefore Doctrine 2 uses the concept of Proxy objects that represent regular entity objects
which have not been populated with data from the database. Take a look at the following example where the entity
Group aggregates a list of User objects in its member property. When accessing the members list, Doctrine 2 provides a
collection of proxy objects instead of complete User objects. When an object of this collection is being asked for
one of its properties, Doctrine loads the object's data from the database. This way, the users' data is not loaded
until it is really needed.
This section describes some advanced concepts that are required when mapping entity classes that have relationships to
other entity classes. Possible relationship types are association and inheritance. Inheritance is the mechanism used for
representing subtypes in object-oriented programming languages. An example would be a class User that implements
methods every user of a software should have and a class Administrator inheriting from User that adds methods for
determining the administrator's access rights.
Association is a weaker relation type. It means that an entity object can be related to other entity objects of other types. In terms of relational databases, there are three types of association which differ in the number of entities an object is related to: 1:1, 1:n and n:m relationships. n and m are placeholders and mean multiple.
To put objects of an entity type into relation, you just have to mention this relation in the entity class' mapping
information. The simplest case is a unidirectional 1:1 relationship. In the following example we describe a User entity
which has its access information (user credentials) encapsulated into another entity class called UserCredential. Since
every user has at most one credential object and every credential object may only be associated to one user object,
this is a 1:1 relationship.
If the relationship should be bidirectional, include the OneToOne attribute in the other class, too, and add an
attribute which denotes the attribute of the other entity that mapps the related object:
This way, you can access the user object from the credentials object, too.
Most of the times, developers have to deal with relationships which include many objects on at least one side.
These relationships are called 1:n or n:m relationships. This means that either one or multiple entities are standing
in relationship with an arbitrary number of entities of another type. To accomplish this, you have to use the mapping
keywords OneToMany or ManyToMany when describing your entities. Besides that, the mapping works the exact same way as
with 1:1 relationships.
There are however some tricks you should know when dealing with collections of associated entity objects. Consider
the following relationship between the entity classes User and Group:
When a group has at least one member, the group object will have a collection of the type
Doctrine\Common\Collections\ArrayCollection set as its members property. This collection contains
all user objects (or proxy objects as we have seen before) and can be modified intuitively with the methods add and
removeElement. To honor object-orientation, you might want to introduce custom methods for these tasks. If you do so,
you get into trouble when the group object does not have any users associated. In this case, the collection will simply be
set to null. To avoid checks whether the collection has already be initialized, you should to this by yourself
in the entity class' constructor:
It is also important to notice that one entity has to update the other entity's state as well when a relationship between to objects is created or removed. Take care to do this only in one class to avoid endless recursion loops! This class is called the Owning Side of the relationship. When implementing a bidirectional relationship, the other class is called the Inverse Side. It is important to determine owning and inverse side and implement the the classes accordingly to avoid greater trouble during debugging.
There are some more features implemented by Doctrine 2 enabling developers to specify their entities' relationships including sorting, pre-fetching and indexing. These topics are not covered in this article but are explained very understandable in the Doctrine 2 documentation.
Subtyping can be implemented in different ways using Doctrine 2. The main difference between these implementations is how the inheritance is mapped to the database. The options are to have one table for every class (Class Table Inheritance), to have one table for all classes in a hierarchy (Single Table Inheritance) and to have a table for every specialized sub-class of a given super-class (Mapped Super Class).We will give a short overview on all three alternatives, you have to pick the right one yourself. This decision should be made based on how many common attributes there are in your sub-classes.
Introducing a mapped superclass is probably the easiest way for specifying inheritance but might lead to many duplicate columns in your database schema. The superclass of your entities is not being declared as an entity itself (and might also be declared abstract) but provides attributes and optionally methods that will be available in all subclasses. When creating the database schema, Doctrine 2 merges all attributes and relationships of the superclass into the definitions of the subclasses and processes them as regular entities.
After creating the database from this mapping information, your tables will look like this:
When having entities that are very similar besides some few attributes, you might want to store them together in one database table. This approach is called Single Table Inheritance. To distinguish between the different types, there is always a column marked as discriminator column and a discriminator map that tells Doctrine 2 which values in the discriminator indicate what entity types.
These definitions cause the existing of one single table called User with all the attributes declared inside the classes User and Administrator plus a column type the discriminator column. When working with entities of these types, Doctrine will manage the type flag automatically for you.
The resulting database schema looks as illustrated by the following diagram:
Having each entity type stored in its own table is always good for keeping your schema extensible. When you have to create a new subtype, Doctrine 2 will just create a new table for this type and it can inherit the logic and common attributes of a superclass. The only overhead you have with this approach is that all tables that correspond to subtypes have to maintain a relationship to their supertype's table. Using class table inheritance, the example with the entities User and Administrator looks like this:
Besides the inheritance type, there is no difference to the example using single table inheritance. The outcome on the resulting database scheme is huge. Now you have to separate tables which store users and administrators. Every record in the table Administrator has a corresponding record in the User table.
This was the first part of this article. Stay tuned for part II which will be published tomorrow (on 6th of December 2011)! In the second part, we will integrate Doctrine 2 into a Zend Framework application and include a generic sandbox (ZF-)project with Doctrine 2!