WordPress on S3: no more backups

How much trouble will it be if your webserver failed?  No trouble at all, if your website keeps its content on reliable Amazon S3 storage.

There are a lot of nuances in ensuring proper backups and restores of websites. When was the last backup taken? How much data might have been lost? How long will it take to recover it? When was the last time you tested restore? Do you even have an offsite backup?

Now that you can run dynamic websites off Amazon S3 storage, we’ll demonstrate why you no longer need to worry about backing up and restoring your website data. Losing the webserver is no longer a disaster. Cloud storage offers almost unsurpassable reliability a lot of website owners (small & large) would benefit from. In a way you get an "instantaneous backup" to the cloud. Your data is always safe with the cloud storage. The webserver can simply be replaced in minutes.

Contents

OblakSoft has released the 1^st ever dynamic WordPress site running on top of Amazon S3: Yapixx.  Yapixx is ready-to-run WordPress on S3, it stores all its data (content and media) in Amazon S3 storage.  Yapixx stands for Yet Another Picture Sharing Site.

Yapixx taps into power of Amazon S3.  Amazon S3 is inexpensive, highly reliable, available and scalable storage service.  Using Amazon S3 to store Yapixx data has the following benefits:

• No backup and recovery of the site data is needed
• Storage is extremelyreliable and durable by Amazon S3 design
• Pictures are served by Amazon S3 directly, which makes Yapixx highly scalable
• Storage cost scales with usage, no upfront reservation is needed
• Storage consumption scales up and down with the amount of data stored

The crash test is going to illustrate the first point: no backup and recovery is needed.  The data for the web site is safely stored in Amazon S3, so the machine that Yapixx runs on can be replaced in minutes.

Crash test. Start the Webserver.

To get started with the crash test, follow the five steps to deploy Yapixx:

1. Sign up for an AWS account.
2. Create an S3 bucket.
3. Start EC2 instance using read-only Yapixx AMI.
4. Connect to the web application from a web browser.
5. Enter the S3 data location and authentication information.

Refer to the complete step-by-step guide for extra pointers in setting up Yapixx.

Now Yapixx is up and running and you can upload pictures.  Upload some pictures to Yapixx.  Make sure the pictures are uploaded successfully.

Crash the Webserver – it is a crash test after all.

Then terminate the EC2 instance that is running Yapixx.  As a result the server should get completely destroyed.

Is all lost?  Not at all! 

Launch the replacement Webserver.

To get a new server running, repeat the steps 3-5 of the deployment instructions:

3. Start EC2 instance using read-only Yapixx AMI.
4. Connect to the web application from a web browser.
5. Enter the S3 data location and authentication information.

The full step-by-step guide for Yapixx crash test is available at here.

Make sure that you enter the same S3 data location information!   Yapixx is going to ask for a confirmation that this is the only instance that accesses the S3 data location: running multiple instances accessing the same S3 data location may lead to data corruption and data loss.

Feeling adventurous? Change the Webserver machine.

To add extra thrill, try using a different instance size: if you originally deployed a micro instance, try using a small instance, or vice versa.  This will get you a feeling of how easy it is to scale servers up and down.  Now, that’s truly elastic!

It’ll be even more fun if it was your WordPress site, will not it be? You can rely on Yapixx as the starting point to take your site to Amazon S3. Launch your own WordPress site to the cloud today!  It’s easy and FREE, no writing code is required.  Start at http://www.oblaksoft.com/downloads.

We hope you enjoyed the crash test!  Are you interested in making your site highly available with the help of cloud storage?

See also

WordPress on S3: run a beautiful website on Amazon cloud storage.

WordPress on S3: how it works.

WordPress on S3: the beauty of simplicity (blog).

My first computer program was written almost quarter a century ago on a BK-0010 computer.  It was very simple: the program asked the user to enter their name and then greeted the user using the entered name, like “Hello, Artem!”.  I was fascinated.  A couple of lines written in Vilnius BASIC transformed a piece of metal and silicon into a considerate thing that cared about a person’s name enough to remember it :-).  Of course, the first experience doesn’t represent the day-to-day routine of software development, but the moments when I see a couple of lines making an amazing transformation still enchant me, and remind me why I’ve been writing code all this time.

I’ve just experienced this very same first-time feeling as we’ve released Yapixx – a picture sharing web application using the cloud storage.   The most amazing thing about Yapixx is that we wrote very little code to make it happen: most of its functionality is provided by WordPress, which by the way we didn’t modify at all. 

On one hand Yapixx is just WordPress, enhanced with plugins and configured to provide good picture sharing experience. 

On the other hand, Yapixx has gone where WordPress could not go before – Yapixx runs completely on top of Amazon S3, using the enormous power of S3 to make serving the users’ pictures highly scalable and storing all data extremely durable. 

As an engineer I’ve been on the never ending quest of finding new ways to do more with less.  How do I write less code and provide more functional solution?  How do I empower my customers to accomplish more with applying very little effort (or preferably no effort at all)?  Yapixx is one of those gems – it shows how unbelievably simple it is to run a beautiful website on top of Amazon S3: you don’t need to learn new APIs, you don’t have to know how to write code, you don’t even have to know that you are relying on MySQL + ClouSE to store data reliably and securely in the cloud!

Yapixx is not a toy application: it’s a fully functional ready-to-run WordPress on Amazon S3. With a few clicks Yapixx can be transformed into anything WordPress can do, while storing all website content in Amazon S3.  Just continue doing what you set to do with your Web site – to build a beautifully powerful representation of you, your company, and your cause.

Amazon S3 is a very powerful service designed for building one-of-a-kind massively distributed applications.  This is its strength but it’s also its weakness: if you are not building one-of-a-kind massively distributed application, the low-level vendor-specific APIs and eventual consistency guarantees are just the unnecessary complexities that you pay for, but don’t use.  WordPress is an example of how to make Amazon S3 a true cloud storage utility service that can be easily used by millions.  It’s a people-oriented (as opposed to technology-oriented) approach to building a distributed system – as a constellation of beautiful websites powered by creativity and uniqueness of individuals, like the Internet itself is!

‘Nuff said :-).  Get your own WordPress on S3 for free now.

Artem

WordPress on S3: run a beautiful website on Amazon S3 cloud storage

OblakSoft is proud to introduce the 1^st ever dynamic WordPress site running on top of Amazon S3: Yapixx.  Now you too can launch your own beautiful website on Amazon S3.

While Yapixx stands for Yet Another Picture Sharing Site, it is actually one of a kind.  Yapixx is WordPress that was moved to run on top of Amazon S3 storage without changing a line of code in the WordPress core engine.

What is Yapixx?

Yapixx is ready-to-run WordPress on Amazon S3.  It took about one man-week to create Yapixx, thanks to power of WordPress.  WordPress is a mature and versatile platform that powers millions of websites with diverse purposes from blogging to social networking.  There are thousands of plugins and themes written for WordPress that can transform the website into pretty much anything.  And now it can run with Amazon S3 without writing a line of code.

Yapixx taps into power of Amazon S3.  Amazon S3 is inexpensive, highly reliable, available and scalable storage service.  It got even less expensive recently and its usage is growing 2.5-2.9 times year-over-year.  Using Amazon S3 to store Yapixx data has the following benefits:

•          Storage cost scales with usage, no upfront reservation is needed
•          Storage consumption scales up and down with the amount of data stored
•          Storage is extremely reliable and durable by Amazon S3 design
•          Pictures are served by Amazon S3 directly, which makes Yapixx highly scalable
•          No database backup and recovery is needed, now that Amazon S3 holds both for the website content stored in WordPress database and website’s media files

Yapixx demonstrates how easy it is to create web applications on top of Amazon S3.  Amazon S3 has been able to only power static websites; now S3 can power sophisticated dynamic web software such as WordPress.

Try out Yapixx

Here are the five easy steps to get Yapixx up and running:

1. Sign up for an AWS account.
2. Create an S3 bucket.
3. Start EC2 instance using Yapixx AMI.
4. Connect to the web application from a web browser.
5. Enter the S3 data location and authentication information.

The full step-by-step guide for launching Yapixx is available here.

All done!  You’ve got a picture sharing site running on top of Amazon S3.

Get started with Yapixx today for FREE!  It may be the boost that you need to propel your creative ideas and turn them into the next $1B acquisition.

Take Your Own WordPress Site to Amazon S3

You can use Yapixx as a starting point to create your own beautiful WordPress site.  The plugins that turn WordPress into Yapixx, such as the wp2cloud plugin, the Yapixx theme, etc. are available under the GPL completely FREE.  Creating a new AMI with your own web application is a matter of two mouse clicks in the AWS management console:

Moreover, you may even get your web application hosted completely free in the Amazon cloud, if your AWS usage doesn’t exceed the AWS free tier limits.  This is a great way to get a new web application to production quickly, absolutely FREE!  And when your web application usage grows, you can easily move it to larger cloud instance or to your own hardware: the data is stored in Amazon S3 and can be accessed from anywhere.

Launch your own WordPress site to the cloud today!  It’s easy and FREE, no writing code is required.  Start at http://www.oblaksoft.com/downloads.

Let us know what you think about Yapixx!  How can we further simplify the migration of websites to the cloud storage?

See also:

WordPress on S3: how it works.

OblakSoft is pleased to showcase how simple it is to run LAMP applications on the cloud storage.  OblakSoft configured the WordPress web publishing platform to run on Amazon S3 storage and made the recipe available for anyone to replicate.  A ready-to-run WordPress site (configured as Yapixx) – is available for public use for FREE.  Yapixx is WordPress configured as a picture sharing website that runs on top of Amazon S3 (Yapixx stands for Yet Another Picture Sharing Site).

Yapixx consists of the following software components:

• ·         WordPress configured as a picture sharing site
• ·         Yapixx theme based on zBench with modifications to look better for picture sharing
• ·         wp2cloud plugin that stores pictures and other media in Amazon S3 via ClouSE
• ·         LAMP software stack (Linux, Apache, MySQL, PHP)
• ·         ClouSE – the Cloud Storage Engine for MySQL

The Yapixx architecture looks like the following:
 

The WordPress platform provides the web user interface.  WordPress stores the web site content in a MySQL database.  MySQL uses ClouSE to store data in Amazon S3.  The end users use a web browser to interact with WordPress; web pages come from WordPress.  All the pictures (and other media) in the pages are served by Amazon S3, where they are stored as weblobs by WP2Cloud WordPress plugin.

WordPress is a mature and versatile platform that powers millions of websites with diverse purposes from blogging to social networking.  There are thousands of plugins and themes written for WordPress that can transform the website into pretty much anything.  And now with ClouSE WordPress can run on Amazon S3.

ClouSE is the key component of Yapixx application.  ClouSE is the Cloud Storage Engine for MySQL that can utilize cloud storage such as Amazon S3 to store user data.  Applications powered by ClouSE enjoy all the benefits of cloud storage – scaling cost with usage, high storage availability and reliability, quick and easy disaster recovery, etc.  And weblobs that store pictures for Yapixx can be served directly from cloud storage relying on its enormous power to make serving the content highly scalable.

ClouSE is the first and only relational database technology that overcomes cloud storage adoption barriers that prevents other databases from using cloud storage.  With ClouSE, cloud storage can be just plugged into a database server seamlessly and securely.

ClouSE and Amazon S3 provide seamless and secure data protection, scalability and availability for websites, static or dynamic.  A website on cloud storage has the following characteristics:

•          Storage cost scales with usage, no upfront reservation is needed
•          Storage consumption scales up and down with the amount of data stored
•          Storage is extremely reliable and durable by Amazon S3 design
•          Data is compressed, which significantly reduces storage size and cost
•          Data is encrypted, which provides data confidentiality
•          Pictures are served by Amazon S3 directly, which makes the website highly scalable
•          No database backup and recovery is needed, which makes the website highly available

ClouSE brings the power of Amazon S3 to websites, providing an easy and inexpensive built-in solution for scenarios that previously required expensive infrastructure and operation.  Overprovisioning primary storage, implementing onsite + offsite backup and replication are not needed any more.  There is no need to implement and routinely test disaster recovery procedures.  No need to buy more expensive software editions.  The end-to-end data continuity is in the box: no extra tools, processes or equipment is required.  That’s the true realization of the cloud promise – commoditization and democratization of solutions that previously could only be afforded by large IT organizations with sizeable budgets.

The most amazing thing about Yapixx is that WordPress platform was not initially designed to store its data in cloud storage.  Yet, now it runs on Amazon S3 without any changes to the WordPress core engine!  We like to say that Yapixx was assembled and configured for the cloud, rather than designed and implemented for the cloud.

Launch your own WordPress site to the cloud today!  It’s easy and FREE, no writing code is required.  Start at http://www.oblaksoft.com/downloads.

We’d love to know what you think of Yapixx.  Are you considering to move your website to the cloud storage?

See also:

WordPress on S3: run a beautiful website on Amazon cloud storage.

OblakSoft is pleased to announce the release of WebStor 1.0b.1.1– the library providing high-performance cloud storage access.

WebStor is used in ClouSE – the Cloud Storage Engine for MySQL.   ClouSE makes cloud storage to be a drop-in replacement for local storage.   The outmost efficient and reliable cloud storage access is one of the key requirements that make the solution viable.

We’ve got a lot of questions about how ClouSE achieves such a great performance working with cloud storage.  Cloud storage has a perception of being slow, but ClouSE works with cloud storage as fast as with local storage.  WebStor is one of three pillars for high-performance cloud storage access (compression and caching are the other two).

In our benchmarks we were able to achieve up to 80+ MB/s transfer rate with Amazon S3.  While ClouSE needs much, much less than that, thanks to compression and caching, when push comes to shove, WebStor is up for the job.

Here is the WebStor performance benchmark chart for transferring data from / to Amazon S3:

The chart shows how the transfer rate was growing, as the number of parallel operations was increasing from 1 to 64.  It started with ~8 MB/sec with one operation running at a time, and grew all the way to ~80 MB/sec with 64 parallel operations.  The benchmark was run in the Amazon US-standard region.  The object size used in the benchmark was 1 MB.   The benchmark source code is included along with the WebStor library.

WebStor has the following features:

-          Supports Amazon S3 and Eucalyptus Walrus.  Other cloud storage provides may be added in the future.

-          Runs on Linux and Microsoft Windows.  Should be easily portable to Linux-like platforms.

-          Supports parallel put / get / del operations which allow achieving high throughput.

-          Supports SSL and plain-text HTTP.

-          Supports HTTP proxy.

-          Supports Amazon S3 multi-part uploads.

-          Provides C++ API.  APIs for other programming languages may be added in the future.

WebStor source code is available under the Apache 2.0 license absolutely FREE.  Get your own copy now from at http://www.oblaksoft.com/downloads.

I went to a GSLUG meeting last Sat and gave a talk about ClouSE. The slides are mostly the same as here, with the addition of the cloud database technologies comparison chart:

RDBaaS stands for Relational Database as a Service.

Data confidentiality: ClouSE is the only cloud database technology that provides full data confidentiality in the cloud. To guarantee data confidentiality in the cloud ClouSE encrypts the data using military grade AES-256 encryption before it puts the data to the cloud storage. This ensures that the data is visible neither over the wire nor to the cloud storage provider. The data confidentiality is fully controlled with the encryption key that the customer provides to ClouSE. Customers are in charge of encryption key management and fully control who has access to their data, if anyone.

Fast DB access: ClouSE can run on the same machine as the server application (e.g. Web Server), so access from the server application to the database is fast and reliable. Other cloud databases need to be accessed over the Internet, so the server applications need to be re-designed to deal with longer response times and higher failure rates when accessing the database. Most server applications are designed to assume that database access is fast and reliable; ClouSE fully supports that assumption.

ACID transactions, Secondary indices, SQL: NoSQL services don’t support ACID transactions, secondary indices or structured queries. Both ClouSE and RDBaaS provide standard, well-known and widely used relational database management.

Storage elasticity: With RDBaaS, the primary storage size needs to be specified upfront. Users are charged for reserved storage, not for used storage. The storage consumption cannot shrink even if some of the data is deleted. With ClouSE and NoSQL services, the storage consumption grows and shrinks with the amount data, users pay only for storage they use.

Automatic scale out: I put a yellow dot for the automatic scale out for ClouSE to reflect its ability to scale out content delivery from cloud storage directly. Applications that manage a lot of content (e.g. images, videos) may achieve significant scale-out.

ClouSE version 1.0b.1.0 released

OblakSoft is pleased to announce the release of ClouSE version 1.0b.1.0. This release addresses reliability and performance issues that were encountered by our early adopters, as well as it provides new functionality to better align with real life usage scenarios.

• The weblob URLs now support user-defined names.
• The weblobs now has to be of the LONGBLOB type.
• The AUTO_INCREMENT fields have now have semantics that is similar to InnoDB’s “interleaved” lock mode.

Thank you all for your feedback!

• The weblob URLs now support user-defined names.

We’ve got a lot of feedback that auto-generated names are not SEO-friendly, so instead of specifying an extension (like jpg, gif, etc.) the user can now specify a name (like creative-thinking.jpg).

• The weblobs now has to be of the LONGBLOB type.

The BLOB type only supports up to 64 KB of content. Users who followed the examples in our documentation ran into what looked like data corruption: uploading more than 64 KB with streaming worked well, direct access by URL worked well, but getting the content from the SELECT query resulted in content truncation. To avoid confusion, ClouSE now enforces LONGBLOB type for weblob fields.

• The AUTO_INCREMENT fields have now have semantics that is similar to InnoDB’s “interleaved” lock mode.

Previously, AUTO_INCREMENT was implemented as a global monotonically increasing counter; the existing values in the table were not taken into account. The old implementation did not work well in cases like restoring tables from a backup taken by mysqldump: the values from the backup could conflict with the new values produced by the ClouSE’s global counter. The new implementation uses existing values to generate new auto-incremented values, such that the new values are higher than any existing value in the table.

 The full list of changes included into ClouSE version 1.0b.1.0 can be found here.

Download ClouSE now for FREE at  http://www.oblaksoft.com/downloads/.

We use nginx and its features a lot in Scribd. Many times in the last year we needed some pretty interesting, but not supported feature – we wanted nginx X-Accel-Redirect functionality to work with remote URLs. Our of the box nginx supports this functionality for local URIs only. In this short post I want to explain how did we make nginx serve remote content via X-Accel-Redirect.

First of all, here is what you may need this feature. Let’s imagine you have a file storage on Amazon S3 where you store tons of content. And you have an application where you have some content downloading functionality that you want to be available for logged-in/paying/premium users and/or you want to keep track of downloads your users perform on your site. If your content was on your web server, you could have used simple controlled downloads functionality built-in to nginx out of the box. But the problem is that your content is remote.

Here is what we do to solve this problem.

First, we create a special location on our nginx server. This location will be used as a proxy for all our accelerated file downloads:

After adding this location to our nginx config we could start sending responses with headers like the following:

Here is an example code you could use in a Rails application to use our internal redirect location:

As you can see, nginx is really powerful tool and when you turn your creativity on you can make it even more powerful. Stay tuned for more Nginx-Fu posts.