PlanetMysql.ru – информация о СУБД MySQL

A new release of the PrimeBase Media Streaming daemon is now available for download at
http://www.blobstreaming.org .

This release doesn't contain any major new features just some bug fixes and a lot of house keeping changes.

If you look at the download section on http://www.blobstreaming.org you will see that there are now more packages that can be downloaded. I have separated out different client side components from the PBMS project and created separate launchPad projects for each one. You can see them listed in the "Related Links" side panel to the right of this post.

• The "PBMS Client Library" facilitates communication with the PBMS daemon. This library is independent of the PBMS daemon's host server and can be used to communicate with a PBMS daemon hosted by the MySQL or Drizzle database servers.
• The "PBMS PHP extension" is a PHP module that enables PHP to connect directly to the PBMS engine and stream BLOB data in and out of a MySQL or Drizzle database.
• The "Streaming enabled JDBC Driver" is a streaming enable version of the standard MySQL Connector/J, JDBC Driver.

One minor new feature that was added is that the PBMS HTTP server how understands the HTTP "range" header that can be used to request a section of BLOB data. The Client Library and PHP extension have both been updated with pbms_get_data_range() functions.


PlanetMySQL Voting: Vote UP / Vote DOWN

If you haven't already heard PBMS is now part of the Drizzle tree.

Getting it there was a fair bit of work but not as much as I had thought it would be. The process of getting it to work with Drizzle and running it thorough Hudson has improved the code a lot. It is amazing what some compilers will catch that others will let by. I am now a firm believer in treating all compiler warnings as errors.

I am just in the process of updating the PBMS plugin so that it will build and install the PBMS client library (libpbmscl.so) as well as the plugin. The PBMS client library is a standalone library that can be used to access the PBMS daemon weather it is running as part of MySQL or Drizzle. So a PBMS client library built with Drizzle can be used to access a PBMS daemon running as part of MySQL and vice-versa.

There is also PHP extension for PBMS that is basically just a wrapper for the library. Currently this is part of the PBMS project on launchpad but I am working on getting it into pecl. The PHP extension has a set of test cases with it which is what I use to test PBMS with.

If anybody is interested in taking on the task of creating a python module for PBMS I would be happy to provide what ever help you may need. I think it would just be a wrapper around the PBMS client library almost identical to the PHP extension. I would recoment just taking the PHP extension and converting it.

Now that I have PBMS in Drizzle I am planning on getting replication working with PBMS. I have decided that the best way to do this is to do a bit of work rearranging how PBMS uses the BLOB URLs to reference the BLOB data. The URLs already contain a server, database, and table id so the plan is to change things so that PBMS can handle BLOB URLS from other servers being inserted or referenced. Once this is working then I will automatically have replication and 95% of what is needed to support clustered servers.

I will go into detail on this in a later posting which will include pretty pictures.

Barry
PlanetMySQL Voting: Vote UP / Vote DOWN

Some of you may have noticed that blob streaming has been merged into the main Drizzle tree recently. There are a few hooks inside the Drizzle kernel that PBMS uses, and everything else is just in the plug in.

For those not familiar with PBMS it does two things: provide a place (not in the table) for BLOBs to be stored (locally on disk or even out to S3) and provide a HTTP interface to get and store BLOBs.

This means you can do really neat things such as have your BLOBs replicated, consistent and all those nice databasey things as well as easily access them in a scalable way (everybody knows how to cache HTTP).

This is a great addition to the AlsoSQL arsenal of Drizzle. I’m looking forward to it advancing and being adopted (now much easier that it’s in the main repository)

PlanetMySQL Voting: Vote UP / Vote DOWN

Recently when talking to someone about PBMS it occurred to me that I had been thinking about BLOBs in the traditional database sense in that they were atomic blocks of data the content of which the server knew nothing about. But with PBMS that need not be the case.

The simplest enhancement would be to allow the client to send a BLOB request to the PBMS daemon with an offset and size to just return a chunk of the BLOB. Depending on the application and the BLOB contents this may make perfectly good sense, why force the client to retrieve the entire BLOB if it only want part of it.

A much more interesting idea would be to enable the user to provide custom server side functions that they could run against the BLOB.

So how would his work?

The PBMS daemon would provide its own "BLOB functions" plugin API. The API would be quite simple where the plugin would register the function names it supports. When the PBMS daemon receives a BLOB request specifying a BLOB function name, it calls the BLOB function passing it a hook to the BLOB data and then returns to the client what ever the function returns.

The first use of this that I can imagine would be to provide a function that would return the thumbnail from a jpeg image rather than the entire image. Other functions may just return the jpeg metadata.

The idea is that BLOBs are not just blobs but are highly structured documents which, given the knowledge of the document structure, it is possible to return portions of the BLOB that are of interest to particular applications.
PlanetMySQL Voting: Vote UP / Vote DOWN

I am planning on presenting a session on PBMS at the OpenSQL camp. I am in hopes of have a chance to discuss PBMS with people and find out how they are planning on using it and what features they would like to see in it.

But even if you are not interested in PBMS you should still come if for no other reason than the free pizza!

I am proud to say the PrimeBase Technologies is one of the organizations who's sponsorship money is helping to provide the free pizza.

I will see you all there,

Barry
PlanetMySQL Voting: Vote UP / Vote DOWN

By making PBMS transactional I have been able to greatly simplify the engine interface making it much easier for engine builders to build in support for PBMS. How much simpler is it? From the time I decided to make InnoDB PBMS enabled to when I started the rebuild of MySQL was less than half an hour!

The same way that I added PBMS support to InnoDB it can be added directly to the MySQL server so that the PBMS engine will be used for BLOB storage for all engines regardless of if they have been enabled or not. PBMS support for drizzle will be provided via a data filter plug-in which I have yet to write but will soon.

To add PBMS support all you need to do is add the file pbms_enabled.cc to your source code and add the following to your handler code. I will use the InnoDB handler code as an example:


File ha_innodb.cc:

#ifdef USE_PRAGMA_IMPLEMENTATION
#pragma implementation    // gcc: Class implementation
#endif
:
:
:
/* Include necessary InnoDB headers */
extern "C" {
#include "../storage/innobase/include/univ.i"
:
:
:
#include "../storage/innobase/include/ha_prototypes.h"
}

#define PBMS_ENABLED

#ifdef PBMS_ENABLED
#include "pbms_enabled.h"
#endif

static const long AUTOINC_OLD_STYLE_LOCKING = 0;
static const long AUTOINC_NEW_STYLE_LOCKING = 1;
:
:
:
innobase_init(
/*==========*/
           /* out: 0 on success, error code on failure */
void    *p) /* in: InnoDB handlerton */
{
:
:
:
err = innobase_start_or_create_for_mysql();

if (err != DB_SUCCESS) {
   my_free(internal_innobase_data_file_path,
                    MYF(MY_ALLOW_ZERO_PTR));
   goto error;
}

#ifdef PBMS_ENABLED
   PBMSResultRec result;
   if (!pbms_initialize("InnoDB", &result)) {
       sql_print_error("pbms_initialize() Error: %s", result.mr_message);
       goto error;
   }
#endif
(void) hash_init(&innobase_open_tables,system_charset_info, 32, 0, 0,
               (hash_get_key) innobase_get_key, 0, 0);
:
:
:
error:
DBUG_RETURN(TRUE);
}
:
:
:
innobase_end(handlerton *hton, ha_panic_function type)
/*==============*/
           /* out: TRUE if error */
{
int err= 0;

DBUG_ENTER("innobase_end");

#ifdef __NETWARE    /* some special cleanup for NetWare */
if (nw_panic) {
   set_panic_flag_for_netware();
}
#endif
if (innodb_inited) {

:
:
:

#ifdef PBMS_ENABLED
       pbms_finalize();
#endif
}

DBUG_RETURN(err);
}

:
:
:
int
ha_innobase::write_row(
/*===================*/
       /* out: error code */
uchar* record) /* in: a row in MySQL format */
{
int      error = 0;
ibool    auto_inc_used= FALSE;
ulint    sql_command;
trx_t*   trx = thd_to_trx(user_thd);

DBUG_ENTER("ha_innobase::write_row");

#ifdef PBMS_ENABLED
   PBMSResultRec result;
   error = pbms_write_row_blobs(table, record, &result);
   if (error) {
       sql_print_error( "pbms_write_row_blobs() Error: %s", result.mr_message);
       DBUG_RETURN(error);
   }
#endif

:
:
:
#ifdef PBMS_ENABLED
   pbms_completed(table, (error == 0));
#endif
DBUG_RETURN(error);
}
:
:
:
ha_innobase::update_row(
/*====================*/
      /* out: error number or 0 */
const uchar*  old_row,  /* in: old row in MySQL format */
uchar*        new_row)  /* in: new row in MySQL format */
{
upd_t* uvect;
int    error = 0;
trx_t* trx = thd_to_trx(user_thd);

DBUG_ENTER("ha_innobase::update_row");

#ifdef PBMS_ENABLED
   PBMSResultRec result;

   error = pbms_delete_row_blobs(table, old_row, &result);
   if (error) {
       sql_print_error( "update_row:pbms_delete_row_blobs() Error: %s",
           result.mr_message);
       DBUG_RETURN(error);
   }
   error = pbms_write_row_blobs(table, new_row, &result);
   if (error) {
       sql_print_error( "update_row:pbms_write_row_blobs() Error: %s",
           result.mr_message);
       goto pbms_done;
   }
#endif

:
:
:
#ifdef PBMS_ENABLED
   pbms_done:
   pbms_completed(table, (error == 0));
#endif
DBUG_RETURN(error);
}
:
:
:
int
ha_innobase::delete_row(
/*====================*/
                       /* out: error number or 0 */
const uchar*    record) /* in: a row in MySQL format */
{
int    error = 0;
trx_t* trx = thd_to_trx(user_thd);

DBUG_ENTER("ha_innobase::delete_row");

#ifdef PBMS_ENABLED
   PBMSResultRec result;

   error = pbms_delete_row_blobs(table, record, &result);
   if (error) {
       sql_print_error( "pbms_delete_row_blobs() Error: %s", result.mr_message);
       DBUG_RETURN(error);
   }
#endif

:
:
:
#ifdef PBMS_ENABLED
   pbms_completed(table, (error == 0));
#endif
DBUG_RETURN(error);
}
:
:
:
int
ha_innobase::delete_table(
/*======================*/
                   /* out: error number */
const char* name)   /* in: table name */
{
:
:
:
#ifdef PBMS_ENABLED
   /* Call pbms_delete_table_with_blobs() last because it cannot be undone. */
   if (!error) {
       PBMSResultRec result;

       if (pbms_delete_table_with_blobs(name, &result)) {
           sql_print_error( "pbms_delete_table_with_blobs() Error: %s",
               result.mr_message);
       }
  
       pbms_completed(NULL, true);
   }
#endif
DBUG_RETURN(error);
}

:
:
:
int
ha_innobase::rename_table(
/*======================*/
                  /* out: 0 or error code */
const char* from,  /* in: old name of the table */
const char* to)    /* in: new name of the table */
{
ulint   name_len1;
ulint   name_len2;
int     error;
trx_t*  parent_trx;
trx_t*  trx;
char    norm_from[1000];
char    norm_to[1000];
THD*    thd = ha_thd();

DBUG_ENTER("ha_innobase::rename_table");

#ifdef PBMS_ENABLED
   PBMSResultRec result;

   error = pbms_rename_table_with_blobs(from, to, &result);
   if (error) {
       sql_print_error( "pbms_rename_table_with_blobs() Error: %s", result.mr_message);
       DBUG_RETURN(error);
   }
#endif
:
:
:
#ifdef PBMS_ENABLED
   pbms_completed(NULL, (error == 0));
#endif
DBUG_RETURN(error);
}

Now you are probably wondering how I spent half an hour just adding that.

The PBMS engine now has built in support for transaction so that if you reference or dereference BLOBs during a transaction the changes you made will be committed or rolled back with the transaction. Up until now PBMS had depended on the host engine to handle transactions and inform it what needed to be done in the event of a rollback.

I have implemented transaction support by adding a circular transaction log and transaction cache. The circular transaction log is dynamic and can grow and shrink as required. The transaction records are all small (30 byte) fixed length records so the log doesn’t need to be that large to be able to handle a lot of transactions. A head and tail pointer are maintained to indicate where the head and tail of the circular log is. If the circular log becomes full then the records just overflow and are written to the end of the log. Once the transaction reader has caught up after an overflow has occurred it simply resizes the circular log to include the overflow and everything continues as normal with a larger circular log. The circular log will be reduced to its normal size once free space becomes available at the end of the log.

If anyone has any use for something like this them selves it has been written as a standalone module so it would not be to difficult to pull it out and build it into something else.