My personal feeling is to have no automatic reconnect within Connector/Python and the programmer has to come up with retrying transactions herself.

For example:

	cnx.disconnect() # For testing..
	tries = 2
	while tries > 0:
		tries -= 1
		try:
			cursor.execute("INSERT INTO t1 (c1) VALUES ('ham')")
			cnx.commit()
		except mysql.connector.InterfaceError:
			if tries == 0:
				print "Failed inserting data after retrying"
				break
			else:
				print "Reconnecting.."
				cnx.reconnect()
		else:
			break

The above mimics how you would handle transactions and trying them reconnecting. I have ideas how to get this into Connector/Python, but it would not really fit PEP-249.

Would the above use case of reconnecting be enough?

shell> easy_install mysql-connector

Please report problems either using Launchpad or MySQL Bugs website.

As of November 2011 I’ll be working full-time on MySQL Connector/Python and other goodies within the MySQL development team at Oracle. Before, this was more or less a pet project done after working hours. However, with the birth of our son Tomas more than a year ago, I’ve been slacking and family got priority.

The idea is to make MySQL Connector/Python the best choice for connecting to MySQL from within your Python code. We still got a long road ahead of us, but I’m confident that we are on the right track.

However, since MySQL Connector/Python only needs itself and Python, dependencies are reduced to a minimum. Here you’ll find a refactored version of Haidong’s version (can of course be made much more sophisticated) using the connector:

import sys
from socket import gethostname
import smtplib
import mysql.connector

emailSubject = "Replication problem on slave %s"
emailTo = "recipient@example.com"
emailFrom = "monitor-tool@example.com"

def runCmd(cmd):
    cnx = mysql.connector.connect(user='root',
                                  unix_socket='/path/to/mysql.sock')
    cur = cnx.cursor(buffered=True)
    cur.execute(cmd)
    columns = tuple( [d[0].decode('utf8') for d in cur.description] )
    row = cur.fetchone()
    if row is None:
        raise StandardError("MySQL Server not configured as Slave")
    result = dict(zip(columns, row))
    cur.close()
    cnx.close()
    return result

try:
    slave_status = runCmd("SHOW SLAVE STATUS")
except mysql.connector.Error, e:
    print >> sys.stderr, "There was a MySQL error:", e
    sys.exit(1)
except StandardError, e:
    print >> sys.stderr, "There was an error:", e
    sys.exit(1)

if (slave_status['Slave_IO_Running'] == 'Yes' and
    slave_status['Slave_SQL_Running'] == 'Yes' and
    slave_status['Last_Errno'] == 0):
    print "Cool"
else:
    emailBody = [
        "From: %s" % emailFrom,
        "To: %s" % emailTo,
        "Subject: %s" % (emailSubject %  gethostname()),
        "",
        '\n'.join([ k + ' : ' + str(v) for k,v in slave_status.iteritems()]),
        "\r\n",
        ]
    server = smtplib.SMTP("localhost")
    server.sendmail(emailFrom, [emailTo], '\r\n'.join(emailBody))
    server.quit()

Disclaimer: Since version 0.3.1 is still a development release, or ‘alpha’, it is not
recommended to run this in production.

MySQL Connector/Python 0.3.1-devel is a maintenance release fixing following bugs:

• lp:695514 – Infinite recursion when setting connection client_flags
• lp:691836 – Incorrect substitution by cursor.execute when tuple args contains ‘%s’

About MySQL Connector/Python: MySQL Connector/Python is implementing the
MySQL Client/Server protocol completely in Python. No MySQL libraries
are needed, and no compilation is necessary to run this Python DB API v2.0
compliant driver. It is compatible with Python v2.4 and later as well as
Python v3.1 and later.

The default client flags for the MySQL Client/Server protocol can be retrieved using the constants.ClientFlag class:

>>> from mysql.connector.constants import ClientFlag
>>> defaults = ClientFlag.get_default()
>>> print ClientFlag.get_bit_info(defaults)
['SECURE_CONNECTION', 'TRANSACTIONS', 'CONNECT_WITH_DB',
 'PROTOCOL_41', 'LONG_FLAG', 'MULTI_RESULTS',
 'MULTI_STATEMENTS', 'LONG_PASSWD']

To set an extra flag when connecting to MySQL you use the client_flags argument of connect()-method. For example, you’d like to have the CLIENT_FOUND_ROWS set:

import mysql.connector
from mysql.connector.constants import ClientFlag
extra_flags = [ClientFlag.FOUND_ROWS]
cnx = mysql.connector.connect(client_flags=extra_flags)

Similar, you can unset a flag passing a list of negative values, or all at the same time. For example, you’d like the CLIENT_FOUND_ROWS set, but you don’t want CLIENT_MULTI_STATEMENTS:

import mysql.connector
from mysql.connector.constants import ClientFlag
extra_flags = [ClientFlag.FOUND_ROWS, -ClientFlag.MULTI_STATEMENTS]
cnx = mysql.connector.connect(client_flags=extra_flags)

It is also possible to pass the client_flags an integer, but you need to get first the defaults, and do bitwise operations to set/unset flags yourself. At the moment of writing, there is a bug about this, see lp:695514, but I recommend using the list-method.

Since version 0.3.0 is still a development release, or ‘alpha’, it is not
recommended to run this in production.

MySQL Connector/Pyhton 0.3.0 adds following features:

• Python v2.4 support is back.
• Support for compressed protocol.
• Support for SSL connections (when Python’s ssl module is available).
• Support for packets which are bigger than 16MB.
• Max allowed packetsize defaults to 1GB.
• Some performance improvements.

See the ChangeLog for extra details.

Please report bugs and comments using the bug tracker on Launchpad:
https://bugs.launchpad.net/myconnpy

About MySQL Connector/Python: MySQL Connector/Python is implementing the
MySQL Client/Server protocol completely in Python. No MySQL libraries
are needed, and no compilation is necessary to run this Python DB API v2.0
compliant driver. It is compatible with Python v2.4 and later as well as
Python v3.1 and later.

Note: this is a proof of concept and is only meant as a demonstration on how to extend MySQL Connector/Python.

Why query caching?

You are doing lots of queries that have the same result. It would be expensive to always run the same exact query. MySQL has already a query cache, and there is also memcached. But you like MySQL Connector/Python so much you’d like to do it yourself.

A cursor caching queries and their result

To demonstrate a simple implementation of a query cache, we inherit from an existing class: MySQLCursorBuffered. It will save the executed operation with their results in a ‘global’ variable. We call this cursor MySQLCursorQueryCache.

We take the buffered cursor because we’d like to save the result right away. Below you see we only changed two methods for MySQLCursorBuffered:

• .execute(): it will now first check using an md5 checksum whether we executed the query before. If we did, we set the make the cached result active. If not, we simply executed.
• ._handle_resultset(): called when .execute() did an operation which has a result set. The result we know save in the QUERY_CACHE global dict.

from hashlib import md5
import mysql.connector

QUERY_CACHE = dict()

class MySQLCursorQueryCache(mysql.connector.cursor.MySQLCursorBuffered):
    def execute(self, operation, params=None):
        self._qmd5 = md5(operation).digest()
        if QUERY_CACHE.has_key(self._qmd5):
            (self._rows, eof) = QUERY_CACHE[self._qmd5]
            self.rowcount = len(self._rows)
            self._handle_eof(eof)
            self._next_row = 0
        else:
            super(MySQLCursorQCache, self).execute(operation, params)

    def _handle_resultset(self):
        (self._rows, eof) = self.db().protocol.get_rows()
        self.rowcount = len(self._rows)
        self._handle_eof(eof)
        self._next_row = 0
        QUERY_CACHE[self._qmd5] = (self._rows, eof)
        try:
            self.db().unread_result = False
        except:
            pass
        self._qmd5 = None

The above code is a proof of concept, there is lots of room for improvement. For example, you need something to invalidate entries in the query cache.

How to use MySQLCursorQueryCache

def main():

    cnx = mysql.connector.connect(database='test')
    cur = cnx.cursor(cursor_class=MySQLCursorQueryCache)

    cur.execute("SELECT NOW()")
    print cur.fetchone()
    time.sleep(2)
    cur.execute("SELECT NOW()")
    print cur.fetchone()

    cur.close()
    cnx.close()

When you would use the default cursor, both executed SQL statements would produce a different result. The above produces the following output:

(datetime.datetime(2010, 11, 22, 21, 20, 4),)
(datetime.datetime(2010, 11, 22, 21, 20, 4),)

When you’d like to have some statements cached, and some not, just create a second cursor cursor_class=MySQLCursorBuffered (see Buffering results with MySQL Connector/Python).

Why buffering result sets?

Buffering or storing the result set on the client side is handy when you, for example, would like to use multiple cursors per connection and you’de like to traverse each one interleaved.

Keep in mind that with bigger result sets, the client side will use more memory. You just need to find out for yourself what’s best. When you know result sets are mostly small, you might opt to buffer.

MySQLdb by default buffers results and you need to use a different cursor to disable it. oursql does not buffer by default. This is good to know when you start using a different MySQL DB API for Python.

Use case: 1 connection, fetch from 2 cursors

You want to fetch data from two tables and process the data using one connection. If you do it without buffering, you would run into the following:

>>> cnx = mysql.connector.connect(database='test')
>>> cur1 = cnx.cursor()
>>> cur2 = cnx.cursor()
>>> cur1.execute("SELECT c1 FROM t1")
-1
>>> cur2.execute("SELECT c1 FROM t1")
..
mysql.connector.errors.InternalError: Unread result found.

MySQL Connector/Python offers two ways to turn buffering on or off. Either per connection or per cursor using the buffered argument set to True

Set buffering per connection

If you want all the cursors for a particular connection to be buffered, you can turn it on when connecting to MySQL setting the buffered-argument to True.

>>> import mysql.connector
>>> cnx = mysql.connector.connect(database='test',buffered=True)
>>> cur1 = cnx.cursor()
>>> cur1.__class__

>>> cur2 = cnx.cursor()
>>> cur1.execute("SELECT c1 FROM t1")
3
>>> cur2.execute("SELECT c1 FROM t1")
3
>>> cur1.fetchone()
(u'Geert',)
>>> cur2.fetchone()

Set buffering per cursor

You can request a buffering cursor buffering from the connection object. Do this use the buffered-argument set to True:

>>> import mysql.connector
>>> cnx = mysql.connector.connect(database='test')
>>> cur1 = cnx.cursor(buffered=True)
>>> cur1.__class__

Note: the above should work with MySQL Connector/Python v0.2 or greater