Archive for the ‘Scripting’ Category

How To Sort Columns of MySQL Data on a Web Page With Perl

Декабрь 2nd, 2011

A friend of mine was building a web site so his customers could view his current inventory of transportation containers, and he asked me for help on how to sort the rows of information that appeared on his site. So, in this post, I will give you a quick example on how to sort columns on a web page.

First, let’s start with an inventory database that we will build in MySQL:

CREATE TABLE `inventory` (
`id` int(6) NOT NULL AUTO_INCREMENT,
`item_name` varchar(30) NOT NULL,
`item_SKU` varchar(20) NOT NULL,
`item_cost` decimal(4,2) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=100000 DEFAULT CHARSET=latin1

Next, here are some SQL statements to populate the MySQL database with some sample data.

use inventory;
insert into inventory (item_name, item_SKU, item_cost) values ('Apple', '10001', '1.04');
insert into inventory (item_name, item_SKU, item_cost) values ('Peach', '10004', '1.28');
insert into inventory (item_name, item_SKU, item_cost) values ('Plum', '10301', '1.17');
insert into inventory (item_name, item_SKU, item_cost) values ('Apricot', '13033', '1.92');
insert into inventory (item_name, item_SKU, item_cost) values ('Grapes', '20422', '1.34');
insert into inventory (item_name, item_SKU, item_cost) values ('Kiwi', '98561', '2.78');
insert into inventory (item_name, item_SKU, item_cost) values ('Mango', '56231', '0.99');
insert into inventory (item_name, item_SKU, item_cost) values ('Strawberry', '24689', '1.52');
insert into inventory (item_name, item_SKU, item_cost) values ('Banana', '65213', '0.39');
insert into inventory (item_name, item_SKU, item_cost) values ('Tangerine', '47112', '1.22');

For this example, I am not going to show you how to create a new record, edit or delete your information in a MySQL database, as I did that in an earlier post.

Now that we have our data, we are going to need a Perl script that will retrieve all of the data and create our web page at the same time. I have named the Perl script inventory_view. Don’t forget to change all of the variables to match your system. I will explain the variables that we will use in the Perl Script after the script below:

#!/usr/bin/perl
#--------------------------------------------------------------------------
# inventory_view.pl
#--------------------------------------------------------------------------

use DBI;
use DBD::mysql;
use CGI qw(:standard);

my $Database = "scripting_mysql";

# get the sort parameters from the URL
my $query = new CGI;
# $sort is the order by which to sort - ascending or descending
# you only need to set a value if it is descending (desc)
$sort = $query->param("sort");
# $sortby is the field name to be used for sorting
$sortby = $query->param("sortby");

# check for sort order - ascending or descending
if ($sort =~ "asc")

{
$order_by = "order by $sortby";
}

else

{
$order_by = "order by $sortby $sort";
}

# if the sort isn't set to a value, set it to a default sort of item_name
if (length($sort) < 1)

{
$order_by = "order by item_name";
}

# print HTML header
print header;

# connect to the database and pull every record
$dbh = ConnectToMySql($Database);
$query = "select item_name, item_SKU, item_cost from inventory $order_by";
$sth = $dbh->prepare($query);
$sth->execute();

# print the table header
print <<HTML;
<table border=0>
<tr>
<td style="text-align:center;border:1px solid gray;padding: 5px 10px 5px 10px;"><a href=http://192.168.1.2/cgi-bin/scripting_mysql/inventory_view.pl?sort=asc&sortby=item_name><img alt="sort ascending" title="sort ascending" height=15 src=http://tonydarnell.com/mysql_blog//arrow_blue_up.png></a>  <a href=http://192.168.1.2/cgi-bin/scripting_mysql/inventory_view.pl?sort=desc&sortby=item_name><img alt="sort descending" title="sort descending" height=15 src=http://tonydarnell.com/mysql_blog//arrow_blue_down.png></a></td>
<td style="text-align:center;border:1px solid gray;padding: 5px 10px 5px 10px;"><a href=http://192.168.1.2/cgi-bin/scripting_mysql/inventory_view.pl?sort=asc&sortby=item_SKU><img alt="sort ascending" title="sort ascending" height=15 src=http://tonydarnell.com/mysql_blog//arrow_blue_up.png></a>  <a href=http://192.168.1.2/cgi-bin/scripting_mysql/inventory_view.pl?sort=desc&sortby=item_SKU><img alt="sort descending" title="sort descending" height=15 src=http://tonydarnell.com/mysql_blog//arrow_blue_down.png></a></td>
<td style="text-align:center;border:1px solid gray;padding: 5px 10px 5px 10px;"><a href=http://192.168.1.2/cgi-bin/scripting_mysql/inventory_view.pl?sort=asc&sortby=item_cost><img alt="sort ascending" title="sort ascending" height=15 src=http://tonydarnell.com/mysql_blog//arrow_blue_up.png></a>  <a href=http://192.168.1.2/cgi-bin/scripting_mysql/inventory_view.pl?sort=desc&sortby=item_cost><img alt="sort descending" title="sort descending" height=15 src=http://tonydarnell.com/mysql_blog//arrow_blue_down.png></a></td>
</tr>

<td style="text-align:center;border:1px solid gray;padding: 5px 10px 5px 10px;font-size:12px;">Item Name</td>
<td style="text-align:center;border:1px solid gray;padding: 5px 10px 5px 10px;">Item SKU</td>
<td style="text-align:center;border:1px solid gray;padding: 5px 10px 5px 10px;">Item Price</td>

HTML

# set the second line background color
$background_color = "#FFFFFF";

# loop through the data fetched from the query
while (@data = $sth->fetchrow_array()) {

$item_name = $data[0];
$item_SKU = $data[1];
$item_cost = $data[2];

# alternate the background colors
if ($background_color =~ "#FFFFFF")

{
$background_color="#FFFFCC";
}

else

{
$background_color="#FFFFFF";
}

# print the table rows, one for each item from the database
print <<HTML;

<tr bgcolor="$background_color">
<td style="text-align:right;border:1px solid gray;padding: 5px 10px 5px 10px;font-size:12px;"</td>$item_name </td>
<td style="text-align:right;border:1px solid gray;padding: 5px 10px 5px 10px;font-size:12px;"</td>$item_SKU </td>
<td style="text-align:right;border:1px solid gray;padding: 5px 10px 5px 10px;font-size:12px;"</td>$item_cost </td>
</tr>
HTML
}

# close the table
print "</table>";

exit;

# more on what I am doing with the accessSM file may be found at:
# http://scriptingmysql.wordpress.com/2011/07/27/connecting-to-mysql-with-perl/
#----------------------------------------------------------------------
sub ConnectToMySql {
#----------------------------------------------------------------------

my ($db) = @_;

open(PW, "<..\/accessSM") || die "Can't access login credentials";
my $db= <PW>;
my $host= <PW>;
my $userid= <PW>;
my $passwd= <PW>;

chomp($db);
chomp($host);
chomp($userid);
chomp($passwd);

my $connectionInfo="dbi:mysql:$db;$host";
close(PW);

# make connection to database
my $l_dbh = DBI->connect($connectionInfo,$userid,$passwd);
return $l_dbh;

}

When you run the script for the first time in a browser, you should see something like this:

In our Perl script, we are using the $query variable to select the three columns from our MySQL database – item_name, item_SKU and item_cost. And we will decide which one to sort by with the variable $sortby. And we will assign ascending or descending order to the variable $sort. This will give us a total of six options for sorting columns and in which order:

$sortby $sort
item_name ascending order
item_name descending order
item_SKU ascending order
item_SKU descending order
item_cost ascending order
item_cost descending order

To build the URL for each, we simply use the script name – inventory_view.pl – and add the variables $sortby and $sort along with their values, using an ampersand for the delimiter.

inventory_view.pl?sortby=[column_name]&sort=[asc or desc]

Example:

inventory_view.pl?sortby=item_name&sort=asc

We will then use an image of an arrow pointing upwards (for ascending order) and downwards (for descending order). Here is an example of the HTML for “sort by item_name and ascending order”: (don’t forget to add the full path names for both the location of the Perl script and the location of your image file)

<a href="cgi-bin/inventory_view.pl?sortby=item_name&sort=asc"><img src="images/arrow_blue_up.png"></a>

You simply have to build a link for each of the six sorting options (two per database column) to correspond to each of the blue up/down arrows. All six links would look like this:

<a href="cgi-bin/inventory_view.pl?sort=asc&sortby=item_name"><img alt="sort ascending" title="sort ascending" height="15" src="images/arrow_blue_up.png"></a>
<a href="cgi-bin/inventory_view.pl?sort=desc&sortby=item_name"><img alt="sort descending" title="sort descending" height="15" src="images/arrow_blue_down.png"></a>

<a href="cgi-bin/inventory_view.pl?sort=asc&sortby=item_SKU"><img alt="sort ascending" title="sort ascending" height="15" src="images/arrow_blue_up.png"></a>
<a href="cgi-bin/inventory_view.pl?sort=desc&sortby=item_SKU"><img alt="sort descending" title="sort descending" height="15" src="images/arrow_blue_down.png"></a>

<a href="cgi-bin/inventory_view.pl?sort=asc&sortby=item_cost"><img alt="sort ascending" title="sort ascending" height="15" src="images/arrow_blue_up.png"></a>
<a href="cgi-bin/inventory_view.pl?sort=desc&sortby=item_cost"><img alt="sort descending" title="sort descending" height="15" src="images/arrow_blue_down.png"></a>

When the inventory_view.pl script is executed the first time, we have not set the $sortby column or $sort order, so it will default to sorting by item_name and ascending order (which really doesn’t have a value, as ascending order is the default sort order). Each time a sort request is made, a new connection will be made to the database, so that is something to consider when you decide which columns you want to be available to sort. And we aren’t putting a limit on the number of rows that are retrieved, so that you have multiple pages of items – but I will try to cover that in a future post.

While there are other solutions that don’t require you to hit the database each time, this should give you a quick (and dirty) way to sort columns of information in a table on a web page.

 

—————————————–

Tony Darnell is a Principal Sales Consultant for MySQL, a division of Oracle, Inc. MySQL is the world’s most popular open-source database program.

Tony may be reached at info [at] ScriptingMySQL.com and on LinkedIn.


PlanetMySQL Voting: Vote UP / Vote DOWN

Automatically Download MySQL Enterprise Monitor Graphs as PNG Files Using Perl

Ноябрь 18th, 2011

I was giving a presentation of the MySQL’s Enterprise Monitor* application to a client recently. I was demonstrating the “graphs” section of MEM, where you can monitor MySQL sessions, connections, replication latency and more with 60+ graphs. Usually, you view the graphs from within the MEM Enterprise Dashboard (via a web browser). But the client asked if there was a way to automatically download graphs. I wasn’t sure why he wanted to download the graphs (I didn’t ask), but I knew it wasn’t possible by using MEM alone. However, in the past I have written Perl scripts to automatically download files from web sites, so I thought I would see if it was possible with MEM.

 
*The MySQL Enterprise Monitor (MEM) continuously monitors your MySQL servers and alerts you to potential problems before they impact your system. Its like having a “Virtual DBA Assistant” at your side to recommend best practices to eliminate security vulnerabilities, improve replication, optimize performance and more. As a result, the productivity of your developers, DBAs and System Administrators is improved significantly. (from: http://www.mysql.com/products/enterprise/monitor.html)
 
 

Of course, you have to install MEM and at least one agent. Let’s assume that you have already accomplished this task, and that MEM is running properly. Open MEM in your browser, login, click on the graphs tab, and then you will see a list of all of the available graphs.

For this example, we are going to automatically download the Agent Reporting Delay and the Disk IO Usage graphs. We will download the first graph for all of the servers in a particular group, and the second graph for an individual server. First, click on a server group in your server list on the left side of MEM.

Next, we will need to change the Time Range settings to “From/To”, so that we can enter the a timeline for the graph in our script. Don’t worry about the time settings that are in MEM, as we will change these settings later, but we need them so that they will be included in the URL that we will use (more on this later). After you have changed the Time Range settings, click on the “Filter” button.

Next, click on the plus sign for the graph that you want to use so that MEM will draw the graph. For this example, we will click on the “Agent Reporting Delay” graph:

You will notice two icons to the right of the graph name. The first icon (on the left) allows you click on the icon to download the graph as a .csv file. The second icon (on the right) allows you to click on the icon and download the graph as a PNG image file.

We need some information from the actual link that is used when you click on the PNG icon. So, we will need to right-click on the icon to get the URL link location information for the Agent Reporting Delay graph:

The URL for this graph is then copied to your clipboard. This is the URL location (which is for all servers in the group that I selected):

http://192.168.1.2:18080/Graph.action?dims_height=300&dims_width=800&graph=f924cb42-fed5-11df-923c-a6466b4620ce&locale=en_US&noDefaults=false&servers_group=0&style=NORMAL&time_fromDate=2011-11-16&time_fromTime=11%3A24&time_toDate=2011-11-16&time_toTime=11%3A54&time_type=FROMTO&tzName=America%2FNew_York

As you can see in the URL above, there are several variable values that we will include in our script to produce our graphs (in blue text above). In this example, we will only be working with the following variables:
- dims_height
- dims_width
- time_fromDate
- time_fromTime
- time_toDate
- time_toTime
- servers_group and servers_server
(the servers_server variable and value are not shown in the above example, but will be in the next example below)

We will be using a text file named files.txt to store some of the graph variable values that will be used by the script. Now that you know how to copy the URL for a graph, you will need to extract the value for the graph variable and the value for the servers variable and place the values into your files.txt file. The graph value for the above URL (shown again below) is in blue text, and the value for the server variable is in red text: (notice that all values are separated on the left by an equal sign “=” and on the right by an ampersand “&”)

http://192.168.1.2:18080/Graph.action?dims_height=300&dims_width=800&graph=f924cb42-fed5-11df-923c-a6466b4620ce&locale=en_US&noDefaults=false&servers_group=0&style=NORMAL&time_fromDate=2011-11-16&time_fromTime=11%3A24&time_toDate=2011-11-16&time_toTime=11%3A54&time_type=FROMTO&tzName=America%2FNew_York

In the above example, we had selected a group of servers in our server list (on the left side of MEM), and therefore the URL will not have a value for the individual server (variable named servers_server). The graph that we will extract will be for this group of servers (in this case servers_group has a value of zero, which is still a value). This is what we had chosen under our Servers list:

Now, we want to select an individual server. In this case, we will click on “iMac-Tony”:

Now that we have chosen an individual server, in the URL for that graph, you will have a value for the variable named “servers_server”, as well as a value for servers_group – and you will need both values together. So, if you want a graph for an individual server, you will need to click on that individual server in your servers list, reselect the “Time Range” value of “From/To”, click “Filter”, and re-copy the PNG graph URL. Once we have copied the URL for this graph for an individual server, you will see a different value for the graph variable (in red) and a value for servers_group and servers_server (in blue) like this:

http://192.168.1.2:18080/Graph.action?dims_height=300&dims_width=800&graph=6d9c8ac0-7a3b-11df-9df0-f30c5eb77a3c&locale=en_US&noDefaults=false&servers_group=0&servers_server=111&style=NORMAL&time_fromDate=2011-11-16&time_fromTime=15%3A27&time_toDate=2011-11-16&time_toTime=15%3A57&time_type=FROMTO&tzName=America%2FNew_York

We will use the above URL information for our second graph – the Disk IO Usage graph. You will need to copy all of the graph and server values for the graphs that you want to download. For the above URL, we will grab these values, to be placed in our files.txt file:
graph = 6d9c8ac0-7a3b-11df-9df0-f30c5eb77a3c
server group and server name = servers_group=0&servers_server=111

This is a tedious process at first, but you should only have to do this once for each graph. (Caveat – I am not sure if the graph values change over time or when you upgrade the MEM software.)

In the files.txt file, we also want a name for the graph (which will also be used for the PNG image file name), the graph value, the servers value and the server or server group values from the above URLs, as well as the name of the server group or individual server. You should separate the values with a delimiter of three tildes “~~~”.

So, for the two example graphs above, your files.txt file should contain the following values – Graph Name~~~graph value~~~server information~~~server or group name: (please note that the graph values that I have here may not be the same values that you would have for the same graph)

Agent Reporting Delay~~~f924cb42-fed5-11df-923c-a6466b4620ce~~~servers_group=0~~~All Servers
Disk IO Usage~~~6d9c8ac0-7a3b-11df-9df0-f30c5eb77a3c~~~servers_group=0&servers_server=111~~~iMac Tony

The first line above will produce an “Agent Report Delay” graph for “All Servers”. The second line will produce a “Disk IO Usage” graph for only the server named “iMac-Tony”.

Now that we have our files.txt file in place (it should be placed in the same folder as the Perl script – or you may modify the Perl script for a different file location), we will use this Perl script to download our graphs as PNG image files. In case you want to place this script in a cron job to run every X number of minutes, we will include a variable to allow you to select the previous number of minutes to include in your graph. For example, in the Perl script, if you set the value of the variable $time_interval to 60 (minutes) and run the job at 30 minutes past the hour, the script will retrieve a graph for the past 60 minutes from the time of script execution.

For this example, we will name the Perl script “get_graphs.pl”. There will be some variables in the script that you will have to change once, to match your system’s information. The variables that you need to change are highlighted in blue text in the script:

#!/usr/bin/perl

use WWW::Mechanize;
use Date::Calc qw(Add_Delta_DHMS);

# file name for input - this contains the Graph Name and Graph URL
$filename = "files.txt";

# time interval must be in minutes
$time_interval = '60';

# the width of your graph
$dims_width = "800";
# the height of your graph
$dims_height = "300";

# IP and port number of your MEM server
$server = "192.168.1.2:18080";

# get the current time using the display_time_now subroutine
$unixtimenow = &display_time_now();
($time_toDate, $time_toTime) = split(" ",$unixtimenow);

# get the past time using the display_time_past subroutine
$unixtimepast = &display_time_past();
($time_fromDate, $time_fromTime) = split(" ",$unixtimepast);

# fool the web server into thinking we are a person
my $mech = WWW::Mechanize->new();
# look like a real person
$mech->agent('User-Agent=Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5');
# we need cookies
$mech->cookie_jar(HTTP::Cookies->new);

# Login to the MySQL Enterprise Monitor
$mech->get('http://192.168.1.2:18080/Auth.action');
$mech->success or die "login GET fail";

# you will need to substitute your user name and password for MEM here
my $user = 'tonydarnell';
my $pass = 'tonyd999';

# find a fill out the login form
my $login = $mech->form_name("DoAuth");
$login->value('username' => $user);
$login->value('password' => $pass);
$mech->submit();
$mech->success or die "login POST fail";

open(line, "$filename") || die (print "\nERROR - could not open file: $filename\n");
while (<line>)

{

chomp $_;

print "\n$_\n";

($imagefilename, $graph_to_get, $servers_to_get, $servers_name) = split(/~~~/);

if (length($imagefilename) > 2)

{

$time_toDate_for_filename = $time_toDate;
$time_toDate_for_filename =~ s/\-/_/g;

$time_toTime_for_filename = $time_toTime;
$time_toTime_for_filename =~ s/\:/_/g;

$servers_name =~ s/ /_/g;

$imagefilename =~ s/ /_/g;
$imagefilename = $servers_name . "_" . $imagefilename . "_" . $time_toDate_for_filename . "_" . $time_toTime_for_filename . ".png";

# you will need to change your settings here to match your URL for your graphs
$graph = "http://". $server . "/Graph.action?dims_height=" . $dims_height . "&dims_width=" . $dims_width . "&graph=" . $graph_to_get . "&locale=en_US&noDefaults=false&" . $servers_to_get . "&style=NORMAL&time_fromDate=" . $time_fromDate . "&time_fromTime=" . $time_fromTime . "&time_toDate=" . $time_toDate . "&time_toTime=" . $time_toTime . "&time_type=FROMTO&tzName=America%2FNew_York";

print "\n$graph\n";

#exit;

# Get the PNG image file from the URL
$mech->get($graph);
$mech->save_content($imagefilename);

}

}

exit;

close($filename);

# ------------------------------------------------
# sub-routines

sub display_time_now {
my ($sec,$min,$hour,$mday,$mon,$year,undef,undef,undef) = localtime time();
$year += 1900;
$mon += 1;
return "$year-".sprintf("%02d-%02d %02d:%02d",$mon,$mday,$hour,$min);
}

sub display_time_past {
my ($sec,$min,$hour,$mday,$mon,$year,undef,undef,undef) = localtime time() - ($time_interval*60);
$year += 1900;
$mon += 1;
return "$year-".sprintf("%02d-%02d %02d:%02d",$mon,$mday,$hour,$min);
}

# ------------------------------------------------

When we executed the script, two files were created and downloaded – All_Servers_Agent_Reporting_Delay_2011_11_16_16_17.png and iMac_Tony_Disk_IO_Usage_2011_11_16_16_17.png:


All_Servers_Agent_Reporting_Delay_2011_11_16_16_17.png


iMac_Tony_Disk_IO_Usage_2011_11_16_16_17.png

You could also create a similar script to download the information as a .csv file, but the syntax is very different (maybe I will do that in a future post). But for now, I have a possible solution for the client – and I hope that he likes it.

 

—————————————–

Tony Darnell is a Principal Sales Consultant for MySQL, a division of Oracle, Inc. MySQL is the world’s most popular open-source database program.

Tony may be reached at info [at] ScriptingMySQL.com and on LinkedIn.


PlanetMySQL Voting: Vote UP / Vote DOWN

Using MySQL and Perl to Create, Edit and Delete Information Via a Web Page

Ноябрь 11th, 2011

A friend of mine was asking me for my recommendation of a good desktop database program to use to keep track of his inventory of cargo containers. I suggested to him that he should use MySQL and write a web page interface to do everything that he needed. He then reminded me that he is a lawyer by trade, and that he doesn’t have any computer programming experience. Then I remembered that he has almost zero computer skills. And his Texas Hold-Em skills are even worse, but I don’t mind taking his money. In his case, he should just use a notepad and a pencil. (As for the question – what is a lawyer doing with cargo containers? – that is a different story.)

If he did decide to broaden his horizons a bit, he could easily write his own software web application for creating and storing almost any kind of data. In this post, I will show you how to create a MySQL database and then the web pages needed to create new addresses, edit the same data and delete the data as well.

Of course, you will need to download and install MySQL. There are a ton of resources on the web for doing this, so let’s assume that you have already this part completed – and that you know how to use MySQL. First, let’s create a MySQL table. In this example, we will create a simple address book, and populate it with a few fake names. Here is the SQL, complete with the fake data to be inserted:

SET NAMES latin1;
SET FOREIGN_KEY_CHECKS = 0;

CREATE TABLE `address` (
`serial` int(4) NOT NULL AUTO_INCREMENT,
`name_first` varchar(30) NOT NULL,
`name_last` varchar(30) NOT NULL,
`address_01` varchar(40) NOT NULL,
`address_02` varchar(40) NOT NULL,
`address_city` varchar(30) NOT NULL,
`address_state` varchar(2) NOT NULL,
`address_postal_code` varchar(10) NOT NULL,
PRIMARY KEY (`serial`)
) ENGINE=InnoDB AUTO_INCREMENT=7 DEFAULT CHARSET=latin1;

insert into `address` values('1','1','Clark','Kent','344 Clinton St','Apt. #3B','Metropolis','NY','10001'),
('2','2','Dave','Jones','500 Second Avenue','Suite 100','Atlanta','GA','30303'),
('3','3','Tom','Watson','123 Golf Course Lane','Suite A','Macon','GA','31066'),
('4','4','Jack','Nicklaus','400 Laurel Oak Dr','Suite 49','Suwanee','GA','31044'),
('5','1','Betty','Smith','100 Main Street','Suite 500','Buffalo','NY','14201'),
('6','2','Bruce','Wayne','1007 Mountain Drive','','Gotham City','NY','10000');

SET FOREIGN_KEY_CHECKS = 1;

We will be creating several Perl scripts as follows. To keep it simple, we will use Perl to create all of the pages, even though the “add_entry.pl” script could be a standard HTML page. This will allow you to put all of the scripts in your cgi-bin directory. Here are the scripts that we will be creating and using:
  — dashboard.pl -> This is your “home page”, and will list all of the entries in your address database.
  — add_entry.pl -> This page will allow you to complete a form containing the new address information.
  — add.pl -> This Perl script will save the information from the add.html web page.
  — edit.pl -> This Perl script will allow you to edit an entry from your address database.
  — edit_save.pl -> This will save the changes made from the edit.pl page.
  — delete.pl -> This is step one in deleting a record, it takes you to the delete_confirm.pl page.
  — delete_confirm.pl -> This will confirm the deletion of a record, or allow you to cancel the request.

For our home page (called dashboard), we need to create a web page (via a Perl script) that will list all of the addresses in the database. In this example, we aren’t using any logic to restrict the number of entries that are displayed, so this page will just display all of them (adding limits to a web page like this is a bit more complicated, and I might try to cover this in a future post). We will use a Perl script to display the HTML, as we will need to pull data from the database to be displayed in the web page. So, you will need to put the “directory.pl” script (home page) and all of the other scripts in your cgi-bin directory (and don’t forget to make all of your Perl scripts executable – via “chmod 755″). You could create a regular HTML page (.htm) using frames, with the top frame using HTML and then use a Perl script to pull the data for the bottom frame, but let’s just stick with this example for now.

Here is what the opening home page (dashboard.pl) should look like:

In this script, we will also create a link that will allow us to edit the information for a single address entry, as well as a link to delete the entry. We will use the serial database field to point to the record that we want to edit or delete. We will also display a link at the top for creating a new address entry, and this Perl script will be named “add.pl”. But first, here is the “dashboard.pl” script:

#!/usr/bin/perl

#--------------------------------------------------------------------------
# dashboard.pl
#--------------------------------------------------------------------------

# DBI is the standard database interface for Perl
# DBD is the Perl module that we use to connect to the <a href=http://mysql.com/>MySQL</a> database
use DBI;
use DBD::mysql;
use CGI qw(:standard);

my $Database = "scripting_mysql";

print header;
print <<HTML;
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>Scripting MySQL - Address Book - Dashboard</title>
</head>

<body>
<center>

HTML
# leave the above line to the left of the page

# list my addresses

$dbh = ConnectToMySql($Database);

$query = "select serial, name_first, name_last, address_01, address_02, address_city, address_state, address_postal_code FROM address order by serial";

$sth = $dbh->prepare($query);

$sth->execute();

$dbh->disconnect;

# list all of the addresses in a table
# be sure that the link for add.html is the correct path according to your system
print "<center><font color=blue>My Addresses - <a href=http://192.168.1.2/cgi-bin/scripting_mysql/add_entry.pl>Create New</a></font><p>";
print "<table border=0>";
# print table row headers
print "<tr><td>ID</td><td>First</td><td>Last</td><td>Address</td><td>Address</td><td>City</td><td>ST</td><td>ZIP</td></tr>";

# set your initial row background color
$bgcolor = "#EEEEEE";

while (@data = $sth->fetchrow_array()) {
$serial = $data[0];
$name_first = $data[1];
$name_last = $data[2];
$address_01 = $data[3];
$address_02 = $data[4];
$address_city = $data[5];
$address_state = $data[6];
$address_postal_code = $data[7];

# print rows of data
# for the $serial, we include the link to the edit.pl script - edit.pl?$serial
print "<tr bgcolor=$bgcolor><td><a href=edit.pl?id=$serial>$serial</a></td><td>$name_first</td><td>$name_last</td><td>$address_01</td><td>$address_02</td><td>$address_city</td><td>$address_state</td><td>$address_postal_code</td></tr>";

# alternate background colors
if ($bgcolor =~ "#EEEEEE") { $bgcolor = "white"; }

else

{$bgcolor = "#EEEEEE";}

# end while (@data = $sth->fetchrow_array())
}
# print bottom of page
print <<HTML;
<table>
</body>
</html>

HTML
# leave the above line to the left of the page

exit;

# more on what I am doing with the accessSM file may be found at:
# http://scriptingmysql.wordpress.com/2011/07/27/connecting-to-mysql-with-perl/

#----------------------------------------------------------------------
sub ConnectToMySql {
#----------------------------------------------------------------------

my ($db) = @_;

open(PW, "<..\/accessSM") || die "Can't access login credentials";
my $db= <PW>;
my $host= <PW>;
my $userid= <PW>;
my $passwd= <PW>;

chomp($db);
chomp($host);
chomp($userid);
chomp($passwd);

# or, you may comment out the above nine lines - starting with open(PW...
# and just put in your database connection information below
# and un-comment these lines
# but I don't like putting my connection information in a Perl script
# my $db = "database_name";
# my $host = "host_name";
# my $userid = "user_name";
# my $passwd = "password";

my $connectionInfo="dbi:mysql:$db;$host";
close(PW);

# make connection to database
my $l_dbh = DBI->connect($connectionInfo,$userid,$passwd);
return $l_dbh;
}

The add_entry.pl web page will look like this (with the data for a new entry already entered):

And here is the script to create the add_entry.pl web page:

#!/usr/bin/perl

#--------------------------------------------------------------------------
# add_entry.pl
#--------------------------------------------------------------------------

use CGI qw(:standard);

print header;

print <<HTML;

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>Scripting MySQL - Address Book - Add Entry</title>
</head>

<body>
<center><font color=blue>My Addresses - <a href=http://192.168.1.2/cgi-bin/scripting_mysql/dashboard.pl>Dashboard</a></font><p>
<table>
<form method="post" name="address" action="http://192.168.1.2/cgi-bin/scripting_mysql/add.pl">

<table>

<tr><td>Name First</td><td><INPUT TYPE=text NAME="name_first" id=name_first size=30></td></tr>
<tr><td>Name Last</td><td><INPUT TYPE=text NAME="name_last" id=name_last size=30></td></tr>
<tr><td>Address 1</td><td><INPUT TYPE=text NAME="address_01" id=address_01 size=40></td></tr>
<tr><td>Address 2</td><td><INPUT TYPE=text NAME="address_02" id=address_02 size=40></td></tr>
<tr><td>City</td><td><INPUT TYPE=text NAME="address_city" id=address_city size=30></td></tr>
<tr><td>State</td><td><INPUT TYPE=text NAME="address_state" id=address_state size=2></td></tr>
<tr><td>Zip</td><td><INPUT TYPE=text NAME="address_postal_code" id=address_postal_code size=10></td></tr>

<tr><td colspan=2><center><input type="submit" value="Add Address" alt="Add Address"></td></tr>
</form>

</body>
</html>

HTML

exit;

The form on this web page calls a Perl script named “add.pl”, which will insert the information to the MySQL database.

Here is the Perl script add.pl:

#!/usr/bin/perl

#--------------------------------------------------------------------------
# add.pl
#--------------------------------------------------------------------------

use DBI;
use DBD::mysql;
use CGI qw(:standard);

read(STDIN, $buffer, $ENV{'CONTENT_LENGTH'});

$buffer =~ tr/+/ /;
$buffer =~ s/\r/ /g;
$buffer =~ s/'/ /g;
$buffer =~ s/\n/ /g;
$buffer =~ s/%([a-fA-F0-9][a-fA-F0-9])/pack("C",hex($1))/eg;
$buffer =~ s/<!--(.|\n)*-->/ /g;
$buffer =~ tr/\\|[|]|<|!|"|$|{|}|*|#|'|>|||;|%/ /;

@pairs = split(/&/,$buffer);
foreach $pair(@pairs){
($key,$value)=split(/=/,$pair);
$formdata{$key}.="$value";
}

# here are the values from the HTML form
$name_first = $formdata{'name_first'};
$name_last = $formdata{'name_last'};
$address_01 = $formdata{'address_01'};
$address_02 = $formdata{'address_02'};
$address_city = $formdata{'address_city'};
$address_state = $formdata{'address_state'};
$address_postal_code = $formdata{'address_postal_code'};

my $Database = "scripting_mysql";

print header;
print <<HTML;
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>Scripting MySQL - Address Book - New</title>
</head>

<body>
<center>

HTML
# leave the above line to the left of the page

# list my addresses

$dbh = ConnectToMySql($Database);

$query = "insert into address (name_first, name_last, address_01, address_02, address_city, address_state, address_postal_code) values (?,?,?,?,?,?,?)";

$sth = $dbh->prepare($query);

$sth->execute("$name_first", "$name_last", "$address_01", "$address_02", "$address_city", "$address_state", "$address_postal_code");

$dbh->disconnect;

print "$name_first $name_last - was added to the database.<p>";

print "Return to the <a href=http://192.168.1.2/cgi-bin/scripting_mysql/dashboard.pl>Dashboard</a>";

# print bottom of page
print <<HTML;
<table>
</body>
</html>

HTML
# leave the above line to the left of the page

exit;

# more on what I am doing with the accessSM file may be found at:
# http://scriptingmysql.wordpress.com/2011/07/27/connecting-to-mysql-with-perl/

#----------------------------------------------------------------------
sub ConnectToMySql {
#----------------------------------------------------------------------

my ($db) = @_;

open(PW, "<..\/accessSM") || die "Can't access login credentials";
my $db= <PW>;
my $host= <PW>;
my $userid= <PW>;
my $passwd= <PW>;

chomp($db);
chomp($host);
chomp($userid);
chomp($passwd);

# or, you may comment out the above nine lines - starting with open(PW...
# and just put in your database connection information below
# and un-comment these lines
# but I don't like putting my connection information in a Perl script
# my $db = "database_name";
# my $host = "host_name";
# my $userid = "user_name";
# my $passwd = "password";

my $connectionInfo="dbi:mysql:$db;$host";
close(PW);

# make connection to database
my $l_dbh = DBI->connect($connectionInfo,$userid,$passwd);
return $l_dbh;
}

After you add an entry to the database, this is the confirmation web page after a record has been added.

The add.pl page gives you a link to go back to the dashboard, which will then display all of your address database entries, including the one you just entered.

To edit an entry, from the dashboard web page, you simply click on the ID of the entry that you want to modify. For example, when you click on the first entry with a serial number of “1″, you will see this link:

http://192.168.1.2/cgi-bin/scripting_mysql/edit.pl?id=1

Clicking on an serial number link will call the Perl script named “edit.pl”. This will create a web page where you can edit the information and save it back to the database. The edit.pl web page looks like this:

Here is the edit.pl script:

#!/usr/bin/perl

#--------------------------------------------------------------------------
# edit.pl
#--------------------------------------------------------------------------

use DBI;
use DBD::mysql;
use CGI qw(:standard);

$query = new CGI;

$id = $query->param("id");

print header;
print <<HTML;
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>Scripting MySQL - Address Book - Edit</title>
</head>

<body>
HTML

# grab the information for this id

$dbh = ConnectToMySql($Database);

$query = "select serial, name_first, name_last, address_01, address_02, address_city, address_state, address_postal_code FROM address where serial = '$id'";

$sth = $dbh->prepare($query);

$sth->execute();

$dbh->disconnect;

# list all of the addresses in a table
# be sure that the link for add.html is the correct path according to your system
print "<center><font color=blue>My Addresses - <a href=http://192.168.1.2/cgi-bin/scripting_mysql/dashboard.pl>Dashboard</a></font><p>";
print "<table border=0>";

while (@data = $sth->fetchrow_array()) {
$id = $data[0];
$name_first = $data[1];
$name_last = $data[2];
$address_01 = $data[3];
$address_02 = $data[4];
$address_city = $data[5];
$address_state = $data[6];
$address_postal_code = $data[7];
}

print <<HTML;

<form method="post" name="address" action="http://192.168.1.2/cgi-bin/scripting_mysql/edit_save.pl">
<input type=hidden name=id value="$id">
<tr><td align=right>Name First  </td><td><INPUT TYPE=text NAME="name_first" id=name_first size=30 value="$name_first"></td></tr>
<tr><td align=right>Name Last  </td><td><INPUT TYPE=text NAME="name_last" id=name_last size=30 value="$name_last"></td"></tr>
<tr><td align=right>Address 1  </td><td><INPUT TYPE=text NAME="address_01" id=address_01 size=40 value="$address_01"></td></tr>
<tr><td align=right>Address 2  </td><td><INPUT TYPE=text NAME="address_02" id=address_02 size=40 value="$address_02"></td></tr>
<tr><td align=right>City  </td><td><INPUT TYPE=text NAME="address_city" id=address_city size=30 value="$address_city"></td></tr>
<tr><td align=right>State  </td><td><INPUT TYPE=text NAME="address_state" id=address_state size=2 value="$address_state"></td></tr>
<tr><td align=right>Zip  </td><td><INPUT TYPE=text NAME="address_postal_code" id=address_postal_code size=10 value="$address_postal_code"</td></tr>
<tr><td colspan=2><center><input type="submit" value="Save Changes" alt="Save Changes"></td></tr>
</form>
</table>

</body>
</html>

HTML

exit;

# more on what I am doing with the accessSM file may be found at:
# http://scriptingmysql.wordpress.com/2011/07/27/connecting-to-mysql-with-perl/

#----------------------------------------------------------------------
sub ConnectToMySql {
#----------------------------------------------------------------------

my ($db) = @_;

open(PW, "<..\/accessSM") || die "Can't access login credentials";
my $db= <PW>;
my $host= <PW>;
my $userid= <PW>;
my $passwd= <PW>;

chomp($db);
chomp($host);
chomp($userid);
chomp($passwd);

# or, you may comment out the above nine lines - starting with open(PW...
# and just put in your database connection information below
# and un-comment these lines
# but I don't like putting my connection information in a Perl script
# my $db = "database_name";
# my $host = "host_name";
# my $userid = "user_name";
# my $passwd = "password";

my $connectionInfo="dbi:mysql:$db;$host";
close(PW);

# make connection to database
my $l_dbh = DBI->connect($connectionInfo,$userid,$passwd);
return $l_dbh;
}

The edit.pl script will create the web page for you to make your changes. After you have made the changes, the form will call another Perl script named “edit_save.pl” to apply those changes to the database. Here is the edit_save.pl script:

#!/usr/bin/perl

#--------------------------------------------------------------------------
# edit_save.pl
#--------------------------------------------------------------------------

use DBI;
use DBD::mysql;
use CGI qw(:standard);
use LWP::UserAgent;

my $Database = "address";

read(STDIN, $buffer, $ENV{'CONTENT_LENGTH'});

$buffer =~ tr/+/ /;
$buffer =~ s/\r/ /g;
$buffer =~ s/'/ /g;
$buffer =~ s/\n/ /g;
$buffer =~ s/%([a-fA-F0-9][a-fA-F0-9])/pack("C",hex($1))/eg;
$buffer =~ s/<!--(.|\n)*-->/ /g;
$buffer =~ tr/\\|[|]|<|!|"|$|{|}|*|#|'|>|||;|%/ /;

@pairs = split(/&/,$buffer);
foreach $pair(@pairs){
($key,$value)=split(/=/,$pair);
$formdata{$key}.="$value";
}

$id = $formdata{'id'};
$name_first = $formdata{'name_first'};
$name_last = $formdata{'name_last'};
$address_01 = $formdata{'address_01'};
$address_02 = $formdata{'address_02'};
$address_city = $formdata{'address_city'};
$address_state = $formdata{'address_state'};
$address_postal_code = $formdata{'address_postal_code'};

$dbh = ConnectToMySql($Database);

$query = "update address set name_first = '$name_first', name_last = '$name_last', address_01 = '$address_01', address_02 = '$address_02', address_city = '$address_city', address_state = '$address_state', address_postal_code = '$address_postal_code' where serial = '$id'";

$sth = $dbh->prepare($query);

$sth->execute();

$dbh->disconnect;

print header;
print <<HTML;
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>Scripting MySQL - Address Book - Edit Saved</title>
</head>

<body>
<center>
<center><font color=blue>My Addresses - <a href=http://192.168.1.2/cgi-bin/scripting_mysql/dashboard.pl>Dashboard</a></font><p>

The following information was updated:<p>
<table>
<tr><td align=right>Name First  </td><td>$name_first</td></tr>
<tr><td align=right>Name Last  </td><td>$name_last</td"></tr>
<tr><td align=right>Address 1  </td><td>$address_01</td></tr>
<tr><td align=right>Address 2  </td><td>$address_02</td></tr>
<tr><td align=right>City  </td><td>$address_city</td></tr>
<tr><td align=right>State  </td><td>$address_state</td></tr>
<tr><td align=right>Zip  </td><td>$address_postal_code</td></tr>
</table>

HTML
# leave the above line to the left of the page

exit;

# more on what I am doing with the accessSM file may be found at:
# http://scriptingmysql.wordpress.com/2011/07/27/connecting-to-mysql-with-perl/

#----------------------------------------------------------------------
sub ConnectToMySql {
#----------------------------------------------------------------------

my ($db) = @_;

open(PW, "<..\/accessSM") || die "Can't access login credentials";
my $db= <PW>;
my $host= <PW>;
my $userid= <PW>;
my $passwd= <PW>;

chomp($db);
chomp($host);
chomp($userid);
chomp($passwd);

# or, you may comment out the above nine lines - starting with open(PW...
# and just put in your database connection information below
# and un-comment these lines
# but I don't like putting my connection information in a Perl script
# my $db = "database_name";
# my $host = "host_name";
# my $userid = "user_name";
# my $passwd = "password";

my $connectionInfo="dbi:mysql:$db;$host";
close(PW);

# make connection to database
my $l_dbh = DBI->connect($connectionInfo,$userid,$passwd);
return $l_dbh;

}

After you have edited your record, the edited results web page will look like this:

You can then click on the Dashboard link to go back to the main dashboard page.

If you want to delete a record, from the dashboard.pl page, you simply click on the red X image on the row that you want to delete. For example, when you click on the X for the serial number of “1″, you will see this link:

http://192.168.1.2/cgi-bin/scripting_mysql/delete.pl?id=1

This link takes you to the delete.pl page, where you are asked if you want to confirm the deletion, or you may cancel and go back to the dashboard. When you click on an X to delete a row, you will see a page like this:

And here is the delete.pl script:

#!/usr/bin/perl

#--------------------------------------------------------------------------
# delete.pl
#--------------------------------------------------------------------------

use DBI;
use DBD::mysql;
use CGI qw(:standard);

$query = new CGI;
$id = $query->param("id");

my $Database = "scripting_mysql";

print header;
print <<HTML;
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>Scripting MySQL - Address Book - Delete Record</title>
</head>

<body>
<center>

HTML

# grab the information for this id

$dbh = ConnectToMySql($Database);

$query = "select serial, name_first, name_last, address_01, address_02, address_city, address_state, address_postal_code FROM address where serial = '$id'";

$sth = $dbh->prepare($query);

$sth->execute();

$dbh->disconnect;

# list all of the addresses in a table
# be sure that the link for add.html is the correct path according to your system
print "<center><font color=blue>My Addresses - <a href=http://192.168.1.2/cgi-bin/scripting_mysql/dashboard.pl>Dashboard</a></font><p>";
print "Are you sure you want to delete this record?<p>";
print "<table border=0>";

while (@data = $sth->fetchrow_array()) {
$id = $data[0];
$name_first = $data[1];
$name_last = $data[2];
$address_01 = $data[3];
$address_02 = $data[4];
$address_city = $data[5];
$address_state = $data[6];
$address_postal_code = $data[7];
}

print <<HTML;

<form method="post" name="address" action="http://192.168.1.2/cgi-bin/scripting_mysql/delete_confirm.pl?id=$id">
<input type=hidden name=id value="$id">
<tr><td align=right>Name First  </td><td>$name_first</td></tr>
<tr><td align=right>Name Last  </td><td>$name_last</td"></tr>
<tr><td align=right>Address 1  </td><td>$address_01</td></tr>
<tr><td align=right>Address 2  </td><td>$address_02</td></tr>
<tr><td align=right>City  </td><td>$address_city</td></tr>
<tr><td align=right>State  </td><td>$address_state</td></tr>
<tr><td align=right>Zip  </td><td>$address_postal_code</td></tr>
<tr><td><center><input type="submit" value="Delete Address" alt="Delete Address"></form></td><td><center><form method="post" name="address" action="http://192.168.1.2/cgi-bin/scripting_mysql/dashboard.pl"><input type="submit" value="Cancel" alt="Cancel"></form></td></tr>
</table>

</body>
</html>

HTML
# leave the above line to the left of the page

exit;

# more on what I am doing with the accessSM file may be found at:
# http://scriptingmysql.wordpress.com/2011/07/27/connecting-to-mysql-with-perl/

#----------------------------------------------------------------------
sub ConnectToMySql {
#----------------------------------------------------------------------

my ($db) = @_;

open(PW, "<..\/accessSM") || die "Can't access login credentials";
my $db= <PW>;
my $host= <PW>;
my $userid= <PW>;
my $passwd= <PW>;

chomp($db);
chomp($host);
chomp($userid);
chomp($passwd);

# or, you may comment out the above nine lines - starting with open(PW...
# and just put in your database connection information below
# and un-comment these lines
# but I don't like putting my connection information in a Perl script
# my $db = "database_name";
# my $host = "host_name";
# my $userid = "user_name";
# my $passwd = "password";

my $connectionInfo="dbi:mysql:$db;$host";
close(PW);

# make connection to database
my $l_dbh = DBI->connect($connectionInfo,$userid,$passwd);
return $l_dbh;
}

If you choose to not delete the record, you are simply taken back to the Dashboard. If you choose to delete the record, then the delete-confirm.pl script is executed, and the record is deleted. Here is the confirmation web page for a deletion:

And here is the delete_confirm.pl script:

#!/usr/bin/perl

#--------------------------------------------------------------------------
# delete_confirm.pl
#--------------------------------------------------------------------------

use DBI;
use DBD::mysql;
use CGI qw(:standard);

$query = new CGI;
$id = $query->param("id");

my $Database = "scripting_mysql";

print header;
print <<HTML;
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>Scripting MySQL - Address Book - Delete Record Confirmed</title>
</head>

<body>
<center>

Are you sure you want to delete this record?

HTML

# grab the information for this id

$dbh = ConnectToMySql($Database);

$query = "select serial, name_first, name_last, address_01, address_02, address_city, address_state, address_postal_code FROM address where serial = '$id'";

$sth = $dbh->prepare($query);

$sth->execute();

$dbh->disconnect;

# list all of the addresses in a table
# be sure that the link for add.html is the correct path according to your system
print "<center><font color=blue>My Addresses - <a href=http://192.168.1.2/cgi-bin/scripting_mysql/dashboard.pl>Dashboard</a></font><p>";
print "The following information was deleted.<p>";
print "<table border=0>";

while (@data = $sth->fetchrow_array()) {
$id = $data[0];
$name_first = $data[1];
$name_last = $data[2];
$address_01 = $data[3];
$address_02 = $data[4];
$address_city = $data[5];
$address_state = $data[6];
$address_postal_code = $data[7];
}

print <<HTML;

<tr><td align=right>Name First  </td><td>$name_first</td></tr>
<tr><td align=right>Name Last  </td><td>$name_last</td"></tr>
<tr><td align=right>Address 1  </td><td>$address_01</td></tr>
<tr><td align=right>Address 2  </td><td>$address_02</td></tr>
<tr><td align=right>City  </td><td>$address_city</td></tr>
<tr><td align=right>State  </td><td>$address_state</td></tr>
<tr><td align=right>Zip  </td><td>$address_postal_code</td></tr>
</table>

</body>
</html>

HTML
# leave the above line to the left of the page

# here we delete the record
$dbh = ConnectToMySql($Database);

$query = "delete from address where serial = '$id'";

$sth = $dbh->prepare($query);

$sth->execute();

$dbh->disconnect;

exit;

# more on what I am doing with the accessSM file may be found at:
# http://scriptingmysql.wordpress.com/2011/07/27/connecting-to-mysql-with-perl/

#----------------------------------------------------------------------
sub ConnectToMySql {
#----------------------------------------------------------------------

my ($db) = @_;

open(PW, "<..\/accessSM") || die "Can't access login credentials";
my $db= <PW>;
my $host= <PW>;
my $userid= <PW>;
my $passwd= <PW>;

chomp($db);
chomp($host);
chomp($userid);
chomp($passwd);

# or, you may comment out the above nine lines - starting with open(PW...
# and just put in your database connection information below
# and un-comment these lines
# but I don't like putting my connection information in a Perl script
# my $db = "database_name";
# my $host = "host_name";
# my $userid = "user_name";
# my $passwd = "password";

my $connectionInfo="dbi:mysql:$db;$host";
close(PW);

# make connection to database
my $l_dbh = DBI->connect($connectionInfo,$userid,$passwd);
return $l_dbh;

}

One alternative to deleting the record would be to include a separate database field called “active”, set the field value to “yes” for all current records (if you have already imported records) and set the default value for this field to “yes” for any new records. Then, instead of deleting the information, you simply set “active” to “no”. So when you retrieve your list on the dashboard.pl page, you will only look for records where “active” is equal to “yes”. You will still have the “deleted” information available to you in the database. You could even duplicate the dashboard.pl script to have a “delete” page that displays the records that have been deleted.

To use the “active” option, you will need to make the following changes:

You would need to add this line in your create table SQL statement: (I would insert the line after the “serial” field in your create table statement)

`active` varchar(3) NOT NULL DEFAULT 'yes',

Then, in your dashboard.pl script, you would simply change the $query to:

$query = "select serial, name_first, name_last, address_01, address_02, address_city, address_state, address_postal_code FROM address where active = 'yes' order by serial";

For your add.pl and add_entry.pl scripts. you don’t need to do anything as the “active” field has a default value of “yes” for new records.

For your edit.pl and edit_save.pl scripts, you don’t need to do anything as you aren’t changing the “active” field.

And in your delete_confirm.pl script, instead of deleting the record, simply change the $query = “delete from address where serial = ‘$id’”; to:

$query = "update address set active = 'no' where serial = '$id'";

With these scripts, you should be able to install everything and have it running in a few minutes. And, if my lawyer friend gets tired of spending all day at the courthouse, he could always learn a new skill.

 

—————————————–

Tony Darnell is a Principal Sales Consultant for MySQL, a division of Oracle, Inc. MySQL is the world’s most popular open-source database program.

Tony may be reached at info [at] ScriptingMySQL.com and on LinkedIn.


PlanetMySQL Voting: Vote UP / Vote DOWN

Checking on the Progress of Large DML Commands in MySQL Using Perl – Part Two

Ноябрь 4th, 2011

Part Two of Two: Checking on database activity when running a large DML (Data Manipulation Language) statement – such as INSERT, DELETE, UPDATE or SELECT.

Part Two: Monitoring the activity via Perl and SHOW ENGINE INNODB STATUS. (part of the InnoDB Monitors)

In part one, I showed you how to use a Perl script to insert a million rows of dummy data into a table. I needed a large database in order to test a Perl script that I would use to monitor the activity when running a large DML statement.

The original reason for creating both of these scripts was to find a quick way to see if a large DML statement was actually being executed. A customer was performing some modifications on tables with tens of millions of rows, and they wanted to know if they were making any progress. Since the customer was using the InnoDB storage engine, I thought of a way that you could check on the progress – but only given the fact that nothing else was happening in the database (more on this reason later).

With InnoDB, you can issue the SHOW ENGINE INNODB STATUS and you will get a wealth of information. I am not going to list any of it here as it would consume too much space. However, towards the bottom of the output, you will notice under “ROW OPERATIONS” a line that contains the words “Number of rows inserted…”:

--------------
ROW OPERATIONS
--------------
0 queries inside InnoDB, 0 queries in queue
1 read views open inside InnoDB
Main thread id 2960240640, state: waiting for server activity
Number of rows inserted 1202598, updated 97249, deleted 806, read 56448551
0.00 inserts/s, 0.00 updates/s, 0.00 deletes/s, 0.00 reads/s

The above line gives you a quick snapshot of how many rows have been inserted, updated, deleted or selected (read). So, by invoking the SHOW ENGINE INNODB STATUS command, you can figure what database activity is occurring. But, if you invoke this command while the database is being used for other purposes, it will be difficult to figure out your progress on any of the four values shown.

So, all I needed to do was to write a quick Perl script to monitor the changes to that particular line of output, and then I could figure out if a certain statement was indeed being executed. My original thinking was that I needed a database with a lot of records in it. So, I decide to first create a dummy table and then shove a million records into it. But I figured out that by creating a database with a million rows, I could simply just monitor this activity versus issuing a command on the database once it had been built.

I created a quick Perl script that issues the SHOW ENGINE INNODB STATUS command every X number of seconds. It then looks for the output total for whatever I want to track – either inserts, updates, deletes or selects (reads). And yes, you could just do this manually, but if you have a command that you want to run overnight, you could execute this script and at least see what happened when you return in the morning.

One caveat – SHOW ENGINE INNODB STATUS does not correspond to a particular point in time, so it may not be consistent – as to ensure consistency would require a global lock which would consume too many resources. But it is good enough for what we want to do here.

The script will check the status, and then depending upon what variable (inserts, updates, deletes, selects) you want to search for, it will show you the total for that variable as it increases. I didn’t take time to add a lot of information about how long it would take to complete the original command, as this would be just a wild guess, given the fact that other things could be happening with the database. You will need to change the variable that you want to look for, how many rows you are expecting to change, how many seconds you want it to refresh, and your database information:

#!/usr/bin/perl -w

# DBI is the standard database interface for Perl
# DBD is the Perl module that we use to connect to the MySQL database
use DBI;
use DBD::mysql;
# use CGI for forms
#use CGI qw(:standard);

$start = '';
$begin = '';
$now = 1;

# How many rows are being changed?
# (whole numbers only)
$rows = 200000;

# how long do you want it to sleep? (seconds)
# you can modify this to be a longer time interval than 60 seconds
# which is advisable, as the SHOW ENGINE query will consume resources
$sleep_time = 60;

# what action do you want to monitor?
# based upon this output: Number of rows inserted xxxxx, updated xxxxx, deleted xxxxx, read xxxxx
# (use the word to the right of the equal sign)
# inserts = inserted
# update = updates
# deletes = deleted
# selects = read
$action = "inserted";

$Database = "scripting_mysql";

     $dbh = ConnectToMySql($Database);
     $query = "SHOW ENGINE INNODB STATUS";
     $sth = $dbh->prepare($query);
     $sth->execute();

$data = $sth->fetchrow_array();

@all_rows = split("\\n",$data);

foreach (@all_rows) {

     if ($_ =~ "Number of rows")

     {
          #print "Line $_ \n";
     
          @total = split (" ", $_);
          
          # viewing this output from SHOW ENGINE INNODB STATUS
          # Number of rows inserted 912268, updated 96931, deleted 806, read 52052215
          if ($action =~ "inserted") { $column = 4 }
          if ($action =~ "updated") { $column = 6 }
          if ($action =~ "deleted") { $column = 8 }
          if ($action =~ "read") { $column = 10 }

               if ($_ =~ "Number of rows ")
               {
                    $total[$column] =~ s/,//;
                    $begin = $total[$column];
               }               
     }
}

print "Execute your SQL statement, and then press enter/return: ";
$start = <>;

print "Beginning with $begin $action:\n";

while ($now < $rows) {

$print_date_time = &get_date_time;

     $dbh = ConnectToMySql($Database);
     $query = "SHOW ENGINE INNODB STATUS";
     $sth = $dbh->prepare($query);
     $sth->execute();

     $data = $sth->fetchrow_array();

     @all_rows = split("\n",$data);

     foreach (@all_rows) {

               @total = split (" ", $_);
          
          if ($action =~ "inserted") { $column = 4 }
          if ($action =~ "updated") { $column = 6 }
          if ($action =~ "deleted") { $column = 8 }
          if ($action =~ "read") { $column = 10 }
          
               if ($_ =~ "Number of rows ")
               {
                    $total[$column] =~ s/,//;
                    $now = $total[$column];
                    $now = $now - $begin;
                    $progress = $now / $rows;
                    $progress = $progress * 100;
                    print "$print_date_time - $progress\% changed - $now out of $rows.\n";
               }
}

# you can modify this to be a longer time interval than 60 seconds
# which is advisable, as the SHOW ENGINE query will consume resources
sleep $sleep_time;

}

     $sth->finish;
     $dbh->disconnect;

# from Connecting to MySQL with Perl
#----------------------------------------------------------------------
sub ConnectToMySql {
#----------------------------------------------------------------------

     my ($db) = @_;

     open(ACCESS_INFO, "<..\/accessBLOG") || die "Can't access login credentials";
     my $database = <ACCESS_INFO>;
     my $host = <ACCESS_INFO>;
     my $userid = <ACCESS_INFO>;
     my $passwd = <ACCESS_INFO>;

# the chomp() function will remove any newline character from the end of a string
chomp ($database, $host, $userid, $passwd);
#print "<br>$database $host $userid $passwd <br>";

     my $connectionInfo="dbi:mysql:$database;$host";
     close(ACCESS_INFO);

     # make connection to database
     my $l_dbh = DBI->connect($connectionInfo,$userid,$passwd);
     return $l_dbh;

}

# ----------------------------------------------------------------------------------
sub get_date_time {
# ----------------------------------------------------------------------------------

     my ($sec,$min,$hour,$mday,$mon,$year) = localtime time;

     $year = $year + 1900;
     $mon = $mon + 1;

     # add a zero if the value is less than 10

     if ($sec < 10) { $sec = "0$sec"; }
     if ($min < 10) { $min = "0$min"; }
     if ($hour < 10) { $hour = "0$hour"; }
     if ($mday < 10) { $mday = "0$mday"; }
     if ($mon < 10) { $mon = "0$mon"; }
     if ($year < 10) { $year = "0$year"; }

     $DateTime = "$year-$mon-$mday $hour:$min:$sec";
     return $DateTime

}
#----------------------------------------------------------------------

Here is the output from the Perl script. For this example, I was assuming that I would be inserting 200,000 rows.

sh-3.2# /usr/bin/perl monitor.pl
Execute your SQL statement, and then press enter/return:
Beginning with 1196128 inserted:
2011-11-01 17:48:06 - 0.029% changed - 58 out of 200000.
2011-11-01 17:48:16 - 0.464% changed - 928 out of 200000.
2011-11-01 17:48:26 - 0.8645% changed - 1729 out of 200000.
2011-11-01 17:48:36 - 1.259% changed - 2518 out of 200000.
2011-11-01 17:48:46 - 1.628% changed - 3256 out of 200000.
2011-11-01 17:48:56 - 2.0045% changed - 4009 out of 200000.
2011-11-01 17:49:06 - 2.3845% changed - 4769 out of 200000.
2011-11-01 17:49:16 - 2.7685% changed - 5537 out of 200000.
2011-11-01 17:49:26 - 3.1415% changed - 6283 out of 200000.

This script was just a quick hack. Another way to potentially see if your script is still running is to check out the “TRANSACTIONS” section of the SHOW ENGINE INNODB STATUS output. In this example below, you can see the insert statement that the Perl script was performing during part one (where I was inserting the 1,000,000 rows). However, since the database that I was using doesn’t have a lot of activity on it, I was able to see the insert statement multiple times when I resubmitted the SHOW ENGINE INNODB STATUS command. If you have a database with a lot of transactions running, you might not be able to see your statement in the output.

------------
TRANSACTIONS
------------
Trx id counter 1818DA1
Purge done for trx's n:o < 1817E3F undo n:o < 0
History list length 3830
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 1818DA0, not started, OS thread id 2961469440 flushing log
mysql tables in use 1, locked 1
MySQL thread id 335232, query id 2818136 192.168.1.2 WebUser query end
insert into test_large (name_first, name_last, address_01, address_02, city, state, zip) values ('1BPnJiuWsyajA4b3SH7OjS4BFJgedK','JPuJ4xh_QEbNokZZGlpcHEHKLk2W__','24IT20mW0moAwWmoYTMOwsv44yRL9mAMoo0mLyLoGoTjaW78O6','6zX2j4PLShQ_IfiOzd0LbTVi8ZaYGk3_6LIafpnFfdFL7kRMTa','mbENuc0kHQz9NLGkn5iy','xR','81941')
---TRANSACTION 0, not started, OS thread id 2960855040
MySQL thread id 326621, query id 2818137 localhost root
SHOW ENGINE INNODB STATUS
---TRANSACTION 1815A3B, not started, OS thread id 2961059840
MySQL thread id 322078, query id 2778540 192.168.1.5 WebUser
---TRANSACTION 1815A32, not started, OS thread id 2960650240
MySQL thread id 319917, query id 2778444 192.168.1.5 WebUser
---TRANSACTION 0, not started, OS thread id 2962083840
MySQL thread id 319845, query id 2771552 192.168.1.5 WebUser
---TRANSACTION 0, not started, OS thread id 2963107840
MySQL thread id 43198, query id 2817803 localhost 127.0.0.1 WebUser

I am sure that there are better ways of monitoring large DML statements, and this will only work for InnoDB tables. If you try the same command for MyISAM, SHOW ENGINE MyISAM STATUS, the output is blank:

mysql> SHOW ENGINE MyISAM STATUS;
Empty set (0.00 sec)

If you know of a better or different way to do this, please add your thoughts in the comment section below.

 

—————————————–

Tony Darnell is a Principal Sales Consultant for MySQL, a division of Oracle, Inc. MySQL is the world’s most popular open-source database program.

Tony may be reached at info [at] ScriptingMySQL.com and on LinkedIn.


PlanetMySQL Voting: Vote UP / Vote DOWN

Checking on the Progress of Large DML Commands in MySQL Using Perl – Part One

Ноябрь 2nd, 2011

Part One of Two: Checking on database activity when running a large DML (Data Manipulation Language) statement – such as INSERT, DELETE, UPDATE or SELECT.

Part One: Inserting a million rows into a database.

A friend of mine had asked a question – “Is there any way you can track how far you have advanced in a slow-moving ALTER or OPTIMIZE statement?”. A customer was performing some modifications on a database with tens of millions of rows, and they wanted to be able to see if the command was making any progress.

Since the customer was using the InnoDB storage engine, I thought of a way that you could check on the progress – but only given the fact that nothing else (major) was happening in the database (more on this reason later).

With InnoDB, you can issue the SHOW ENGINE INNODB STATUS (part of the InnoDB Monitors) and you will get a wealth of information. I am not going to list any of the output from the command here as it would consume too much space. However, towards the bottom of the output from the command, you will notice under “ROW OPERATIONS” a line that contains the words “Number of rows inserted…”:

--------------
ROW OPERATIONS
--------------
0 queries inside InnoDB, 0 queries in queue
1 read views open inside InnoDB
Main thread id 2960240640, state: waiting for server activity
Number of rows inserted 1202598, updated 97249, deleted 806, read 56448551
0.00 inserts/s, 0.00 updates/s, 0.00 deletes/s, 0.00 reads/s

The above line starting with “Number of rows…” gives you a quick snapshot of how many rows have been inserted, updated, deleted or selected (read) up to that point in time. So, by invoking the SHOW ENGINE INNODB STATUS command, you can figure what database activity has occurred since the last time you issued the command. But, if you invoke this command while the database is being used for other purposes, it will be difficult to figure out your progress on any of the four values shown, as the stats aren’t just for your activity.

So, all I needed to do was to write a quick Perl script to monitor that particular line of output, and then I could figure out if a certain statement was indeed being executed, and I could monitor the progress (assuming nothing else was happening on the database). The problem that I had was that I didn’t have a database with a lot of records in it. But, I did have a database server that didn’t have much activity. So, in part one, I will show you how to create a dummy table and then shove a million records into it. In part two (my next blog entry), I will show you the Perl script to monitor the changes to the output from the SHOW ENGINE INNODB STATUS command. (And yes, you can just run the SHOW ENGINE INNODB STATUS command manually over and over again, but what fun would that be?)

First I created a dummy table named “test_large”:

CREATE TABLE `test_large` (
`id` int(9) NOT NULL AUTO_INCREMENT,
`name_first` varchar(30) NOT NULL,
`name_last` varchar(30) NOT NULL,
`address_01` varchar(50) NOT NULL,
`address_02` varchar(50) NOT NULL,
`city` varchar(20) NOT NULL,
`state` varchar(2) NOT NULL,
`zip` varchar(5) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=latin1

Next, I created a Perl script that would create dummy/fake data and insert it into the table. This script uses a subroutine named “ConnectToMySql” that I covered in an earlier post – Connecting to MySQL with Perl. It also uses a subroutine to create fake alpha/numeric data (generate_random_string) and one that creates fake numeric data (generate_random_numbers). A third subroutine (get_date_time) is used to display the date and time (I use this subroutine when I want to insert a date and time into a MySQL database).

The script is fairly easy to use. You only need to modify the $total_rows variable to the number of rows that you want to insert, as well as the information about your particular database. I didn’t include anything in the script to show you the progress of this script, but you could certainly add that if you wish.

#!/usr/bin/perl -w

# DBI is the standard database interface for Perl
# DBD is the Perl module that we use to connect to the MySQL database
use DBI;
use DBD::mysql;

# how many total dummy rows of data do you need?
# (must be an integer)
$total_rows = 10000000;

# make sure that $total_rows is an integer
if ($total_rows =~ /\D/) {
     print "Total rows is not an integer = value = $total_rows.\n";
     exit;
}

# keep count of how many rows we have inserted
$total_completed = 0;

# your database name
$Database = "scripting_mysql";

# get the current system date and time
$print_date_time = &get_date_time;

# print the starting time (optional)
print "Starting $print_date_time\n";

# keep looping until we reach the total number of $total_rows
while ($total_completed < $total_rows) {

     # create dummy data
     $fake_data_01 = &generate_random_string(30);
     $fake_data_02 = &generate_random_string(30);
     $fake_data_03 = &generate_random_string(50);
     $fake_data_04 = &generate_random_string(50);
     $fake_data_05 = &generate_random_string(20);
     $fake_data_06 = &generate_random_string(2);
     $fake_data_07 = &generate_random_numbers(5);

     $dbh = ConnectToMySql($Database);

     $query = "insert into test_large (name_first, name_last, address_01, address_02, city, state, zip) values (?,?,?,?,?,?,?)";
     
     $sth = $dbh->prepare($query);

     $sth->execute($fake_data_01, $fake_data_02, $fake_data_03, $fake_data_04, $fake_data_05, $fake_data_06, $fake_data_07);
     $sth->finish;
          
     $total_completed++;
}

     $dbh->disconnect;

$print_date_time = &get_date_time;

# print our finishing time
print "Finishing $print_date_time\n";
exit;

# ----------------------------------------------------------------------------------
# subroutines
# ----------------------------------------------------------------------------------

# ----------------------------------------------------------------------------------
sub get_date_time {
# ----------------------------------------------------------------------------------

     my ($sec,$min,$hour,$mday,$mon,$year) = localtime time;

     $year = $year + 1900;
     $mon = $mon + 1;

     # add a zero if the value is less than 10

     if ($sec < 10) { $sec = "0$sec"; }
     if ($min < 10) { $min = "0$min"; }
     if ($hour < 10) { $hour = "0$hour"; }
     if ($mday < 10) { $mday = "0$mday"; }
     if ($mon < 10) { $mon = "0$mon"; }
     if ($year < 10) { $year = "0$year"; }

     $DateTime = "$year-$mon-$mday $hour:$min:$sec";
     return $DateTime

}
#----------------------------------------------------------------------

# from Connecting to MySQL with Perl
#----------------------------------------------------------------------
sub ConnectToMySql {
#----------------------------------------------------------------------

     my ($db) = @_;

     open(ACCESS_INFO, "<..\/accessBLOG") || die "Can't access login credentials";
     my $database = <ACCESS_INFO>;
     my $host = <ACCESS_INFO>;
     my $userid = <ACCESS_INFO>;
     my $passwd = <ACCESS_INFO>;

# the chomp() function will remove any newline character from the end of a string
chomp ($database, $host, $userid, $passwd);
#print "<br>$database $host $userid $passwd <br>";

     my $connectionInfo="dbi:mysql:$database;$host";
     close(ACCESS_INFO);

     # make connection to database
     my $l_dbh = DBI->connect($connectionInfo,$userid,$passwd);
     return $l_dbh;

}
#----------------------------------------------------------------------

#----------------------------------------------------------------------
sub generate_random_string {
#----------------------------------------------------------------------

     my $length_of_randomstring=shift;# the length of
                # the random string to generate

     my @chars=('a'..'z','A'..'Z','0'..'9','_');
     my $random_string;
     foreach (1..$length_of_randomstring)
     {
          # rand @chars will generate a random
          # number between 0 and scalar @chars
          $random_string.=$chars[rand @chars];
     }
     return $random_string;
}
#----------------------------------------------------------------------

#----------------------------------------------------------------------
sub generate_random_numbers {
#----------------------------------------------------------------------

     my $length_of_randomstring=shift;# the length of
                # the random string to generate

     my @chars=('0'..'9');
     my $random_string;
     foreach (1..$length_of_randomstring)
     {
          # rand @chars will generate a random
          # number between 0 and scalar @chars
          $random_string.=$chars[rand @chars];
     }
     return $random_string;
}
#----------------------------------------------------------------------

This script took about three hours to insert a million rows into the new database. And since this was a new table, every so often I would just check to see how many rows were in the table with a simple select command:

mysql> select count(*) from test_large;
+----------+
| count(*) |
+----------+
| 98187 |
+----------+
1 row in set (0.00 sec)

In my next post, I will show you the Perl script that I used to monitor the SHOW ENGINE INNODB STATUS results.

 

—————————————–

Tony Darnell is a Principal Sales Consultant for MySQL, a division of Oracle, Inc. MySQL is the world’s most popular open-source database program.

Tony may be reached at info [at] ScriptingMySQL.com and on LinkedIn.


PlanetMySQL Voting: Vote UP / Vote DOWN

Using MySQL, Perl and jQuery to Auto-Populate a Form Field on a Web Page

Октябрь 28th, 2011

If you have ever built a form on a web page, you might have used a drop-down menu to display the choices available for a particular field. With a drop-down menu, you restrict the choices a user may select so that the user doesn’t enter invalid data (among other reasons). If a user misspells an entry, then a subsequent search for that value would not produce a found result.

A friend of mine who runs an online forum sent me an email about a problem he was having. He was trying to modify an existing registration web page using jQuery to auto-populate the state names, and then pass the state abbreviation back to his MySQL database. Believe it or not, he was actually having problems with people knowing their own state abbreviation. He had searched and found an example of what he wanted to do, but he couldn’t get it to work. So, I took the example that he found and figured out what he was doing wrong.

I had first suggested that he just hard-code the list of states and their abbreviations in the HTML code. But, he said that if he figured out how to use jQuery with his state abbreviation problem, he would also use it on other parts of his web page where he had dynamic data. With drop-down menus, it is fairly easy to create a list of static data to use in the HTML form. But, in his case, what happens if the data is dynamic, or if there are too many items to list in a drop-down menu? If you have ever had to include your country in a form (and you live in the US), it is always a pain to have to scroll down to the bottom of the long list to find “United States”.

An easy way to create a dynamic list (or even a long static list) is to use jQuery to access your data from a MySQL database, and to build an auto-populated field in your form. When the user starts typing their entry, jQuery will search the MySQL database for a list of “like” matches, and then return a list of all of the matching results for that field.

For this example, we will be using jQuery, Perl and a MySQL database to auto-populate a “state” field in a form. Once a user has typed three characters in the state field box, a list of matching states will appear in a drop-down menu. It will also return the state abbreviation and state id (code), which can then be saved in the MySQL database after the form has been submitted. In this picture from our example, the user has typed “new”, and returned all states that match that string:

First, let’s start by creating the MySQL database that we will use. Here is the SQL code to create the database, which is named “states”. The table will contain the state name, the state abbreviation, and an ID number.

CREATE TABLE `states` (
`id` int(5) NOT NULL AUTO_INCREMENT,
`state` varchar(50) NOT NULL,
`abbrev` char(2) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=latin1

Next, you will want to import values into the database. You could do this by importing a .csv file containing this information into MySQL, but to make it quicker for this example, here is the SQL:

insert into states (state, abbrev) values ('Alabama', 'AL');
insert into states (state, abbrev) values ('Alaska', 'AK');
insert into states (state, abbrev) values ('Arizona', 'AZ');
insert into states (state, abbrev) values ('Arkansas', 'AR');
insert into states (state, abbrev) values ('California', 'CA');
insert into states (state, abbrev) values ('Colorado', 'CO');
insert into states (state, abbrev) values ('Connecticut', 'CT');
insert into states (state, abbrev) values ('Delaware', 'DE');
insert into states (state, abbrev) values ('District of Columbia', 'DC');
insert into states (state, abbrev) values ('Florida', 'FL');
insert into states (state, abbrev) values ('Georgia', 'GA');
insert into states (state, abbrev) values ('Hawaii', 'HI');
insert into states (state, abbrev) values ('Idaho', 'ID');
insert into states (state, abbrev) values ('Illinois', 'IL');
insert into states (state, abbrev) values ('Indiana', 'IN');
insert into states (state, abbrev) values ('Iowa', 'IA');
insert into states (state, abbrev) values ('Kansas', 'KS');
insert into states (state, abbrev) values ('Kentucky', 'KY');
insert into states (state, abbrev) values ('Louisiana', 'LA');
insert into states (state, abbrev) values ('Maine', 'ME');
insert into states (state, abbrev) values ('Maryland', 'MD');
insert into states (state, abbrev) values ('Massachusetts', 'MA');
insert into states (state, abbrev) values ('Michigan', 'MI');
insert into states (state, abbrev) values ('Minnesota', 'MN');
insert into states (state, abbrev) values ('Mississippi', 'MS');
insert into states (state, abbrev) values ('Missouri', 'MO');
insert into states (state, abbrev) values ('Montana', 'MT');
insert into states (state, abbrev) values ('Nebraska', 'NE');
insert into states (state, abbrev) values ('Nevada', 'NV');
insert into states (state, abbrev) values ('New Hampshire', 'NH');
insert into states (state, abbrev) values ('New Jersey', 'NJ');
insert into states (state, abbrev) values ('New Mexico', 'NM');
insert into states (state, abbrev) values ('New York', 'NY');
insert into states (state, abbrev) values ('North Carolina', 'NC');
insert into states (state, abbrev) values ('North Dakota', 'ND');
insert into states (state, abbrev) values ('Ohio', 'OH');
insert into states (state, abbrev) values ('Oklahoma', 'OK');
insert into states (state, abbrev) values ('Oregon', 'OR');
insert into states (state, abbrev) values ('Pennsylvania', 'PA');
insert into states (state, abbrev) values ('Rhode Island', 'RI');
insert into states (state, abbrev) values ('South Carolina', 'SC');
insert into states (state, abbrev) values ('South Dakota', 'SD');
insert into states (state, abbrev) values ('Tennessee', 'TN');
insert into states (state, abbrev) values ('Texas', 'TX');
insert into states (state, abbrev) values ('Utah', 'UT');
insert into states (state, abbrev) values ('Vermont', 'VT');
insert into states (state, abbrev) values ('Virginia', 'VA');
insert into states (state, abbrev) values ('Washington', 'WA');
insert into states (state, abbrev) values ('West Virginia', 'WV');
insert into states (state, abbrev) values ('Wisconsin', 'WI');
insert into states (state, abbrev) values ('Wyoming', 'WY');
insert into states (state, abbrev) values ('American Samoa');
insert into states (state, abbrev) values ('Guam', 'GU');
insert into states (state, abbrev) values ('Northern Mariana Islands', 'MP');
insert into states (state, abbrev) values ('Puerto Rico', 'PR');

Now that we have the database, we will need to create the Perl script. Here is the Perl script, and you will need to change the database variables to match your system:

#!/usr/bin/perl -w
use CGI;
use DBI;
use DBD::mysql;
use JSON;

# print the http header specifically for json
print "Content-type: application/json; charset=iso-8859-1\n\n";

# your database variables
my $database = "scripting_mysql";
my $host = "192.168.1.2";
my $port = "3306";
my $tablename = "states";
my $user = "root";
my $pass = "root_password";
my $cgi = CGI->new();
my $term = $cgi->param('term');

# mysql connection information
$dsn = "dbi:mysql:$database:$host:$port";

# open the database connection
$connect = DBI->connect($dsn, $user, $pass) || die print "Can't connect - error...";

# prepare the query
$query = $connect->prepare(qq{select id, state AS value, abbrev FROM states where state like ?;});

# execute the query
$query->execute('%'.$term.'%');

# obtain the results
while ( my $row = $query->fetchrow_hashref ){
push @query_output, $row;
}

# close the database connection
$connect->disconnect();

# print the json output to be returned to the HTML page
print JSON::to_json(\@query_output);

For this example, the Perl script should be named states.pl. When executed from the jQuery script in the web page, it will perform a search on the database where the search criteria is in a “like” statement, with a percent sign on both sides of the search term, like this:

# execute the query
$query->execute('%'.$term.'%');

You might want to change this query if you have a long data set where entering a few letters would bring up a result that isn’t what the user would probably want. Here is an example of the same form where we are asking the user to enter their state name, and they enter “min” for Minnesota, but it also brings up Wyoming:

If you want to only search for the beginning of the word, remove the first percent sign, like this:

# execute the query
$query->execute($term.'%');

This would remove Wyoming from being populated as a choice:

Lastly, here is a simple web page that uses jQuery. We are using the jQuery hosted by Google, and we are using one of their css templates. You may delete their css template and use your own. Also, you will need to change the path of your Perl script on this line:

source: "http://192.168.1.2/cgi-bin/mysql/jquery/state.pl"

And here is the HTML:

<html>

<link rel="stylesheet" type="text/css" href="http://ajax.googleapis.com/ajax/libs/jQueryui/1.8.16/themes/base/jQuery-ui.css">

<!-- source = http://code.google.com/apis/libraries/devguide.html#jQuery -->
<script src="https://ajax.googleapis.com/ajax/libs/jQuery/1.6.4/jQuery.min.js"></script>
<script src="https://ajax.googleapis.com/ajax/libs/jQueryui/1.8.16/jQuery-ui.min.js"></script>

<!-- Here is the jQuery code -->
<script>
   $(function(get_states) {
    $('#abbrev').val("");
    $("#state").autocomplete({
    source: "http://192.168.1.2/cgi-bin/mysql/jQuery/state.pl",
    minLength: 3,
    select: function(event, ui) {
     $('#state_id').val(ui.item.id);
     $('#abbrev').val(ui.item.abbrev);
    }
    });
   });
</script>

<!-- The HTML is simplistic for the example: -->
<form method="post">
<table>
<tr><td><p class="ui-widget" ><label for="state"/>State: </label></td>
<td><input type="text" id="state" name="state" size=30/> </td>
<td><input readonly="readonly" type="text" id="abbrev" name="abbrev" maxlength="2" size="2"/></td></tr>
</table>
<input type="hidden" id="id" name="id" />
<p><input type="submit" name="submit" value="Submit" /></p>
</form>

<!-- This section will populate the form field with a list of matching states -->
<script>
  $("#auto_populate_field").submit(function(get_states){
   $("#submitted").html("State: " + $("#state").val() + "<br />State Abbreviation: " + $("#abbrev").val() + "<br />State ID: " + $("#state_id").val());
   return false;
  });
</script>

In this example, once the user starts typing past three characters, a drop-down menu will appear with choices that match the letters being typed. You can decrease/increase the total number of characters that the user will need to type before it queries the database. This is done via this line in the jQuery code:

minLength: 3,

Every time that the person types a character past the initial three characters, the jQuery script will query the MySQL database. This needs to be taken into consideration if you will have a lot of traffic on the pages where you are using jQuery.

The Perl script returns three values in json output – the id, state and abbreviation fields. The values of these fields are place in the text boxes by the autocomplete jQuery function.

In this example, the abbreviation for the state field is returned and placed in the “abbrev” field in the form. The value for the state id is also returned, but since the id field is a hidden field in the form, you won’t see the output/value – but it will be stored in the form.

If you want to see the value of the state id, simply change your web form to look like this:

<form method="post">
<table>
<tr><td><p class="ui-widget" style="textalign:right;"><label for="state"/>State: </label></td>
<td><input type="text" id="state" name="state" size=30/> </td>
<td><input readonly="readonly" type="text" id="abbrev" name="abbrev" maxlength="2" size="2"/></td>
<td><input type="text" size=4 id="state_id" name="state_id" /></td>
</tr>
</table>
<p><input type="submit" name="submit" value="Submit" /></p>
</form>

You can now see the third field which will contain the state id:

When you enter “min” and then click on Minnesota, you will see the state ID (24) in the third field:

You could also use this to auto-populate city and state fields after someone enters their zip code. Or you could even use this to auto-populate the shipping costs for an online order (I might tackle this one in a future post). Hopefully you will be able to copy and paste this example and have a working jQuery form web page example in just a few minutes.

 

—————————————–

Tony Darnell is a Principal Sales Consultant for MySQL, a division of Oracle, Inc. MySQL is the world’s most popular open-source database program.

Tony may be reached at info [at] ScriptingMySQL.com and on LinkedIn.


PlanetMySQL Voting: Vote UP / Vote DOWN

Steve Jobs, you will be missed. Greatly.

Октябрь 6th, 2011

I remember my first computer. It was a TI-99/4A. I bought it back in 1982 (I think), and it cost around $300. The entire computer fit inside what looked like a really thick keyboard. It had a slot on the right for cartridges, and I had a cassette tape drive that I used for backing up the BASIC computer programs that I wrote. I thought that it was a great computer at the time, but I really didn’t have anything to compare it to. The games were the best part of the computer (really the only fun part – my BASIC skills were lacking). And even though the games were fairly lame by even 1982 standards, but they were still plenty of fun to play.

I also remember when I saw a Macintosh for the first time. I had followed Apple for some time, but I had never had the opportunity to actually see a Macintosh. I think it was in 1987 or 1988, when I was a student at the University of Georgia. I believe it was a Mac SE, and it had one megabyte of RAM and a twenty-megabyte hard drive. Up to that point, I had only played around with MS-DOS, and that was on a computer at work. I was amazed at the graphical interface. The mouse was definitely cool. I had to have one, but at that time, they were a couple thousand dollars (at least), and I was a “poor college student”. I spent a lot of time at the computer lab, and I waited.

In 1989, I landed my first job out of college, and one of the first things on my “to-buy” list was a Mac. I just didn’t WANT a Mac – I NEEDED a Mac. So, with a loan from the bank (yes, we took out a loan), my brother and I bought two Mac SE’s, and an HP Deskwriter dot-matrix printer that cost $1,195 (we had to share the printer). I remember opening the box and putting the computer on my desk – and I was thrilled and amazed. With my 2400-baud modem, I could connect to the Unix computers at work. Life was grand. The first major improvement that I made was to upgrade the RAM to four megabytes. I think the upgrade was a whopping $400. And then, about a year later, the hard drive died (one of the only problems that I have ever had with a Mac). So, I upgraded to a forty-megabyte drive for $500. I remember thinking that I will wouldn’t need any more additional hard drive space for a really long time. Besides, I had several hundred 3.5 inch floppies. I later upgraded to a faster processor and bought an internal video card that let me use a huge monitor at the same time (I think it was a 19″ monitor – but it was a grayscale monitor). But having two monitors was cool – very cool – even though a color monitor still eluded me.

And then, about every year or two, I would upgrade to a newer model. I had an SE/30, LC, IIsi, Quadra 840 AV (which I used to edit my first videos), a Power Mac G3, G4 and then several iMac’s. I remember staying up all night playing SIM City on the LC and playing on Prodigy. I had other computers in the house – a Windows machine and a couple of Unix boxes. But I was all-Mac. I remember my first web cam, which I set up to record people driving my the house. The motion-detection software would record a few grayscale photos when cars would drive by. I found out when the trash was picked up and when the mail was delivered. Technology was amazing.

I followed all of the Apple news. Even before I bought my first Mac, I remember when Steve was booted from Apple. I winced as I remember watching Scully and Amelio take Apple to the brink of bankruptcy. When I traveled to the San Francisco area in the mid-1990′s on a business trip, I took some time to drive to Cupertino to the Apple Campus. While I would have loved to have been an Apple employee back then, moving to California was pretty much out of the question. But, I wanted to at least state that I had applied for a job there – which I did. I wasn’t hired, but at least I tried, at least I did that. And I remember when Steve Jobs returned. Mac lovers now had hope. The love affair was rekindled – and it had the opportunity to be much, much stronger than before. Steve would fix it. He had to fix it. And he did.

When I started a company a few years ago, the office was filled with Intel iMac’s and a Mac Pro Server (running MySQL of course!). My wife has a MacBook Pro, my kids have Mac Mini’s, and I just recently purchased a MacBook Pro. We even have an iMac in the kitchen. My Mom has an iMac and my Dad has my Mom’s old G4 (when she upgrades, he gets the hand-me-downs). My sister recently “upgraded” from Windows to an iMac. I gave my niece a MacBook when she started graphics arts school. (My brother is the only current hold-out, but he is on the way to converting back to the Mac). I think you get my point – I really love Macintosh computers and I enjoy “converting people” to the cause.

Needless to say, my life has been touched in some way throughout the past 20-plus years by the Macintosh. I have done everything from creating proposals, creating and listening to music, to editing my youngest son’s birth video (yes, it is rated G) – all on a Macintosh. It is even where I store my tens of thousands of digital photos. I have designed web sites, created software products and ran a business on Macintosh computers. My wife produced a video for my 40th birthday party on her Mac. And, I can’t forget my first iPod, my first iPod Touch, and my iPhones – versions 3 and 4 (I am holding out for the iPad 3). Heck, I even still use the old AppleWorks application every once in a while.

Even though the Mac isn’t and has never been the most dominant “PC” in the marketplace – if you ask anyone that uses a Mac if they would ever use anything else, the vast majority of them would answer with a strong and resounding “no” (or even HELL NO). Even when Apple stock was trading at less than five bucks per share, and it seemed like it was going out of business, I remember telling friends that Apple would pull through (heck – it HAD to pull through). I couldn’t imagine having to use a Windows machine.

And so, I owe a lot of my memories over the past twenty years to Apple Computer, and of course, to Steve Jobs. Steve wouldn’t just manufacture a computer, he would design a work of art (albeit a more expensive work of art). And the operating system was, well, extremely easy to use. Having an Apple computer is tantamount to having a love affair – a comment that I usually don’t hear from users of that other operating system. Steve’s vision for what a computer should do for the user and how the user interacts with the computer was pure genius. And being an ex-Unix admin, once the Mac OS switched to a Unix-based OS, I was even more smitten.

When I had heard the Steve Jobs had passed, I was sad. Well, maybe a bit more than sad. I was never really a big fan of Steve Jobs as a person, mainly because I really didn’t know a lot about him personally – I just loved his products, his vision. When he would announce a new product, you could tell that he was really, really proud of what he had achieved. He wasn’t just up on stage hawking widgets, he was introducing something truly historic. Something that would cause other companies to scurry and to try and play catch-up (they rarely succeeded). I can only hope that the team that he has placed at Apple will continue his legacy of producing innovation at the same level of genius that he has cultivated over the years. If not, then I will also be sad for my two kids, as they won’t get to experience the joy of using a Macintosh as I have. Steve Jobs, you will be missed. Greatly.



PlanetMySQL Voting: Vote UP / Vote DOWN

Using MySQL to Import and Retrieve Blobs and Display as Image Files in HTML

Сентябрь 21st, 2011

I received a phone call from a friend of mine who had some questions about storing image files as blobs in MySQL. He wanted to design a web site that would allow someone to upload an image, and then that image would be stored in the MySQL database for viewing later. He stated that he was going to be storing thousands of images. I mentioned that it might be better to store the images on the file system, and then to just store the location of the image in the database. But, he still wanted to know how to do it and would decide which solution he would incorporate later.

I already had a Perl script that allowed me to upload files to a web site, as I would give out this URL to people that wanted to send me large files. And, I know that you can store images (and other files) in MySQL as a blob, but I wasn’t sure how to display the blob as an image in HTML (once I had retrieved it from the MySQL database). A search on Google provide bits and pieces of the solution, but the solution would only provide the ability to retrieve one image at a time. My friend wanted to be able to display multiple images on a single web page.

I figured out that I needed five things – a database to store the blob, a web page to upload the file into the database, a Perl script to retrieve the results of a search from the database, a Perl script to go back to the database and retrieve the information and blob (image) from a query, and finally a Perl script to display the blobs where an HTML img tag could “read” it. I tried to do it all with just three scripts (ignoring the last script), but I had a problem retrieving and displaying more than one image at a time from a single query. Google just wasn’t providing the answers.

First, I created a “blobs” database to store the images. The database consists of two columns – a serial_number (int, 10) and a mediumblob. In the scripts, I do not check to make sure that the image size is smaller than what is allowed by a mediumblob (16,777,215 bytes), but this could easily be added. Here is the create table statement:

CREATE TABLE `blobs` (
`serial_number` varchar(10) NOT NULL,
`blob_file` mediumblob NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1

Next, I needed a simple web page that will allow the user to upload an image file (and the database may also be used for any other type of file). This form will take the file and place it into the blobs MySQL database. For this example, I have hard-coded a serial number (1234567890) into the web form, but you could easily generate a unique serial number or pull an auto-increment value from the database.

<HTML>
<BODY> </BODY>
<title>Uploading A File</title>
<FORM ACTION="http://192.168.1.2/cgi-bin/mysql/blob_upload.pl" METHOD="post" ENCTYPE="multipart/form-data">
<table>
<tr>
<td>
<font face=verdana size=1 color=#003366> File to Upload:  
</td>
<td>
<INPUT size=70 TYPE="file" NAME="file">
</td>
</tr>
<td>
<input type=hidden name="serial_number" value="1234567890">
</td>
<td>
<INPUT TYPE="submit" NAME="Submit" VALUE="Submit Form">
</td>
</tr>
</table>
</FORM>
</BODY>
</HTML>

The web page sends the information to a Perl script named “blob_upload.pl”, which then places the information (serial number and image file) into the database. The script also creates a directory for today’s date, and then stores a copy of the image on the server in that directory. (For more information on using Perl to connect to MySQL, especially my use of the subroutine ConnectToMySql – see this earlier post. You could also insert the location of the file into another column in the database, but I chose to bypass that functionality for this example. Here is the script:

#! /usr/bin/perl

# DBI is the standard database interface for Perl
# DBD is the Perl module that we use to connect to the MySQL database
use DBI;
use DBD::mysql;
# use CGI for forms
use CGI;
use POSIX 'strftime'; # Need to explicitly load the functions in POSIX

# Note that if you pass no argument to localtime, it assumes the current time
$Date = strftime '%Y-%m-%d', localtime;

$database = "scripting_mysql";
$upload_dir = "$Date\_uploaded_files";

#make directory unless it already exists
mkdir "$upload_dir", 0777 unless -d "$upload_dir";

$query = new CGI;

$filename = $query->param("file");
$serial_number = $query->param("serial_number");

print $query->header ( );

$filename =~ s/.*[\/\\](.*)/$1/;
$filename =~ s/ /_/g;
$upload_filehandle = $query->upload("file");
$directory_filename = "$upload_dir/$filename";

print "$filename $serial_number";

# upload the file to the server

open UPLOADFILE, ">$directory_filename";
binmode UPLOADFILE;

while ( <$upload_filehandle> )
{
    print UPLOADFILE;
}

close UPLOADFILE;

# Open the file
open MYFILE, $directory_filename || print "Cannot open file";
my $blob_file;

# Read in the contents
while (<MYFILE>) {
    $blob_file .= $_;
    }

close MYFILE;

#----------------------------------------------------------------------
# insert the values into the database
#----------------------------------------------------------------------

# invoke the ConnectToMySQL sub-routine to make the database connection
    $dbh = ConnectToMySql($database);

    # set the value of your SQL query
    $query = "insert into blobs (serial_number, blob_file) values (?, ?) ";

    $sth = $dbh->prepare($query);
    $sth->execute($serial_number, $blob_file) || print "Can't access database";

    $sth->finish;
    $dbh->disconnect;

#----------------------------------------------------------------------

print " <TITLE>File Uploaded</TITLE>";

print " <font face=verdana size=2 color=#003366>Thanks for uploading your file!<br>";
print " <font face=verdana size=2 color=#003366>File directory: $Date\_uploaded_files<br>";
print " <font face=verdana size=2 color=#003366>Your file: $filename<br>";
print " <font face=verdana size=2 color=#003366>Your serial number: $serial_number<br>";

#----------------------------------------------------------------------
sub ConnectToMySql {
#----------------------------------------------------------------------

my ($db) = @_;

my $host="192.168.1.2";

open(ACCESS_INFO, "<..\/accessBLOB") || die "Can't access login credentials";
    my $database = <ACCESS_INFO>;
    my $host = <ACCESS_INFO>;
    my $userid = <ACCESS_INFO>;
    my $passwd = <ACCESS_INFO>;

    # the chomp() function will remove any newline character from the end of a string
    chomp ($database, $host, $userid, $passwd);

my $connectionInfo="dbi:mysql:$database;$host";
close(ACCESS_INFO);

# make connection to database
my $l_dbh = DBI->connect($connectionInfo,$userid,$passwd);
return $l_dbh;

}


Now we have a web page to upload the image files and a Perl script to store the image files. We just need a script to print out the serial number and the blob – but we need to print the blob out as an image in HTML by using the img tag. This is the tricky part. To accomplish this, we create another Perl script that simply “prints” the blob out in a way that an HTML img tag can read it. But first, here is a script to go to the database and retrieve the blob based upon a serial number (which is hard coded in this example). This script is titled “blob_view.pl”. In this script, we are sending a query to the MySQL database to retrieve the image based upon the serial number. But, the script has a “while loop” so that you can bring back more than one image if you desire – and in this loop we call the fourth script – get_blob.pl – to bring each blob back one at a time and to display it inside the HTML img tag. Here is the script to retrieve the image with the same serial number from the web page (hard coded – serial number 1234567890):

#! /usr/bin/perl

# DBI is the standard database interface for Perl
# DBD is the Perl module that we use to connect to the MySQL database
use DBI;
use DBD::mysql;
# use CGI for forms
use CGI qw(:standard);

print header;
print "<html><body>Here is your picture:<p>";

$dbh = ConnectToMySql($Database);

$query = "select serial_number from blobs where serial_number = '1234567890'";

$sth = $dbh->prepare($query);

$sth->execute();

while (@data = $sth->fetchrow_array()) {
$serial_number = $data[0];

print "<p>$serial_number <p> <img width=200 src=\"get_blob.pl?sn=$serial_number\"><br>get_picture.pl?sn=$serial_number";

}

$sth->finish;
$dbh->disconnect;

#----------------------------------------------------------------------
sub ConnectToMySql {
#----------------------------------------------------------------------

my ($db) = @_;

my $host="192.168.1.2";

open(ACCESS_INFO, "<..\/accessBLOB") || die "Can't access login credentials";
my $database = <ACCESS_INFO>;
my $host = <ACCESS_INFO>;
my $userid = <ACCESS_INFO>;
my $passwd = <ACCESS_INFO>;

# the chomp() function will remove any newline character from the end of a string
chomp ($database, $host, $userid, $passwd);
#print "<br>$database $host $userid $passwd <br>";

my $connectionInfo="dbi:mysql:$database;$host";
close(ACCESS_INFO);

chomp($userid);
chomp($passwd);
# make connection to database
my $l_dbh = DBI->connect($connectionInfo,$userid,$passwd);
return $l_dbh;

}

The key here is to use another script named get_blob.pl that only returns the image, which allows us to use an HTML img tag to display the image. You pass the serial number to get_blob.pl as a variable named “sn”, and it retrieves the blog so that the img tag can display it. There are other ways in Perl to display a single image without using another script, but I wanted this to be able to handle multiple images from a single query. If I changed the query to something like “$query = “select serial_number from blobs where serial_number like ’123456789%’”;”, it would retrieve more than one row (in my case), and print each image separately. So, you will need this script as well to retrieve the images:

#! /usr/bin/perl

# DBI is the standard database interface for Perl
# DBD is the Perl module that we use to connect to the MySQL database
use DBI;
use DBD::mysql;
# use CGI for forms
use CGI qw(:standard);

print header;
print "<html><body>Here is your picture:<p>";

     $dbh = ConnectToMySql($Database);

     $query = "select serial_number from blobs where serial_number like '1234567890'";
     
     $sth = $dbh->prepare($query);

     $sth->execute();
     

while (@data = $sth->fetchrow_array()) {
$serial_number = $data[0];
#$blob_file = $data[1];

print "<p>$serial_number <p> <img width=200 src=\"get_picture.pl?sn=$serial_number\"><br>get_picture.pl?sn=$serial_number";

}

     $sth->finish;
     $dbh->disconnect;

#----------------------------------------------------------------------
sub ConnectToMySql {
#----------------------------------------------------------------------

     my ($db) = @_;

     my $host="192.168.1.2";

     open(ACCESS_INFO, "<..\/accessBLOB") || die "Can't access login credentials";
     my $database = <ACCESS_INFO>;
     my $host = <ACCESS_INFO>;
     my $userid = <ACCESS_INFO>;
     my $passwd = <ACCESS_INFO>;

     # the chomp() function will remove any newline character from the end of a string
     chomp ($database, $host, $userid, $passwd);

     my $connectionInfo="dbi:mysql:$database;$host";
     close(ACCESS_INFO);

     chomp($userid);
     chomp($passwd);
     # make connection to database
     my $l_dbh = DBI->connect($connectionInfo,$userid,$passwd);
     return $l_dbh;

}

This may seem like a lot of code, and I am sure that some Perl experts will show me what I am doing wrong (for example, I didn’t use strict or warnings), or a way to do it with less code. But I like to write this code so that a novice Perl user will be able to (hopefully) figure out what I am doing and be able to incorporate it into their project. I know that Google didn’t provide me with an easy answer, so hopefully this will help someone with their next project.

 

—————————————–

Tony Darnell is a Principal Sales Consultant for MySQL, a division of Oracle, Inc. MySQL is the world’s most popular open-source database program.

Tony may be reached at info [at] ScriptingMySQL.com and on LinkedIn.


PlanetMySQL Voting: Vote UP / Vote DOWN

Splitting a MySQL Dump File Into Smaller Files Via Perl

Сентябрь 14th, 2011

I was trolling the MySQL forums web site and noticed a post regarding someone who was trying to load a 50-gigabyte MySQL dump file. The author of the post had stated that the loading of the file had been running for days, and was wondering if there was a better way to import the file. They did not have access to anything else (i.e. – original database) but this file.

I have had to restore several databases in the past from a single large MySQL dump file – which led me to start backing up each database individually. These databases are for my own personal use and are not updated that often, so I don’t need to have point-in-time recovery – and so a MySQL dump works just fine. If I had a production system, I would invest in the MySQL Enterprise Backup and the MySQL Enterprise Monitor applications.

But, I have had problems when I tried to restore a relatively small MySQL instance of 21 databases with a total file size of less than 900 megabytes. So, I wrote a Perl script to simply split the large MySQL dump file into smaller files, which I could then load one at a time to isolate the problem. I am sure that there are many other ways of troubleshooting this person’s problem, but I thought I would dig out this script and post it here. This script is also handy for splitting any text file that you have into smaller files. All you would have to do is to change the keywords that causes a new file to be created.

The script opens the MySQL dump file, and starts outputting each line to a new file, looking for the keywords “CREATE DATABASE”. When the script sees those keywords, it closes the last file and opens a new one, and then prints that line with the keywords out to the new file until the keyword appears again. The script names each new file with a sequential number – as in database_01.sql, database_02.sql, etc. The first file that this script creates (named database_00.sql) contains the header information from the MySQL dump file, so you may ignore this file.

I tested this script on an 850-megabyte MySQL dump file, and it worked perfectly (and executed in less than one minute). To make sure that your new files contain the same amount of data as the original file, you can use the Unix command “wc” to show you the size of each file, and then you can compare the sum of these sizes to the size of your MySQL dump file. I first ran the script, and it produced the following 22 files:

sh-3.2# /usr/bin/perl split_file.pl
2011-09-13 21:24:23 - Creating file database_00.sql
2011-09-13 21:24:23 - Creating file database_01.sql
2011-09-13 21:24:23 - Creating file database_02.sql
2011-09-13 21:24:23 - Creating file database_03.sql
2011-09-13 21:24:23 - Creating file database_04.sql
2011-09-13 21:24:23 - Creating file database_05.sql
2011-09-13 21:24:23 - Creating file database_06.sql
2011-09-13 21:24:23 - Creating file database_07.sql
2011-09-13 21:24:23 - Creating file database_08.sql
2011-09-13 21:24:23 - Creating file database_09.sql
2011-09-13 21:24:23 - Creating file database_10.sql
2011-09-13 21:24:23 - Creating file database_11.sql
2011-09-13 21:24:23 - Creating file database_12.sql
2011-09-13 21:24:23 - Creating file database_13.sql
2011-09-13 21:24:23 - Creating file database_14.sql
2011-09-13 21:24:24 - Creating file database_15.sql
2011-09-13 21:24:24 - Creating file database_16.sql
2011-09-13 21:24:24 - Creating file database_17.sql
2011-09-13 21:24:24 - Creating file database_18.sql
2011-09-13 21:24:24 - Creating file database_19.sql
2011-09-13 21:24:24 - Creating file database_20.sql
2011-09-13 21:25:14 - Creating file database_21.sql

Here are the sizes of the individual database*.sql files, as well as the original source file named all_data.sql:

sh-3.2# wc database*sql
      21      71      844 database_00.sql
      33     109      868 database_01.sql
     395   18313  1134377 database_02.sql
     578  101269  5280603 database_03.sql
     553   14095   366305 database_04.sql
     553   10050   263748 database_05.sql
       8      21      139 database_06.sql
      70    2783    37976 database_07.sql
     547   11055   410751 database_08.sql
     320   10888   426911 database_09.sql
      74     337     6727 database_10.sql
     282   12624   325737 database_11.sql
     255    1863    42550 database_12.sql
      44     152     1337 database_13.sql
      57   82921 13048515 database_14.sql
     737   73955   908495 database_15.sql
      44     258     2972 database_16.sql
     191  535786  8777550 database_17.sql
     129    2325    36632 database_18.sql
       8      21      139 database_19.sql
  1225 28094331 826795680 database_20.sql
      312  134027 5401957 database_21.sql
  6436 29107254 863270813 total

sh-3.2# ls -l all_data.sql
-rw-rw-rw- 1 tonydarnell admin 863270813 Sep 13 20:23 all_data.sql

As you can see, the total size of the database*.sql files is 863270813, and the size of our original MySQL dump file (all_data.sql) is the same size – 863270813.

After splitting the main MySQL dump file, you can now restore/import each file individually, and if you have any problems, they will be easier to troubleshoot.

Here is the script:

#!/usr/bin/perl -w

# Need to explicitly load the functions in POSIX
use POSIX 'strftime';

# file name of the file you want to split
$filename = "all_data.sql";

# Note that if you pass no argument to localtime, it assumes the current time
$time = strftime '%Y-%m-%d %H:%M:%S', localtime;

$count = 0;
$filecount = 0;

# syntax for new file names
# if you have more than 100 files, you could modify this statement to place a leading
# zero if the file is greater than 10 but less than 100
if ($filecount < 10) {

     $new_filename = "database_0" . $filecount . ".sql";

}

else

{

     $new_filename = "database_" . $filecount . ".sql";

}

print "$time - Creating file $new_filename\n";
open(OUTPUT, ">$new_filename") || die "Can't redirect stdout";

open(LINE ,"$filename") || die ("\nERROR - could not open file: $filename\n");
while (<LINE>)

{
     
          if ($_ =~ "^CREATE DATABASE" && $count > 1)
          
          {

               # close the old file
               close(OUTPUT);

               $filecount++;
          
               # syntax for new file names
               if ($filecount < 10) {
               
                    $new_filename = "database_0" . $filecount . ".sql";
               
               }
               
               else
               
               {
               
                    $new_filename = "database_" . $filecount . ".sql";
               
               }
               
               $time = strftime '%Y-%m-%d %H:%M:%S', localtime;
               print "$time - Creating file $new_filename\n";

               open(OUTPUT, ">$new_filename") || die "Can't redirect stdout";
          
               print OUTPUT $_;

          }
          
          else
          
          {
          
               print OUTPUT $_;
          
          }

$count++;

# end while
}

close (LINE);
close (OUTPUT);

 

—————————————–

Tony Darnell is a Principal Sales Consultant for MySQL, a division of Oracle, Inc. MySQL is the world’s most popular open-source database program.

Tony may be reached at info [at] ScriptingMySQL.com and on LinkedIn.


PlanetMySQL Voting: Vote UP / Vote DOWN

MySQL – Use mysqldump and FTP the Backup Files Offsite via Perl

Сентябрь 8th, 2011

In an earlier post (Scripting Backups of MySQL with Perl via mysqldump), I showed you how to use Perl to script a mysqldump backup of selected databases. Reminder – using mysqldump to backup your databases is not as safe or effective as using MySQL Enterprise Backup.

A blog reader sent me an email asking me how they could automate sending the backup files via FTP to another server (preferably one that is off-site). This was easily accomplished by adding a series of FTP commands to the original script.

As in the first post, we use a text file that contains a list of the databases that we want to backup. See this post for more information on using the script to backup your files – I don’t want to duplicate the entire post here.

For the FTP part of this script, you will need to edit the script to make sure that you change the following to match your systems:

# this is your MySQL user name and password
$mysql_user_name = "root";
$mysql_user_password = "my_password";

# this is your FTP site name, user name and password
$ftp_site = "my_hostname.com";
$ftp_user_name = "my_user_name";
$ftp_password = "my_password";
# create this folder on your FTP backup site under your root/login directory
$ftp_backup_directory = "backups";

# set the local directory where you will keep the backup files
$backup_folder = '/Users/tonydarnell/cgi/mysql/mysqlbak';

The script will create a folder on the local server under the $backup_folder directory, with the naming convention of YYYY-MM-DD-HHMM, and place the backup files for each database in this directory. It will then create the same folder on your FTP server under the $ftp_backup_directory, backup each database, and then send all of those files to your FTP server to be placed in the same directory. The script does a backup for each database individually, compresses the file, and then sends each file via FTP before it backs up the next database.

For example, on your local server, it will create a directory such as “2011-09-07-1756″ under the $backup_folder directory (in this example – /Users/tonydarnell/cgi/mysql/mysqlbak). The script will also create the same directory “2011-09-07-1756″ on the FTP server under the $ftp_backup_directory named “backups”. This $ftp_backup_directory should be created under the root directory (or the default login directory) for your FTP user prior to running the script.

Of course, it will be up to you to remove the older backups from the local server and FTP server. I also have quite a few print statements showing you each step of the process. Feel free to comment out these lines if you don’t want the script to print anything.

Here is a sample output from the script, backing up one database named cbgc:

# /usr/bin/perl mysql_backup_ftp.pl
Backing up cbgc...
FTP - 2011-09-07-2008_cbgc.sql.Z... Goodbye.
FTP Server ready.
User u36379513-777 logged in
CWD command successful
Type set to I
Opening BINARY mode data connection for 2011-09-07-2008_cbgc.sql.Z
Transfer complete
Finished backing up cbgc...

Here is the script:


#!/usr/bin/perl

use Net::FTP;
use warnings;
use File::Basename;

# this is your MySQL user name and password
$mysql_user_name = "root";
$mysql_user_password = "my_password";

# this is your FTP site name, user name and password
$ftp_site = "myhostname.com";
$ftp_user_name = "my_user_name";
$ftp_password = "my_password";
# create this folder on your FTP backup site
$ftp_backup_directory = "backups";

# set the directory where you will keep the backup files
$backup_folder = '/Users/tonydarnell/cgi/mysql/mysqlbak';

# we use a config file to keep a list of all of the databases that we want to backup
# You may use a comment to bypass any database that you don't want to backup
# example config file
# # unwanted_database (commented - will not be backed up)
# database_name1
# database_name2

# the config file is a text file with a list of the databases to backup
# this should be in the same location as this script, but you can modify this
# if you want to put the file somewhere else
my $config_file = dirname($0) . "/mysql_bak.config";

# retrieve a list of the databases from the config file
my @databases = removeComments(getFileContents($config_file));

# change to the directory of the backup files.
chdir($backup_folder) or die("Cannot go to folder '$backup_folder'");

# grab the local time variables
my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(time);
$year += 1900;
$mon++;
#Zero padding
$mday = '0'.$mday if ($mday<10);
$mon = '0'.$mon if ($mon<10);
$hour = "0$hour" if $hour < 10;
$min = "0$min" if $min < 10;

# create the name of the backup folder that will contain all of the backup files
my $folder = "$year-$mon-$mday-$hour$min";
mkdir($folder) or die("Cannot create a folder called '$folder'");

    # login to the FTP server and create the folder to store the backup files
    $ftp = Net::FTP->new("$ftp_site", Debug => 0)
    or die "Cannot connect to $ftp_site: $@\. ", $ftp->message;

    # optional - comment or remove this line to supress print statement
    print $ftp->message;
    $ftp->login("$ftp_user_name","$ftp_password")
    or die "Cannot login with $user_name. ", $ftp->message;

    # optional - comment or remove this line to supress print statement
    print $ftp->message;
    $ftp->cwd("/$ftp_backup_directory")
    or die "Cannot change working directory to $ftp_backup_directory. ", $ftp->message;

    # optional - comment or remove this line to supress print statement
    print $ftp->message;
    $ftp->mkdir("$folder")
    or die "Cannot create directory $folder. ", $ftp->message;

    # optional - comment or remove this line to supress print statement
    print $ftp->message;
    $ftp->quit;
    # end FTP

# backup each database contained in the @databases array
foreach my $database (@databases) {
    next if ($database eq '');
    chomp($database);

    my $table = '';
    # Get just 1 table in the database - if there is a ' '(space) in the db name
    if(index($database,' ')+1) {
     my @parts = split(' ',$database);
     $database = $parts[0];
     $table = $parts[1];
    }

    print "Backing up $database...\n";

    # we add the year, month, day and time of day to the backup filename
    my $file = "$year-$mon-$mday-$hour$min\_$database";
    $file .= '_' . $table if($table ne '');
    $file .= ".sql";

    # perform a mysql dump on each database
    # change the path of mysqldump to match your system's location
    # and add/change any of the backup options - this is just an example
    # make sure that you change the root password above to match the correct password
    `/usr/local/mysql/bin/mysqldump -R --triggers -u $mysql_user_name --password=$mysql_user_password $database $table | compress > $folder/$file.Z`;

    # start FTP
    print "FTP - $file.Z... ";

    # optional - comment or remove this line to supress print statement
    print $ftp->message;
    $ftp = Net::FTP->new("$ftp_site", Debug => 0)
    or die "Cannot connect to $ftp_site: $@\. ", $ftp->message;

    # optional - comment or remove this line to supress print statement
    print $ftp->message;
    $ftp->login("$ftp_user_name","$ftp_password")
    or die "Cannot login with $user_name. ", $ftp->message;

    # optional - comment or remove this line to supress print statement
    print $ftp->message;
    $ftp->cwd("/$ftp_backup_directory/$folder")
    or die "Cannot change working directory to $ftp_backup_directory. ", $ftp->message;

    # optional - comment or remove this line to supress print statement
    print $ftp->message;
    $ftp->binary()
    or die "Change to binary mode failed. ", $ftp->message;

    # optional - comment or remove this line to supress print statement
    print $ftp->message;
    $ftp->put("$folder/$file.Z")
    or die "Put failed - $folder. ", $ftp->message;

    # optional - comment or remove this line to supress print statement
    print $ftp->message;
    $ftp->quit;
    # FTP end

    print "Finished backing up $database...\n\n";
}

exit;

# this subroutine simply creates an array of the list of the databases
sub getFileContents {
    my $file = shift;
    open (FILE,$file) || die("Can't open '$file': $!");
    my @lines=<FILE>;
    close(FILE);

    return @lines;
}

# remove any commented tables from the @lines array
sub removeComments {
    my @lines = @_;

    @cleaned = grep(!/^\s*#/, @lines); #Remove Comments
    @cleaned = grep(!/^\s*$/, @cleaned); #Remove Empty lines

    return @cleaned;
}

 

—————————————–

Tony Darnell is a Principal Sales Consultant for MySQL, a division of Oracle, Inc. MySQL is the world’s most popular open-source database program.

Tony may be reached at info [at] ScriptingMySQL.com and on LinkedIn.


PlanetMySQL Voting: Vote UP / Vote DOWN