A Master-Slave-Slave Replication in Production

Two new slave servers are added to the Production System. The master server is ‘Alpha’ and the new slave servers are ‘Beta’ and ‘Gamma’. The environment is Red Hat Enterprise Linux. If the binary logging is not already enabled, we need to enable it on the master server by editing the “my.cnf” as shown below. If it is already enabled, find out the current location of the binary logging by using the command: show master status.

1. Configure the Master Server: Alpha

Add the following lines in my.cnf:

[mysqld]

log-bin = alpha-bin

server-id = 1

Restart the server:

# /etc/init.d/mysql restart

See the master status:

mysql> show master status;

+————————–+———-+————–+——————+

| File  | Position | Binlog_Do_DB | Binlog_Ignore_DB |

+————————–+———-+————–+——————+

|alpha-bin.000028 | 98      |                |                  |

+————————–+———-+————–+——————+

1 row in set (0.00 sec)

             Create a Replication user:

mysql> create user ‘replicuser’@’alpha.localdomain.com’ identified by ‘replicpass’;

mysql> GRANT REPLICATION SLAVE ON *.* TO replicuser@”%” IDENTIFIED BY ‘replicpass’;

It is important to note the position of the master log (binary log) exactly shown above. This position is given in the slave servers to point to the location to start the replication. If the position is not specified correctly, the slave server will simply fail to start replication.

2. Backup the Master Server

It is safe to take a cold backup on the server by shutting down MySQL on the server and using mysqldump, as shown below:

mysqldump –u root –p –all-databases > backup.sql

It is not online backup. So –single-transaction, –flush-logs options on mysqldump are not required. It may take hours to complete, based on the size of your databases. Once it is completed, copy the dump file from Alpha to Beta and Gamma:

# scp backup.sql mysqldba@beta.localdomain.com:/var/lib/mysql

# scp backup.sql mysqldba@gamma.localdomain.com:/var/lib/mysql

where ‘mysqldba’ is a Linux user on the slave servers and ‘localdomain’ is your domain in which alpha, beta and gamma are servers.

Restore the databases on Beta and Gamma as shown below:

# mysql –u root –p < backup.sql

The restoration used to fail many times because of some inconsistency in the table structure. If it encounters such an error, mysql will not continue and you have not other option but to repeat it all from the beginning. In case it fails during restoration, you must delete the databases already created and then restore it. Safely you may issue a forced restoration too:

# mysql –u root –p –force < backup.sql

Once the dumpfile is created on the master server, you may start MySQL on the the master server. If the dumpfile is very large in size, you may want to create different dumpfiles instead of one single large one. I have created three separate dumpfiles by grouping together some large tables from a database like this:

mysqldump -u root -p production hist0 hist1 hist2 hist31 hist32 hist33 hist34 hist41 hist42 hist43 hist44 hist51 hist52 hist53 hist54 hist61 hist62 hist63 hist64 hist71 hist72 hist73 hist74 > productionone.sql

mysqldump -u root -p production chist0 chist1 chist2 chist31 chist32 chist33 chist34 chist41 chist42 chist43 chist44 chist51 chist52 chist53 chist54 chist61 chist62 chist63 chist64 chist71 chist72 chist73 chist74 > productiontwo.sql

mysqldump -u root -p production –ignore-table=production.hist0 –ignore-table=production.hist1 –ignore-table=production.hist2 –ignore-table=production.hist31 –ignore-table= hist32 –ignore-table=hist33 –ignore-table=hist34 –ignore-table=hist41 –ignore-table=hist42 –ignore-table=hist43 –ignore-table=hist44 –ignore-table=hist51 –ignore-table=hist52 –ignore-table=hist53 –ignore-table=hist54 –ignore-table=hist61 –ignore-table=hist62 –ignore-table=hist63 –ignore-table=hist64 –ignore-table=hist71 –ignore-table=hist72 –ignore-table=hist73 –ignore-table=hist74 –ignore-table=chist0 –ignore-table=chist1 –ignore-table=chist2 –ignore-table=chist31 –ignore-table=chist32 –ignore-table=chist33 –ignore-table=chist34 –ignore-table=chist41 –ignore-table=chist42 –ignore-table=chist43 –ignore-table=chist44 –ignore-table=chist51 –ignore-table=chist52 –ignore-table=chist53 –ignore-table=chist54 –ignore-table=chist61 –ignore-table=chist62 –ignore-table=chist63 –ignore-table=chist64 –ignore-table=chist71 –ignore-table=chist72 –ignore-table=chist73 –ignore-table=chist74 > productionthree.sql

3. Configure the Slave Server: Beta

Add the following lines in my.cnf:

[mysqld]

server-id = 2

Restart the server:

# /etc/init.d/mysql restart

Run the following:

mysql> CHANGE MASTER TO

MASTER_HOST = ‘alpha.localdomain.com’,

MASTER_USER = ‘replicuser’,

MASTER_PASSWORD = ‘replicpass’,

MASTER_LOG_FILE=’alpha-bin.000028’,

MASTER_LOG_POS=98;

mysql> start slave;

mysql> show slave status \G

4. Configure the Slave Server: Gamma

Add the following lines in my.cnf:

[mysqld]

server-id = 3

Restart the server:

# /etc/init.d/mysql restart

Run the following:

mysql> CHANGE MASTER TO

MASTER_HOST = ‘alpha.localdomain.com’,

MASTER_USER = ‘replicuser’,

MASTER_PASSWORD = ‘replicpass’,

MASTER_LOG_FILE=’alpha-bin.000028’,

MASTER_LOG_POS=98;

mysql> start slave;

mysql> show slave status \G

We do not want the users to modify the tables in the databases on the slave servers. That means we need to make the databases “read-only”. Also, we do not want the tables in “mysql” database to be replicated to the slave servers. The following two lines are added to the “my.cnf” file on both slave servers.

–read-only

–replicate-ignore-db=mysql

But in most cases, users want some database to write their intermediate or temporary tables. We can create a database called “sandbox” on each server. Here the users must be given permissions to create tables. Also those users must have the global privilege of ’super’ to do so.

Errors can occur on the slave servers if a query was aborted on the master server or a table does not exist on the slave which originally exist on the master. The replication stops by showing the following error. We can choose to execute the query on the slave server, or ignore it if you are sure that this statement can be skipped. At MySQL prompt on the slave server, set global sql_slave_skip_counter=1 and then start slave.

Relay_Master_Log_File: alpha-bin.000028

Slave_IO_Running: Yes

Slave_SQL_Running: No

Replicate_Do_DB:

Replicate_Ignore_DB: mysql,mysql

Replicate_Do_Table:

Replicate_Ignore_Table:

Replicate_Wild_Do_Table:

Replicate_Wild_Ignore_Table:

Last_Errno: 1053

Last_Error: Query partially completed on the master (error on master: 1053) and was aborted. There is a chance that your master is inconsistent at this point. If you are sure that your master is ok, run this query manually on the slave and then restart the slave with SET GLOBAL SQL_SLAVE_SKIP_COUNTER=1; START SLAVE; . Query: ‘update liq_all, arbitration.claim_form SET liq_all.arb_date = replace(insertdate,’-',”) WHERE db_num=camref’

Skip_Counter: 0

SQL statements will cease being run on the slave (via replication) until the above error is resolved either by executing the query or by skipping the statement. On MySQL prompt, run the following:

mysql> set global sql_slave_skip_counter=1;

mysql> start slave;

Before proceeding to production on a complex database server, it is always necessary to benchmark your server. There are different tools available for this purpose like that of Todd.

There is a tool in MySQL itself. Though primitive, it is interesting to try it out. The MySQL Benchmark Tool, sql-bench, is a perl program, which can be found in the folder sql-bench under the MySQL program folder.

Installation of Perl DBD/DBI for MySQL

On Windows, you should do the following to install the MySQL DBD module and DBI with ActiveState Perl.  Get ActiveState Perl 5.6 or later from

http://www.activestate.com/Products/ActivePerl/

and install it. Open a command window.  Type ppm at the command prompt. The Perl Package Manager starts as follows: 

 

Here you may add new packages. Add the package DBD-MySQL, the MySQL Driver for Perl Database Interface (DBI). Then add the package DBI also. 

Start the MySQL Benchmark Tool 

At sql-bench directory of MySQL, run the following command: 

Sql-bench:\> perl run-all-tests – -host {host name} - -user root – -password {password}

See below one of the sample screens it produces on Innodb stress test:

Testing server ‘MySQL 5.0.36sp1 enterprise gpl nt log/’ at 2007-0627 14:15:44 Innotest2: MySQL/InnoDB stress test in Perl for FOREIGN keys This is a randomized stress test for concurrent inserts, updates, deletes, commits and rollbacks with foreign keys with the ON DELETE … clause. The test will generatealso a lot of deadlocks, duplicate key errors, and other SQL errors.You should run innotest2, innotest2a, and innotest2b concurrently.The thing to watch is that the server does not crash or does not print to the .err log anything. Currently, due to a buglet in MySQL, warnings about MySQL lock reservations can appear in the .err log. The test will run very long, even several hours. You can kill the perl processes running this test at any time and do CHECK TABLE on tables innotest2a, b, c, d in the ‘test’ database.

The two common tools of Backup in MySQL are the following: 

1.        mysqldump

2.        mysqlhotcopy

If we are using innodb database engine, it is possible to perform online backup with only mysqldump. It is possible for you to use mysqlhotcopy, in case you are using MyISAM database engine.

Assume that we have scheduled the full database backup daily at 3 am. As the database goes into production, we may need to revise this plan. We may still continue to do a full database backup daily. In addition, there could be incremental backups on an hourly basis.

C:\Program Files\MySQL\MySQL Server 5.0\bin> mysqldump –u root –p – - all-databases – - single-transaction > backup.sql

In order to make the incremental backups, we need to start MySQL server with the option --log-bin. If you have implemented Replication, it creates the binary log. It is stored in, for example, E:\mysqlBinlog, as shown below:

The last file ‘mysql-bin.000081’ is being updated currently.  These binary logs form the incremental backups.  Hence while we conduct a full backup, ensure that the cache is flushed so that a new binary log is created. The full backup plus all the binary logs created after the full backup constitute the data at present.

C:\Program Files\MySQL\MySQL Server 5.0\bin> mysqldump –u root –p – -all-databases – -single-transaction – -flush-logs – -master-data=2 > e:\Backup\2007-06-25_10213136.sql 

It adds the following lines to the file 2007-06-25_10213136.sql, which will be useful at the time of Recovery:

– Position to start replication or point-in-time recovery from 

– CHANGE MASTER TO MASTER_LOG_FILE=’mysql-bin.000082′, MASTER_LOG_POS=98; 

The listing in E:\mysqlBinlog currently will be as shown below: 

Essentially, 2007-06-25_10213136.sql and mysql-bin.000082 together form the most up to date database at any point of time. We may copy this binary file and the later ones to a safe location on every hour.The following is a backup script with name “backup.bat” created on the server in the folder C:\Program Files\MySQL\MySQL Server 5.0\data. It creates backup file once daily at 3 am and stores it in the folder E:\Backup. The script file contains the password of root user. So we need to control access to it.

 This script is scheduled to run at 3 am everyday, by Windows Scheduler.    

MySQL has a lot of parameters to fine tune the server and databases. Many more parameters are there specific to the storage engines you choose. Here are some important MySQL and InnoDB parameters that are identified by using tools such as MySQL Administrator and phpMysqladmin.

Installing & Configuring PHP with IIS

Download the Windows binary of PHP 5 from php.net. Unzip it to, say, C:\PHP. Open the “php.ini-dist” in C:\PHP and save it as “php.ini”. Locate the following line in php.ini:                         ; cgi.force_redirect = 1

We need to uncomment this line and change the setting to ‘0’ to run PHP under IIS, as shown below:

cgi.force_redirect = 0

Click Start -> Control Panel -> Administrative Tools -> Internet Information Services:

Select Default Web Site, right-click on it and select Properties:

Select the Home Directory tab and click on Configuration. In the Application Configuration window, select Mappings and click on the add button.

Enter the executable as C:\PHP\PHP-CGI.EXE and the extension as .php, as shown below.  

Click on OK button. This configuration enables IIS to run PHP files using PHP interpreter.  Finally, right-click on My Computer, select Properties, select Advanced tab and click on Environment Variables. In System Variables, select Path and click on Edit button. Add these paths to the variable value:                        C:\PHP;C:\PHP\EXT

Click OK to finish.

Configuring PHP with SQL Server  PHP supports SQL Server with a set of mssql_xxx functions. The PHP extension directory (normally \PHP\EXT) contains all the database libraries, like php_mssql.dll for SQL Server. Open php.ini. Verify that the “extension_dir=” points to the folder in which the database libraries are stored, as shown below:extension_dir = “c:\php\ext”

Next, look for the line “;extension=php_mssql.dll” in php.ini. Uncomment this line. It tells PHP to load the php_mssql.dll extension library into memory, which makes the mssql_xxx set of functions available to us.  You need the SQL client tools installed on the Web Server. The minimum requirement is the file “ntwdblib.dll”, which can be located in the \Windows\System32 folder on your SQL Server. Copy it to \Windows\System32 folder on your Web Server.  Now restart your web server, IIS. Create a file called “test.php” in notepad and enter the following:$server=”Beta”;$username=”sa”;$password=”";$sqlconnect=mssql_connect($server, $username, $password);$sqldb=mssql_select_db(“opus”,$sqlconnect);$sqlquery=”SELECT order_no FROM Orderlines;”;$results= mssql_query($sqlquery);while ($row=mssql_fetch_array($results)){echo $row['order_no'].”<br>\n”;}mssql_close($sqlconnect); 

Save the file in the web server folder “c:\Inetpub\wwwroot”. Open Internet Explorer and type the URL:           

http://alpha/test.php

Here Alpha is the Web Server and Beta is the Database Server. The above URL returns the query results on the browser. It is possible to execute SQL statements as well as Stored Procedures of SQL Server in a similar manner. 

You have Windows XP or Vista. Download the other packages from their respectives sites:

1. Apache from httpd.apache.org
2. MySQL from www.mysql.com
3. PHP from www.php.net

Download the latest stable versions. In this section, we are using Apache 2.0.55, MySQL 5.0.18 and PHP 5.1.5.

After installing Apache, you may see the files in the folder C:\Program Files\Apache Group\Apache2\. The web pages are stored in “htdocs” sub-folder and the configuration files are stored in “conf” sub-folder. You may save your web pages in a new folder “c:\htdocs”. In this case you need to modify the “httpd” file in “conf” folder to make these changes:

DocumentRoot “C:/htdocs”

<Directory “C:/htdocs”>

Open a browser and type

http://localhost

If the Apache is installed properly, you will see a screen as shown below:

There are two main ways to install PHP for Windows: either manually or by using the installer. Once you have PHP installed on your Windows system, you may also want to load various extensions for added functionality.

Try the PHP 5.1.5 installer downloaded. It will install PHP into C:\PHP. You may see a screen to choose the Server Type from a number of options, which include web servers like PWS, IIS, Apache and Xitami.

Unfortunately, I was asked to configure the web server manually, when opted for Apache. Let us do that!

Unzip the downloaded PHP 5.1.5 zip file into the C:\PHP folder. Open the file “php.ini-recommended” and save it as “php.ini” to your Windows folder (e.g., C:\Windows). Copy “php5ts.dll” to any path mentioned in PATH environment variable (e.g., C:\Windows\System32).

At his point of time, Apache does not recognize PHP pages. If you load the test page now in your browser, you will get a screen similar to the following, since Apache does not get the help of PHP to interpret the page:

Now modify Apache configuration file httpd to reflect the following lines:

 LoadModule php5_module “c:/php/php5apache2.dll”

AddType application/x-httpd-php .php

Restart Apache web server. Test the page “test.php” with your browser. If the installations are correct, you may get a screen similar to the following:

Scroll down the page. There is much information available on the PHP setup on Apache. Note that we have now a database engine called “sqlite” as part of the PHP.

Next install the full fledged database MySQL 5.0.18. The installation will complete the service configuration as well, so that Windows will start MySQL as a service. Once the installation is over, you may try the command line client. It will appear as shown below:

Again, configure PHP to talk to MySQL. Open php.ini (in C:\WINDOWS) and make the following modifications:

1. Find “extension_dir” and modify it to show the directory in which the loadable extensions (modules) reside.

extension_dir = “c:\php\ext”

2. Locate Windows extensions and uncomment the line which reads:

                        Extension = php_mysql.dll

3. In order to use PHP’s session functions, you need to modify session save path as shown here:

session.save_path = “c:\windows\temp”

Finally, add C:\PHP as a path in Windows system environment variable PATH. Otherwise, while Apache tries to load php-mysql.dll, it may show an error warning like this:

“PHP Startup: Unable to load dynamic library ‘php_mysql.dll’ –

The specified module could not be found.”

If everything is done right, if you again test the page “test.php” in your browser, you will see screens with MySQL variables defined as shown below:

On Partioning tables in SQL Server 2005, here is a good article.

Database Specialists – Presentations and White Papers

Oracle Topics  

Jeff Hunter’s Oracle DBA Tips

Oracle Berkeley DB

Oracle Commands

Oracle Covering today’s Oracle topics

Oracle Database Online Documentation 10g Release 2 (10.2)

Oracle FAQ Quizzes

Oracle FusionZone Business Intelligence

Oracle performance tuning

Oracle Database Journal Articles

Oracle SQL Quick Reference

Oracle Technology Network 

ORACLE-BASE – Upgrading to Oracle Database 10g

OraFusion.com

OraPerf.com – The Oracle Instance Performance Portal

PUSCHITZ.COM – Oracle Linux, Linux Security, Linux Tips