Comments for One Free Voice http://blog.onefreevoice.com a blog about databases and stuff Thu, 28 Aug 2008 22:56:48 +0000 http://wordpress.org/?v=2.6.1 Comment on Creating an Intermediate Replication Layer by Gregory Haase http://blog.onefreevoice.com/2008/07/15/creating_intermediate_slaves/#comment-671 Gregory Haase Mon, 28 Jul 2008 15:10:04 +0000 http://blog.onefreevoice.com/?p=103#comment-671 I made one final edit to this procedure. The <code>START SLAVE UNTIL</code> directive only causes the slave SQL thread to stop at the given bin-log and position. At this point, the slave IO thread is still running. The original script had a <code>RESET SLAVE</code> call immediately after <code>MASTER_POS_WAIT</code>. This resulted in the error: <pre lang="sql">ERROR 1198 (HY000) at line 6: This operation cannot be performed with a running slave; run STOP SLAVE first</pre> Inserting <code>STOP SLAVE</code> before <code>RESET SLAVE</code> resolves the issue. I made one final edit to this procedure. The START SLAVE UNTIL directive only causes the slave SQL thread to stop at the given bin-log and position. At this point, the slave IO thread is still running. The original script had a RESET SLAVE call immediately after MASTER_POS_WAIT. This resulted in the error:

ERROR 1198 (HY000) at line 6: This operation cannot be performed WITH a running slave; run STOP SLAVE first

Inserting STOP SLAVE before RESET SLAVE resolves the issue.

]]> Comment on How to Keep Binlogs in Sync? by Gregory Haase http://blog.onefreevoice.com/2008/06/24/how-to-keep-binlogs-in-sync/#comment-649 Gregory Haase Tue, 15 Jul 2008 19:52:33 +0000 http://blog.onefreevoice.com/?p=100#comment-649 I've edited this procedure to reflect the fact that START SLAVE UNTIL is asynchronous (see <a href="http://blog.onefreevoice.com/2008/07/15/creating_intermediate_slaves/#comment-647" rel="nofollow">comment</a> from Mats Kindahl). I’ve edited this procedure to reflect the fact that START SLAVE UNTIL is asynchronous (see comment from Mats Kindahl).

]]> Comment on Creating an Intermediate Replication Layer by Gregory Haase http://blog.onefreevoice.com/2008/07/15/creating_intermediate_slaves/#comment-648 Gregory Haase Tue, 15 Jul 2008 19:46:08 +0000 http://blog.onefreevoice.com/?p=103#comment-648 Thank you Mats for catching that one. I've edited the original thread to include MASTER_POS_WAIT as you suggested. Thank you Mats for catching that one. I’ve edited the original thread to include MASTER_POS_WAIT as you suggested.

]]> Comment on Creating an Intermediate Replication Layer by Mats Kindahl http://blog.onefreevoice.com/2008/07/15/creating_intermediate_slaves/#comment-647 Mats Kindahl Tue, 15 Jul 2008 19:22:18 +0000 http://blog.onefreevoice.com/?p=103#comment-647 Uhm... as I understand what you are trying to accomplish, there is a problem with following the procedure given. If the instructions are followed by the letter, the intermediate slave will be reset <em>immediately</em> and not when the slave reaches the until-position given in the <code>START SLAVE UNTIL</code>. This is because the <code>START SLAVE UNTIL</code> starts the slave asynchronously, i.e., will return immediately with the slave running. To wait for the slaves to reach the given position before resetting them, it is necessary to add a <code>SELECT MASTER_POS_WAIT('bin-logs.000025', 106)</code> after the <code>START SLAVE UNTIL</code> but before the <code>RESET MASTER</code> and the <code>RESET SLAVE</code> respectively. Uhm… as I understand what you are trying to accomplish, there is a problem with following the procedure given. If the instructions are followed by the letter, the intermediate slave will be reset immediately and not when the slave reaches the until-position given in the START SLAVE UNTIL. This is because the START SLAVE UNTIL starts the slave asynchronously, i.e., will return immediately with the slave running. To wait for the slaves to reach the given position before resetting them, it is necessary to add a SELECT MASTER_POS_WAIT('bin-logs.000025', 106) after the START SLAVE UNTIL but before the RESET MASTER and the RESET SLAVE respectively.

]]> Comment on How to Keep Binlogs in Sync? by One Free Voice » Blog Archive » Creating an Intermediate Replication Layer http://blog.onefreevoice.com/2008/06/24/how-to-keep-binlogs-in-sync/#comment-646 One Free Voice » Blog Archive » Creating an Intermediate Replication Layer Tue, 15 Jul 2008 17:52:45 +0000 http://blog.onefreevoice.com/?p=100#comment-646 [...] few weeks ago, I discussed how to keep binlogs in sync in a tree or pyramid replication scheme. That thread discussed how to re-distribute load in case of [...] [...] few weeks ago, I discussed how to keep binlogs in sync in a tree or pyramid replication scheme. That thread discussed how to re-distribute load in case of [...]

]]> Comment on How to Keep Binlogs in Sync? by Gregory Haase http://blog.onefreevoice.com/2008/06/24/how-to-keep-binlogs-in-sync/#comment-634 Gregory Haase Thu, 26 Jun 2008 00:06:54 +0000 http://blog.onefreevoice.com/?p=100#comment-634 I revised the process outlined above to include Mark's suggestion about START SLAVE UNTIL. I also changed MASTER_LOG_FILE to RELAY_MASTER_LOG_FILE reflecting Baron's comment. RELAY_MASTER_LOG_FILE is the name of the master binary log file containing the most recent event executed by the SQL thread. It's companion EXEC_MASTER_LOG_POSITION is the position of the last event executed by the SQL thread from the master's binary log. See <a href="http://dev.mysql.com/doc/refman/5.1/en/show-slave-status.html" title="show slave status" rel="nofollow">Documentation</a>. I would note that in the case of one of my intermediate slaves going down, the SQL thread on the end slaves is not going to stop. It's going to process all the way through the relay logs. The I/O thread would keep trying to connect to the master every 60 seconds. If the slaves were guaranteed in sync, one could theoretically map the IP Address of the failed server to one of your working servers and never miss a beat. I revised the process outlined above to include Mark’s suggestion about START SLAVE UNTIL.

I also changed MASTER_LOG_FILE to RELAY_MASTER_LOG_FILE reflecting Baron’s comment.

RELAY_MASTER_LOG_FILE is the name of the master binary log file containing the most recent event executed by the SQL thread. It’s companion EXEC_MASTER_LOG_POSITION is the position of the last event executed by the SQL thread from the master’s binary log. See Documentation.

I would note that in the case of one of my intermediate slaves going down, the SQL thread on the end slaves is not going to stop. It’s going to process all the way through the relay logs. The I/O thread would keep trying to connect to the master every 60 seconds. If the slaves were guaranteed in sync, one could theoretically map the IP Address of the failed server to one of your working servers and never miss a beat.

]]> Comment on How to Keep Binlogs in Sync? by Xaprb http://blog.onefreevoice.com/2008/06/24/how-to-keep-binlogs-in-sync/#comment-633 Xaprb Wed, 25 Jun 2008 20:20:24 +0000 http://blog.onefreevoice.com/?p=100#comment-633 MASTER_LOG_FILE, EXEC_MASTER_LOG_POSITION is quicksand for the unwary :-) Master_log_file is the I/O thread's position, and you care about what updates have been applied, not which have been read from the master. I'm still working on automating this with mk-slave-move from Maatkit. You can already do it *before* the server crashes, with the current code. But after it crashes -- that's harder. Even while the servers are all functioning normally there are a lot of tricky cases to cover. MASTER_LOG_FILE, EXEC_MASTER_LOG_POSITION is quicksand for the unwary :-) Master_log_file is the I/O thread’s position, and you care about what updates have been applied, not which have been read from the master.

I’m still working on automating this with mk-slave-move from Maatkit. You can already do it *before* the server crashes, with the current code. But after it crashes — that’s harder. Even while the servers are all functioning normally there are a lot of tricky cases to cover.

]]> Comment on How to Keep Binlogs in Sync? by Gregory Haase http://blog.onefreevoice.com/2008/06/24/how-to-keep-binlogs-in-sync/#comment-632 Gregory Haase Wed, 25 Jun 2008 12:18:06 +0000 http://blog.onefreevoice.com/?p=100#comment-632 That's a good point. I keep forgetting about START SLAVE UNTIL. That’s a good point. I keep forgetting about START SLAVE UNTIL.

]]> Comment on How to Keep Binlogs in Sync? by Mark Leith http://blog.onefreevoice.com/2008/06/24/how-to-keep-binlogs-in-sync/#comment-631 Mark Leith Wed, 25 Jun 2008 10:12:52 +0000 http://blog.onefreevoice.com/?p=100#comment-631 Yes there is a chance that there could be a statement issued from the slave SQL thread between the MASTER_POS_WAIT and the RESET MASTER. You want to look in to STOP SLAVE ; START SLAVE UNTIL instead, this guarantees that no statements run after the point that you want to be at. This was discussed a little before on Baron's blog (and in the comments) here: http://www.xaprb.com/blog/2007/01/20/how-to-make-mysql-replication-reliable/ Yes there is a chance that there could be a statement issued from the slave SQL thread between the MASTER_POS_WAIT and the RESET MASTER.

You want to look in to STOP SLAVE ; START SLAVE UNTIL instead, this guarantees that no statements run after the point that you want to be at.

This was discussed a little before on Baron’s blog (and in the comments) here:

http://www.xaprb.com/blog/2007/01/20/how-to-make-mysql-replication-reliable/

]]> Comment on MySQL Event Scheduler - Is it Enterprise Ready? by Mark Robson http://blog.onefreevoice.com/2008/06/02/mysql-event-scheduler/#comment-623 Mark Robson Tue, 03 Jun 2008 08:54:34 +0000 http://blog.onefreevoice.com/?p=96#comment-623 I don't think you should be using "I can't figure out a way to correctly distribute my software" as a valid reason to use mysql scheduler. Distributing software correctly to all nodes in your service and having it only run on ones it needs to (i.e. only on master, not slaves) is a no-brainer and anyone can resolve it trivially. Your software build / testing system should show up any problems in your distribution / install system. Yes there are problems with MySQL scheduler. Yes, "cron" works. In practice it is quite easy to make cron jobs work correctly - for example, you can build your application into a set of RPMs (or other favourite package manager) which drop files into /etc/cron.d. Your cron jobs can check whether they're on a master or slave to decide what to do. All this can be tested in your development / testing environment (which clearly needs to have the same master/slave setup as production!). Come on - it's not rocket science. As regards spurious logging - mysql is not very good for this - the slave thread writes "[ERROR] Error reading relay log event: slave SQL thread was killed" when you shut it down administratively for a backup. Of course any sane person puts a trigger on their monitoring system which spots things like [ERROR] on mysqld.log and generates alerts ... :) Mark I don’t think you should be using “I can’t figure out a way to correctly distribute my software” as a valid reason to use mysql scheduler.

Distributing software correctly to all nodes in your service and having it only run on ones it needs to (i.e. only on master, not slaves) is a no-brainer and anyone can resolve it trivially.

Your software build / testing system should show up any problems in your distribution / install system.

Yes there are problems with MySQL scheduler. Yes, “cron” works. In practice it is quite easy to make cron jobs work correctly - for example, you can build your application into a set of RPMs (or other favourite package manager) which drop files into /etc/cron.d. Your cron jobs can check whether they’re on a master or slave to decide what to do. All this can be tested in your development / testing environment (which clearly needs to have the same master/slave setup as production!).

Come on - it’s not rocket science.

As regards spurious logging - mysql is not very good for this - the slave thread writes “[ERROR] Error reading relay log event: slave SQL thread was killed” when you shut it down administratively for a backup.

Of course any sane person puts a trigger on their monitoring system which spots things like [ERROR] on mysqld.log and generates alerts … :)

Mark

]]>