mysqlbinlog can not parse row based events

    最近的MHA测试过程中，碰到了mysqlbinlog客户端的版本低于服务端版本的问题。即这个错误提示：mysqlbinlog is 3.2 (included in MySQL Client 5.0 or lower), but MySQL server version is 5.6.22-log. mysqlbinlog can not parse row based events。这个应该是个比较常见的错误。主要是由于在安装Linux期间通常在自带安装mysql相关rpm包，后来又安装了高版本的mysql而引发的一些版本问题。下面是这个问题的主要描述。

1、故障现象
# masterha_check_repl --conf=/etc/app1.cnf
Mon Feb 16 20:46:12 2015 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Mon Feb 16 20:46:12 2015 - [info] Reading application default configuration from /etc/app1.cnf..
Mon Feb 16 20:46:12 2015 - [info] Reading server configuration from /etc/app1.cnf..
Mon Feb 16 20:46:12 2015 - [info] MHA::MasterMonitor version 0.56.
Mon Feb 16 20:46:13 2015 - [info] GTID failover mode = 0
Mon Feb 16 20:46:13 2015 - [info] Dead Servers:
Mon Feb 16 20:46:13 2015 - [info] Alive Servers:
Mon Feb 16 20:46:13 2015 - [info]   192.168.1.6(192.168.1.6:3306)
Mon Feb 16 20:46:13 2015 - [info]   192.168.1.7(192.168.1.7:3306)
Mon Feb 16 20:46:13 2015 - [info]   192.168.1.8(192.168.1.8:3306)
Mon Feb 16 20:46:13 2015 - [info] Alive Slaves:
Mon Feb 16 20:46:13 2015 - [info]   192.168.1.7(192.168.1.7:3306)  Version=5.6.22-log (oldest major version between slaves) log-bin:enabled
Mon Feb 16 20:46:13 2015 - [info]     Replicating from 192.168.1.6(192.168.1.6:3306)
Mon Feb 16 20:46:13 2015 - [info]     Primary candidate for the new Master (candidate_master is set)
Mon Feb 16 20:46:13 2015 - [info]   192.168.1.8(192.168.1.8:3306)  Version=5.6.22-log (oldest major version between slaves) log-bin:enabled
Mon Feb 16 20:46:13 2015 - [info]     Replicating from 192.168.1.6(192.168.1.6:3306)
Mon Feb 16 20:46:13 2015 - [info] Current Alive Master: 192.168.1.6(192.168.1.6:3306)
Mon Feb 16 20:46:13 2015 - [info] Checking slave configurations..
Mon Feb 16 20:46:13 2015 - [info]  read_only=1 is not set on slave 192.168.1.7(192.168.1.7:3306).
Mon Feb 16 20:46:13 2015 - [warning]  relay_log_purge=0 is not set on slave 192.168.1.7(192.168.1.7:3306).
Mon Feb 16 20:46:13 2015 - [info] Checking replication filtering settings..
Mon Feb 16 20:46:13 2015 - [info]  binlog_do_db= , binlog_ignore_db=
Mon Feb 16 20:46:13 2015 - [info]  Replication filtering check ok.
Mon Feb 16 20:46:13 2015 - [info] GTID (with auto-pos) is not supported
Mon Feb 16 20:46:13 2015 - [info] Starting SSH connection tests..
Mon Feb 16 20:46:14 2015 - [info] All SSH connection tests passed successfully.
Mon Feb 16 20:46:14 2015 - [info] Checking MHA Node version..
Mon Feb 16 20:46:15 2015 - [info]  Version check ok.
Mon Feb 16 20:46:15 2015 - [info] Checking SSH publickey authentication settings on the current master..
Mon Feb 16 20:46:15 2015 - [info] HealthCheck: SSH to 192.168.1.6 is reachable.
Mon Feb 16 20:46:16 2015 - [info] Master MHA Node version is 0.56.
Mon Feb 16 20:46:16 2015 - [info] Checking recovery script configurations on 192.168.1.6(192.168.1.6:3306)..
Mon Feb 16 20:46:16 2015 - [info]   Executing command: save_binary_logs --command=test --start_pos=4 --binlog_dir=/data/mysqldata
   --output_file=/var/tmp/save_binary_logs_test --manager_version=0.56 --start_file=mysql-bin.000010
Mon Feb 16 20:46:16 2015 - [info]   Connecting to [email protected](192.168.1.6:22)..
  Creating /var/tmp if not exists..    ok.
  Checking output directory is accessible or not..
   ok.
  Binlog found at /data/mysqldata, up to mysql-bin.000010
Mon Feb 16 20:46:16 2015 - [info] Binlog setting check done.
Mon Feb 16 20:46:16 2015 - [info] Checking SSH publickey authentication and checking recovery script configurations on all alive slave servers..
Mon Feb 16 20:46:16 2015 - [info]   Executing command : apply_diff_relay_logs --command=test --slave_user=‘mha‘ --slave_host=192.168.1.7
  --slave_ip=192.168.1.7 --slave_port=3306 --workdir=/var/tmp --target_version=5.6.22-log --manager_version=0.56
  --relay_log_info=/data/mysqldata/relay-log.info  --relay_dir=/data/mysqldata/  --slave_pass=xxx
Mon Feb 16 20:46:16 2015 - [info]   Connecting to [email protected](192.168.1.7:22)..
mysqlbinlog version is 3.2 (included in MySQL Client 5.0 or lower). This is not recommended. Consider upgrading MySQL Client to 5.1 or higher.
mysqlbinlog is 3.2 (included in MySQL Client 5.0 or lower), but MySQL server version is 5.6.22-log. mysqlbinlog can not parse row based events.
  Terminating script for safety reasons. at /usr/bin/apply_diff_relay_logs line 493
Mon Feb 16 20:46:16 2015 - [error][/usr/lib/perl5/site_perl/5.8.8/MHA/MasterMonitor.pm, ln412] Slaves settings check failed!
Mon Feb 16 20:46:16 2015 - [error][/usr/lib/perl5/site_perl/5.8.8/MHA/MasterMonitor.pm, ln275] Slave configuration failed.
Mon Feb 16 20:46:16 2015 - [error][/usr/lib/perl5/site_perl/5.8.8/MHA/MasterMonitor.pm, ln520] Error happened on checking configurations. 
  at /usr/bin/masterha_check_repl line 48
Mon Feb 16 20:46:16 2015 - [error][/usr/lib/perl5/site_perl/5.8.8/MHA/MasterMonitor.pm, ln691] Error happened on monitoring servers.
Mon Feb 16 20:46:16 2015 - [info] Got exit code 1 (Not master dead).

MySQL Replication Health is NOT OK!

2、故障分析
###查看当前的的mysqlbinlog
# which mysqlbinlog
/app/soft/mysql/bin/mysqlbinlog

###mysqbinlog的版本号，如下为3.4
# mysqlbinlog --version
mysqlbinlog Ver 3.4 for Linux at x86_64

###查看缺省路径下是否存在mysqlbinlog，如下查询的确存在
# ls -hltr /usr/bin/mysqlbinlog
-rwxr-xr-x 1 root root 97K 2013-01-23 /usr/bin/mysqlbinlog

###查看缺省mysqlbinlog的版本号，如下为3.2，与故障描述相符，版本过低。
# /usr/bin/mysqlbinlog --version
/usr/bin/mysqlbinlog Ver 3.2 for redhat-linux-gnu at x86_64

### Author : Leshami
### Blog   : http://blog.csdn.net/leshami

3、故障解决
###直接使用软链的方式来使得MHA调用的mysqlbinlog版本为3.4版
# mv /usr/bin/mysqlbinlog /usr/bin/mysqlbinlog.bk
# ln -s /app/soft/mysql/bin/mysqlbinlog /usr/bin/mysqlbinlog
# /usr/bin/mysqlbinlog --version
/usr/bin/mysqlbinlog Ver 3.4 for Linux at x86_64

###也可以卸载系统当前的mysql rpm包，如果缺省路径下无mysql实例的话，如下方式
# rpm -qa|grep mysql
mysql-5.0.95-5.el5_9
# rpm -e --nodeps mysql-5.0.95-5.el5_9
warning: /etc/my.cnf saved as /etc/my.cnf.rpmsave
# more /etc/my.cnf
/etc/my.cnf: No such file or directory

###注意，卸载缺省的rpm包之后，缺省的my.cnf被复制为/etc/my.cnf.rpmsave，需要复制回去。
# cp /etc/my.cnf.rpmsave /etc/my.cnf

4、延伸参考
   使用mysqlbinlog提取二进制日志
   MySQL抑制binlog日志中的BINLOG部分