canal指定binlog位点启动

canal指定binlog位点启动

1. canal基于集群模式搭建,所用版本为带有可视化页面的canal-1.1.4,包结构如下
canal-adapter
canal-admin
canal-server
2.抽取实例主要配置canal-server,包结构如下
cd canal-server/
bin
conf
lib
logs
#conf目录
[canal-server]$ cd conf/
canal_local.properties
canal.properties
example
logback.xml
metrics
spring
#spring目录
[conf]$ cd spring/
base-instance.xml
default-instance.xml
file-instance.xml
group-instance.xml
memory-instance.xml
tsdb
3.集群模式下每一个抽取实例所记录的binlog位点信息存在与zookeeper中,使用的配置文件为default-instance.xml
a.canal.properties
canal.instance.global.spring.xml = classpath:spring/default-instance.xml
b.instance.properties 中binlog位点信息配置如下
# position info
canal.instance.master.address=127.0.0.1:3306
# binlog日志文件名
canal.instance.master.journal.name=
# binlog具体位点
canal.instance.master.position=
# binlog指定时间戳
canal.instance.master.timestamp=
4.mysql中找到需要指定的日志位点信息
-- 1.查看mysql binlog 日志:
show binary logs ;
-- 2.查看某一binlog具体位点信息:
show binlog events in 'mysql-bin.000001';
-- 3.查看当前binlog最新位点:
show master status;
5.定位具体binlog位点有两种方式
a.通过name和position确定
# binlog日志文件名
canal.instance.master.journal.name='mysql-bin.000001'
# binlog具体位点
canal.instance.master.position=100
b.通过时间戳指定
# binlog指定时间戳
canal.instance.master.timestamp=1611072000000
c.当使用以上方式指定位点启动时需要删除zookeeper中保存的binlog位点信息(如canal为本地模式则操作具体实例下的meta.dat文件)
# 查看位点信息 example 为具体抽取实例的文件名
get /otter/canal/destinations/example/1001/cursor 
{"@type":"com.alibaba.otter.canal.protocol.position.LogPosition","identity":{"slaveId":-1,"sourceAddress":{"address":"127.0.0.1","port":3306}},"postion":{"gtid":"","included":false,"journalName":"mysql-bin.000001","position":1000,"serverId":1,"timestamp":1611082830000}}

# 删除位点信息
delete /otter/canal/destinations/example/1001/cursor
d.如果指定name与position启动,可以直接修改zookeeper中的位点信息无需修改instance.properties
set /otter/canal/destinations/example/1001/cursor  
{"@type":"com.alibaba.otter.canal.protocol.position.LogPosition","identity":{"slaveId":-1,"sourceAddress":{"address":"127.0.0.1","port":3306}},"postion":{"gtid":"","included":false,"journalName":"mysql-bin.000001","position":100,"serverId":1,"timestamp":1611082830000}}
6.需要删除zookeeper中保存的binlog位点信息(canal为本地模式则操作具体实例下的meta.dat文件)是因为canal实例在启动时先会查询上一次成功保存的位点信息,如果存在则不会读取instance.properties,当不存在时才会读取instance.properties中相关的binlog位点信息

版权声明:本文为weixin_44889538原创文章,遵循CC 4.0 BY-SA版权协议,转载请附上原文出处链接和本声明。