Archive

Monthly Archives: September 2014

Thanks to Bucardo team for responding my previous post. My cascaded slave replication works as expected.

Today I notice there is still something to do related with delta and track tables.
Single table replication scenario:
Db-A/Tbl-T1 (master) => Db-B/Tbl-T2 (slave) => Db-C/Tbl-T3 (cascaded slave)

Every change on Table T1 replicated to T2, then T2 to T3. After a while, VAC successfully cleans delta and track tables on Db-A. But not on Db-B.

I detect 2 issues:
1. If cascaded replication T2 to T3 successful, the delta table on Db-B is not be cleaned up by VAC.
2. If cascaded replication T2 to T3 failed before VAC schedule, the delta table on Db-B will be cleaned up by VAC. Then, cascaded replication from T2 to T3 losts.

I fix it by modifying SQL inside bucardo.bucardo_purge_delta(text, text) in Db-A and Db-B:

— Delete all txntimes from the delta table that:
— 1) Have been used by all dbgroups listed in bucardo_delta_targets
— 2) Have a matching txntime from the track table
— 3) Are older than the first argument interval
myst = 'DELETE FROM bucardo.'
|| deltatable
|| ' USING (SELECT track.txntime AS tt FROM bucardo.'
|| tracktable
|| ' track INNER JOIN bucardo.bucardo_delta_targets bdt ON track.target=bdt.target'
|| ' WHERE bdt.tablename::regclass::text='
|| quote_literal($2)

|| ' GROUP BY 1 HAVING COUNT(*) = '
|| drows
|| ') AS foo'
|| ' WHERE txntime = tt'
|| ' AND txntime < now() – interval '
|| quote_literal($1);

Need advice from Bucardo team.

Advertisements

When I read new release of Bucardo 5 with capability of asynchronous multi-master replication, I was eagered to wet my foot before swimming. Not really about multi-master features, just to implement simple master-slave-cascaded_slave replication, as I have done with Slony.

Scenario:
master table A => slave table B => cascaded slave table C.

So, I built Bucardo from source version 5.1.1 then configured databases, tables, syncs. I have turned on “makedelta” for slave table which is source for cascaded replication to another slave table. Master to slave (A to B) replication run well. But, unfortunately, the cascaded replication (B to C) does not work.

Then I dig into Bucardo.pm source code. In sub start_kid, the “does_makedelta hash” populated as follows:

sub start_kid {


for my $dbname (@dbs_source) {

$x = $sync->{db}{$dbname};

for my $g (@$goatlist) {


## Set the per database/per table makedelta setting now
if (defined $g->{makedelta}) {

if ($g->{makedelta} eq ‘on’ or $g->{makedelta} =~ /\b$dbname\b/) {

$x->{does_makedelta}{$S}{$T} = 1;

}

}

}

}

}

I suspect that “does_makedelta hash” reference was incorrectly taken from dbs_source. To my opinion, it should be taken from dbs_target. So, I move “does_makedelta hash” to another new dbs_target loop:

sub start_kid {


for my $dbname (@dbs_source) {

$x = $sync->{db}{$dbname};

for my $g (@$goatlist) {


## Set the per database/per table makedelta setting now
## if (defined $g->{makedelta}) {

## if ($g->{makedelta} eq ‘on’ or $g->{makedelta} =~ /\b$dbname\b/) {

## $x->{does_makedelta}{$S}{$T} = 1;
## …

## }

## }

}

}

for my $dbname (@dbs_target) {

$x = $sync->{db}{$dbname};
for my $g (@$goatlist) {

next if $g->{reltype} ne ‘table’;
($S,$T) = ($g->{safeschema},$g->{safetable});
## Set the per database/per table makedelta setting now
if (defined $g->{makedelta}) {

if ($g->{makedelta} eq ‘on’ or $g->{makedelta} =~ /\b$dbname\b/) {

$x->{does_makedelta}{$S}{$T} = 1;
$self->glog(“Set table $dbname.$S.$T to makedelta”, LOG_NORMAL);

}

}

}

}

}

I have not verified whether it was valid. But, at least, my cascaded replication works.