Percona Toolkit moved to https://jira.percona.com/projects/PT

Merge lp://staging/~percona-toolkit-dev/percona-toolkit/pt-tc-likely-to-skip-table-when-row-count-is-around-chunk-size-times-chunk-size-limit-1389041 into lp://staging/~percona-toolkit-dev/percona-toolkit/release-2.2.14

  1. pt-tc-likely-to-skip-table-when-row-count-is-around-chunk-size-times-chunk-size-limit-1389041
  2. Merge into release-2.2.14
Proposed by Frank Cizmich
Status: Needs review
Proposed branch: lp://staging/~percona-toolkit-dev/percona-toolkit/pt-tc-likely-to-skip-table-when-row-count-is-around-chunk-size-times-chunk-size-limit-1389041
Merge into: lp://staging/~percona-toolkit-dev/percona-toolkit/release-2.2.14
Diff against target: 81 lines (+47/-1)
3 files modified
bin/pt-table-checksum (+1/-1)
t/pt-table-checksum/basics.t (+37/-0)
t/pt-table-checksum/samples/create_skip_table.sql (+9/-0)
To merge this branch: bzr merge lp://staging/~percona-toolkit-dev/percona-toolkit/pt-tc-likely-to-skip-table-when-row-count-is-around-chunk-size-times-chunk-size-limit-1389041
Reviewer Review Type Date Requested Status
Daniel Nichter Needs Information
Review via email: mp+254818@code.staging.launchpad.net

Description of the change

Problem:
pt-table-checksum often skips a table when its size is close to --chunk-size * --chunk-size-limit
This is because it checks if the slave table exceeds that size and decides it's too big to do in one chunk.
Since the size of the slave table is just an estimate of the EXPLAIN query, it is often the case that it exceeds the size of the master table by only a few rows. This leads to it skipping tables that could really be done in one chunk with no problem.

Solution:
Add a 20% tolerance to the size estimate.

Note: includes test

To post a comment you must log in.
Revision history for this message
Daniel Nichter (daniel-nichter) wrote :

Why not just increase --chunk-size-limit? There's always going to be "if only it was just N% bigger!" So this seems to just move the problem because what if the new border is --chunk-size * --chunk-size-limit * 1.2?

review: Needs Information
Reply
Revision history for this message
Frank Cizmich (frank-cizmich) wrote :

Daniel, at first glance it seems as you say, but look what is happening:

chunk-size-limit * chunk-size (without the extra 20%) has already been used earlier in the code to decide that the table will be done in one chunk ( see line 6469 )

The part where I added the 20% tolerance is when it is checking if the slave table can also be done in 1 chunk. This happens *after* it has been decided that the master table can do it.

( see line 9926 )

"if ( $nibble_iter->one_nibble() ) {" # if we're going to do it in one chunk ..
  ... check and be a bit more tolerant with slave here...

The net effect is that now a table will not be skipped for minor discrepancies.

Reply

Unmerged revisions

614. By Frank Cizmich

dont skip when slave rowcount barely higher than chunk-size*chunk-size-limit

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
The diff is not available at this time. You can reload the page or download it.

Subscribers

People subscribed via source and target branches

to all changes: