Author: <email address hidden>
Revision Date: 2015-02-12 19:28:52 UTC

fixes bug: https://launchpad.net/bugs/1413044

Merge from 2.2:

Deleted line that does 'rm' on existing .qp and .xbcrypt files when using
inobackupex --decrypt and/or --decompress.

Also needs doc updates that if someone is using innobackupex with
--decompress and/or --decrypt, they will need to manually verify
that the result is correct (check to see if xtrabackup_checkpoints
is a readable file that makes sense) and manually delete or move
their original .qp and .xbcrypt files before performing a prepare
and --copy_back.

Author: <email address hidden>
Revision Date: 2015-02-12 19:22:02 UTC

fixes bug: https://launchpad.net/bugs/1413044

Deleted line that does 'rm' on existing .qp and .xbcrypt files when using
inobackupex --decrypt and/or --decompress.

Also needs doc updates that if someone is using innobackupex with
--decompress and/or --decrypt, they will need to manually verify
that the result is correct (check to see if xtrabackup_checkpoints
is a readable file that makes sense) and manually delete or move
their original .qp and .xbcrypt files before performing a prepare
and --copy_back.

Author: <email address hidden>
Revision Date: 2015-01-17 19:33:17 UTC

Reverted fix for 1070856 for custom build for i47576.

Author: George Ormond Lorch III
Revision Date: 2014-05-22 23:51:22 UTC

Fixed server options and my.cnf setting and generation that would cause mysqld to get started with defaults and no special options.

Author: George Ormond Lorch III
Revision Date: 2014-04-29 22:05:12 UTC

bug 1313901 - Binlog events with binlog_format=MIXED are unconditionally logged in ROW format

Temporary commit that removes legacy upstream fix and introduces extra restrictions on STATEMENT replication for queries that make use of temporary tables.

Author: George Ormond Lorch III
Revision Date: 2014-04-23 20:08:59 UTC

GOL - checkpoint:
 - xbcrypt v3 backwards compatibility test.
 - public/private key generation in xbcrypt.
 - encryption refactor and public key/hybrid implementation in xbcrypt.
 - decryption refactor and public key/hybrid implementation in xbcrypt.
 - implemented encryption within ds_encrypt.

Author: George Ormond Lorch III
Revision Date: 2014-03-20 22:44:43 UTC

Author: George Ormond Lorch III
Revision Date: 2014-01-27 23:22:01 UTC

Fix for bug1273465 - lintian warnings and errors (new packaging)
Corrected minor spelling errors with the exception of 'yuR' as that does not appear anywhere in the code base.

Author: George Ormond Lorch III
Revision Date: 2013-11-15 01:21:41 UTC

Fix for bug 1243009 --debug and --valgrind options needed for XtraBackup's build-binary.sh

util/build-binary.sh:
  Added new options --debug, --relwithdebinfo and --valgrind. These will set and export the DEBUG, RELWITHDEBINFO and VALGRIND variables to 'yes' before calling build.sh if specified. Even though they are mutually exclusive, there is no explicit test for this case.

util/build.sh:
  Added testing for RELWITHDEBINFO and VALGRIND variables and set up build/make options necessary for each case.

test/run.sh and test/inc/common.sh:
  Added -v N option to run.sh to allow enabling of valgrind when using test suite. N specifies a value that will be bassed to valgrinds --error-exitcode option. Made changes to accomplish this based on Alexey Kopytovs suggestions.
  Moved innobackupex_no_defaults_file into inc/common.sh from t/ib_doublewrite.sh since the concept is used in a few tests and correct fixup of IB_ARGS[@] is necessary in order to preserve args and valgrind options and makes a couple test cases a little simpler.

Fixed up several test cases that use/depend on $IB_ARGS to use "${IB_ARGS[@]}".

Author: George Ormond Lorch III
Revision Date: 2013-11-15 01:00:41 UTC

Fix for bug 1243009 --debug and --valgrind options needed for XtraBackup's build-binary.sh

util/build-binary.sh:
  Added new options --debug, --relwithdebinfo and --valgrind. These will set and export the DEBUG, RELWITHDEBINFO and VALGRIND variables to 'yes' before calling build.sh if specified. Even though they are mutually exclusive, there is no explicit test for this case.

util/build.sh:
  Added testing for RELWITHDEBINFO and VALGRIND variables and set up build/make options necessary for each case.

test/run.sh and test/inc/common.sh:
  Added -v N option to run.sh to allow enabling of valgrind when using test suite. N specifies a value that will be bassed to valgrinds --error-exitcode option. Made changes to accomplish this based on Alexey Kopytovs suggestions.
  Moved innobackupex_no_defaults_file into inc/common.sh from t/ib_doublewrite.sh since the concept is used in a few tests and correct fixup of IB_ARGS[@] is necessary in order to preserve args and valgrind options and makes a couple test cases a little simpler.

Fixed up several test cases that use/depend on $IB_ARGS to use "${IB_ARGS[@]}".

Author: George Ormond Lorch III
Revision Date: 2013-11-14 23:40:24 UTC

Fix for bug 1243009 --debug and --valgrind options needed for XtraBackup's build-binary.sh

util/build-binary.sh:
  Added new options --debug, --relwithdebinfo and --valgrind. These will set and export the DEBUG, RELWITHDEBINFO and VALGRIND variables to 'yes' before calling build.sh if specified. Even though they are mutually exclusive, there is no explicit test for this case.

util/build.sh:
  Added testing for RELWITHDEBINFO and VALGRIND variables and set up build/make options necessary for each case.

test/run.sh and test/inc/common.sh:
  Added -v N option to run.sh to allow enabling of valgrind when using test suite. N specifies a value that will be bassed to valgrinds --error-exitcode option. Made changes to accomplish this based on Alexey Kopytovs suggestions.
  Moved innobackupex_no_defaults_file into inc/common.sh from t/ib_doublewrite.sh since the concept is used in a few tests and correct fixup of IB_ARGS[@] is necessary in order to preserve args and valgrind options and makes a couple test cases a little simpler.

Fixed up several test cases that use/depend on $IB_ARGS to use "${IB_ARGS[@]}".

Author: George Ormond Lorch III
Revision Date: 2013-09-10 02:23:34 UTC

Implementation of xtrabackup http://blueprints.launchpad.net/percona-xtrabackup/+spec/history-on-the-server and test case.

Author: George Ormond Lorch III
Revision Date: 2013-07-17 17:04:46 UTC

Added extra logging to xtrabackup.cc surrpunding the suspend file and changed suspend file contents to include the current pid. Added suspend file validation in innobackupex to ensure that the pid read from the suspend file matches the known pid of the xtrabackup process to help identify mixed/crossed running backup instances.

Author: George Ormond Lorch III
Revision Date: 2013-07-12 19:41:46 UTC

Added new environment variables to control build:
  MAKE_ONLY - Set this to 1 to only re 'make' the server and xtrabackup. Nothing will be downloaded, unpacked or patched. This is not advised if any compiler or other flags are being changed from previous builds.
  RELWITHDEBINFO - Set this to 1 to eliminate optimizations by changing C/CXXFLAGS to -g -O0 and pass CMAKE_BUILD_TYPE=RelWithDebInfo to PS or MySQL 5.5+ cmake.
  WITH_VALGRIND - Set this to 1 to perform Valgrind compatible build. Eliminates optimizations just like RELWITHDEBINFO and also passes WITH_VALGRIND=ON to PS or MySQL 5.5+ cmake.
Added innobackupex option --use-valgrind which will prefix backup and apply-log calls to xtrabackup with "valgrind".
The valgrind binary must be within path and valgrind options may be set by either using the ~/.valgrindrc file or setting the environment variable VALGRIND_OPTS.

Author: George Ormond Lorch III
Revision Date: 2013-06-06 05:47:05 UTC

Fixed issue in testrun.c where specifying testsuite would find the tests in the corredt directory but would still try to execute them out of ./t

Added test status tracking and summary report at end of test run cycle to make it wasy to see what passed, skipped and failed at a glance, similar to MTR.

Changed testrun.sh slightly to return same code out of testrun.sh as the test that was executed when only a single test was run.

Author: George Ormond Lorch III
Revision Date: 2013-06-05 21:01:39 UTC

Fixed issue in testrun.c where specifying testsuite would find the tests in the corredt directory but would still try to execute them out of ./t

Added test status tracking and summary report at end of test run cycle to make it wasy to see what passed, skipped and failed at a glance, similar to MTR.

Changed testrun.sh slightly to return same code out of testrun.sh as the test that was executed when only a single test was run.

Author: George Ormond Lorch III
Revision Date: 2013-05-29 22:14:12 UTC

Added new environment variables to control build:
  MAKE_ONLY - Set this to 1 to only re 'make' the server and xtrabackup. Nothing will be downloaded, unpacked or patched. This is not advised if any compiler or other flags are being changed from previous builds.
  RELWITHDEBINFO - Set this to 1 to eliminate optimizations by changing C/CXXFLAGS to -g -O0 and pass CMAKE_BUILD_TYPE=RelWithDebInfo to PS or MySQL 5.5+ cmake.
  WITH_VALGRIND - Set this to 1 to perform Valgrind compatible build. Eliminates optimizations just like RELWITHDEBINFO and also passes WITH_VALGRIND=ON to PS or MySQL 5.5+ cmake.
Added innobackupex option --use-valgrind which will prefix backup and apply-log calls to xtrabackup with "valgrind".
The valgrind binary must be within path and valgrind options may be set by either using the ~/.valgrindrc file or setting the environment variable VALGRIND_OPTS.

Author: root
Revision Date: 2013-05-10 06:25:38 UTC

Finished work on defaults-file. Added ability to specify server and xtrabackup options matrix via options.

Author: George Ormond Lorch III
Revision Date: 2013-04-30 22:07:23 UTC

This implements the io-block-size option for XtraBackup and innobackupex (just for passthrough) which specifies number of pages to read/write per cycle.
Changes made:
- Increased the default from 64 pages to 256 pages (4M).
- Changed the ds_buffer size when chained behind compression to (UNIV_PAGE_SIZE * (xtrabackup_iosize + 1)), the one extra page is for any additional formatting overhead that would normally be seen in compression and xbstream.
- Corrected the retry logic in fil_cur.cc so that an individual page will be re-read up to a pre-determined number of times and for each validation failure, only that page will be re-read. Previous logic would re-read the entire block and continue counting retry failures even across failed, then successfully read pages.
- Created very simple test case that sets io-block-size to a larger value.
- Corrupt page reads are tested as part of bug766033.sh which creates a corrupted dataset and looks for the error message.
- Not tested as part of any test is a page that has been read and failed validation, then re-read and passed.

Author: George Ormond Lorch III
Revision Date: 2013-04-30 21:43:19 UTC

This implements the io-block-size option for XtraBackup and innobackupex (just for passthrough) which specifies number of pages to read/write per cycle.
Changes made:
- Increased the default from 64 pages to 256 pages (4M).
- Corrected the retry logic in xtrabackup_copy_datafile so that an individual page will be re-read up to a pre-determined number of times and for each validation failure, only that page will be re-read. Previous logic would re-read the entire block and continue counting retry failures even across failed and then successfully read pages.
- Created very simple test case that sets io-block-size to a larger value.
- Corrupt page reads are tested as part of bug766033.sh which creates a corrupted dataset and looks for the error message.
- Not tested as part of any test is a page that has been read and failed validation, then re-read and passed.

Author: George Ormond Lorch III
Revision Date: 2013-03-16 02:51:56 UTC

Fixed a lot of issues and cleaned up 16274

Author: George Ormond Lorch III
Revision Date: 2013-02-20 22:57:02 UTC

Introducing xtrabackup with encryption.

Encryption will be done through the libgcrypt library which can be found documented here: http://www.gnupg.org/documentation/manuals/gcrypt

Addition of libgcrypt requires now that libgcrypt and libgpgerror packages be installed.

Many package repositories still contain older versions of libgcrypt. The current stable version of 1.5.0 is reccomended and will take advantage of the AES-NI instruction set if available.

Similar to compression, encryption is not supported on streamed tar backups.

New options:
  --encrypt=<algorithm> : algorithm may be 'NONE', 'AES128', 'AES192' or 'AES256'. 'NONE' is used mainly for testing and is simply pass-through. Please refer to the libgcrypt manual for more information on these ciphers.

  --encrypt_key=<key> : a proper length encryption key to use. It is not reccomended to use this option where there is uncontrolled access to the machine as the command line and thus the key can be viewed as part of the process info. See option --encrypt_key_file.

  --encrypt_key_file=<keyfile> : the name of a file where the raw key of the appropriate length can be read from. The file must be a simple binary (or text) file that contains exactly the key to be used.

  --encrypt_threads=<threadcount> : number of threads used to encrypt in parallel (default=1).

  --encrypt_chunk_size=<size> : size (in bytes) of the working encryption buffer size for each encryption thread (default=64K).

  --compress_chunk_size=<size> : size (in bytes) of the working compression buffer size for each encryption thread (default=64K).

encrypt_key and encrypt_key_file are mutually exclusive. If --encrypt is specified, one of these MUST be specified else an error will occur.

encryption key lengths must be precise:
AES128=128 bits or 16 bytes
AES192=192 bits or 24 bytes
AES256=256 bits or 32 bytes

When using the encrypt_key_file option, the file specified may have binary or textual content but must be _exactly_ the correct size for the algorithm being used. If using text, caution must be taken so that there are no extra spaces, tabs, carriage returns or line feeds within the file. These will cause the key to be read as an incorrect size and an error will be generated.

The coding task was fairly straight forward with some restructure/refactoring approved by Alexey K:
  - Update/set copyright notice in new and touched files to Copyright (c) 2009-2013 Percona Ireland Ltd.
  - Implemented new ds_stdout data sink.
  - Split the ds_stream data sink into two new, more specific data sinks, ds_archive and ds_xbstream.
  - Implement write callback model for ds_xbstream similar to libarchive.
  - Change ds_archive and ds_xbstream over to using callback write models.
  - Add new option to compression --compress_chunk_size to allow user specified chunk size management.
  - Implement xbcrypt format reader/writer. Format encapsulated as follows:
      8 bytes - magic string "XBCRYP01"
      8 bytes - reserved
      8 bytes - original size
      8 bytes - encrypted size
      4 bytes - checksum
      'encrypted size' bytes - encrypted data
  - Implement a new ds_encrypt datasync modeled after the existing compression datasync which will use libgcrypt for actual encryption task and write callbacks.
  - Add new options, options validations and pass through innobackupex.
  - Implement new utility xbcrypt modeled after xbstream to perform encryption outside of xtrabackup for metadata and to offer a means of decrypting an encrypted backup.
  - Implement new tests and refactor test script structure a little to make adding new, similar test cases a little easier.

Author: George Ormond Lorch III
Revision Date: 2012-12-04 01:08:30 UTC

Incorporated range read prefetch logic from:
  http://www.zarafa.com/wiki/index.php/MySQL_5.5_InnoDB_prefetch_patch

Author: George Ormond Lorch III
Revision Date: 2012-11-27 04:47:35 UTC

Minor fix for lp 1079596:
Fixed unsigned math error in fsp_reserve_free_extents that in some
specific cases (when FSP_SIZE < FSP_FREE_LIMIT) causes the function to
believe that billions more extents have been reserved than have actually
been reserved.

Author: George Ormond Lorch III
Revision Date: 2012-11-26 20:56:28 UTC

Incorporated selective code into PS 5.5.25a sql/log_event.h and sql/log_event.cc from PS 5.5 revno 288 and bug fix from lp:~gl-az/percona-server/5.5-bug1053342.

Author: George Ormond Lorch III
Revision Date: 2012-10-12 17:40:01 UTC

zarafa

Author: George Lorch
Revision Date: 2012-08-20 21:04:17 UTC

.

Author: George Lorch
Revision Date: 2012-06-05 04:38:09 UTC

In order to offer more control over option visibility, access and range
  limits, the following new option modifiers have been added:
    minimum-<option name>=<value> : Sets the minimum value that clients
      may set <option name> to. If a client does attempt to set the value
      of <option name> lesser than <value>, a warning will be issued and
      the value of <option name> will be set to <value>.
    hidden-<option name>=<TRUE/FALSE> : If set to TRUE, makes any system
      variable named in <option name> invisible and immutable to clients.
    readonly-<option name>=<TRUE/FALSE> : If set to TRUE, makes any
      system variable named in <option name> immutable to clients.

Author: George Lorch
Revision Date: 2012-05-22 02:03:19 UTC

Initial options modifiers parsing/setup code. No implementation for enforcement yet.