Merge lp://staging/~jderose/filestore/dbase32 into lp://staging/filestore

Proposed by Jason Gerard DeRose
Status: Merged
Merged at revision: 304
Proposed branch: lp://staging/~jderose/filestore/dbase32
Merge into: lp://staging/filestore
Diff against target: 3659 lines (+1377/-1583)
12 files modified
debian/control (+2/-2)
filestore/__init__.py (+41/-58)
filestore/data/MD5SUMS.json (+23/-0)
filestore/data/V0.json (+24/-0)
filestore/data/V1.json (+24/-0)
filestore/data/test-vectors.json (+0/-50)
filestore/misc.py (+52/-32)
filestore/protocols.py (+68/-164)
filestore/tests/__init__.py (+67/-180)
filestore/tests/test_misc.py (+132/-54)
filestore/tests/test_protocols.py (+943/-1042)
setup.py (+1/-1)
To merge this branch: bzr merge lp://staging/~jderose/filestore/dbase32
Reviewer Review Type Date Requested Status
David Jordan Approve
Review via email: mp+149313@code.staging.launchpad.net

Description of the change

For details, see this bug:

https://bugs.launchpad.net/filestore/+bug/1130156

Changes include:

* Use compatibility functions from `dbase32.rfc3548` so that it will be very easy to switch to Dbase32, in particular use `isb32()` instead of set operations with B32ALPHABET; we can just replace with `isdb32()` when needed

* Removed protocol selection-by-ID-length functionality, for now; this was incomplete and as we have a demanding migration ahead of us, I backed this feature out so we have less complexity to deal with during the migration

* `Protocol.hash_root()` now returns a binary digest instead of a Base32 encoded ID, and `filestore.protocols` is now completely encoding agnostic, does nothing with encoding itself

* `Protocol.hash_leaf()` now transforms *leaf_index* into the key bytes rather than relying on `Protocol._hash_leaf()` to do this

* Likewise, `Protocol.hash_root()` now transforms *file_size* into the key bytes rather than relying on `Protocol._hash_root()` to do this

* `Protocol` is now a base class without any hashing implementation itself; real implementations are now in the `SkeinProtocol` and `OldSkeinProtocol` subclasses

* Changed V1 to use a 240-bit digest and DBase32 encoding, updated test vectors accordingly

* Split old test-vectors.json file (which contained both V0 and V1) into separate V0.json and V1.json files

* Added MD5SUMS.json with md5sums of various values used by the test vectors; these are helpful for debugging an implementation, as mention in the specification

* Huge refactor of the unit tests for `filestore.protocols`; in particular, no digest values are in the unit tests anymore; instead, digests come from the JSON data files only, making it much easier to tweak a protocol without having to make huge changes in the unit tests

* misc.build_vectors() now takes optional *encoder* kwarg, which defaults to `db32enc`, so the test vector building is no longer hard-coded to a specific encoding

To post a comment you must log in.
Revision history for this message
Jason Gerard DeRose (jderose) wrote :

As this diff is pretty sprawling and noisy, you can probably understand the key changes more easily by just looking at filestore/protocols.py:

http://bazaar.launchpad.net/~jderose/filestore/dbase32/view/head:/filestore/protocols.py

Revision history for this message
David Jordan (dmj726) wrote :

Approve.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
The diff is not available at this time. You can reload the page or download it.

Subscribers

People subscribed via source and target branches