Merge lp://staging/~michael.nelson/launchpad/588288-log-parser-dont-read-entire-file into lp://staging/launchpad

Proposed by Michael Nelson
Status: Merged
Approved by: Graham Binns
Approved revision: no longer in the source branch.
Merged at revision: 10942
Proposed branch: lp://staging/~michael.nelson/launchpad/588288-log-parser-dont-read-entire-file
Merge into: lp://staging/launchpad
Diff against target: 167 lines (+67/-13)
4 files modified
configs/testrunner/launchpad-lazr.conf (+1/-0)
lib/canonical/config/schema-lazr.conf (+4/-0)
lib/lp/services/apachelogparser/base.py (+30/-11)
lib/lp/services/apachelogparser/tests/test_apachelogparser.py (+32/-2)
To merge this branch: bzr merge lp://staging/~michael.nelson/launchpad/588288-log-parser-dont-read-entire-file
Reviewer Review Type Date Requested Status
Graham Binns (community) code Approve
Review via email: mp+26498@code.staging.launchpad.net

Commit message

log parser please don't read entire file into memory.

Description of the change

This branch fixes bug 588288, which was identified after running the ppa log parser for the first time (bug 139855).

It does two things:
1) Refactors the parse_file() function so that it does not read the entire remaining file contents to be parsed into memory (lines = fd.readlines()).

2) Allows an optional config logparser_max_parsed_lines to determine the maximum number of lines that should be parsed in one run.

No lint. To test:
bin/test -vvm test_apachelogparser

To post a comment you must log in.
Revision history for this message
Graham Binns (gmb) :
review: Approve (code)
Revision history for this message
Graham Binns (gmb) :
review: Approve (code)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
The diff is not available at this time. You can reload the page or download it.