Ubuntu Apps Directory

Merge lp://staging/~elachuni/ubuntu-webcatalog/celery into lp://staging/ubuntu-webcatalog

celery
Merge into trunk

Proposed by Anthony Lenton on 2012-06-27

Status:	Merged
Approved by:	Anthony Lenton on 2012-07-02
Approved revision:	157
Merged at revision:	156
Proposed branch:	lp://staging/~elachuni/ubuntu-webcatalog/celery
Merge into:	lp://staging/ubuntu-webcatalog
Diff against target:	1262 lines (+551/-136) 35 files modified django_project/config/main.cfg (+10/-1) django_project/settings.py (+4/-1) setup.py (+2/-0) src/webcatalog/admin.py (+2/-4) src/webcatalog/decorators.py (+32/-0) src/webcatalog/department_filters.py (+1/-4) src/webcatalog/forms.py (+1/-4) src/webcatalog/management/commands/cleanup.py (+24/-16) src/webcatalog/management/commands/import_all_app_install_data.py (+3/-5) src/webcatalog/management/commands/import_all_ratings_stats.py (+3/-5) src/webcatalog/management/commands/import_app_install_data.py (+1/-4) src/webcatalog/management/commands/import_sca_apps.py (+1/-4) src/webcatalog/managers.py (+1/-5) src/webcatalog/models/applications.py (+25/-5) src/webcatalog/preflight.py (+1/-4) src/webcatalog/schema.py (+11/-0) src/webcatalog/static/css/webcatalog.css (+1/-1) src/webcatalog/tasks.py (+54/-0) src/webcatalog/templates/forbidden.html (+23/-0) src/webcatalog/templates/webcatalog/task_list.html (+84/-0) src/webcatalog/templatetags/webcatalog.py (+1/-4) src/webcatalog/tests/__init__.py (+1/-0) src/webcatalog/tests/factory.py (+27/-6) src/webcatalog/tests/helpers.py (+1/-4) src/webcatalog/tests/test_commands.py (+4/-6) src/webcatalog/tests/test_department_filters.py (+1/-5) src/webcatalog/tests/test_forms.py (+1/-4) src/webcatalog/tests/test_managers.py (+1/-4) src/webcatalog/tests/test_models.py (+1/-4) src/webcatalog/tests/test_preflight.py (+3/-16) src/webcatalog/tests/test_tasks.py (+42/-0) src/webcatalog/tests/test_templatetags.py (+1/-5) src/webcatalog/tests/test_views.py (+116/-4) src/webcatalog/urls.py (+6/-4) src/webcatalog/views.py (+61/-7)
To merge this branch:	bzr merge lp://staging/~elachuni/ubuntu-webcatalog/celery
Related bugs:	Link a bug report

Reviewer	Review Type	Date Requested	Status
Danny Tamez (community)		2012-06-27	Approve on 2012-07-02
Review via email: mp+112398@code.staging.launchpad.net

Commit message

Added a view to schedule asynchronous tasks via Celery.

Description of the change

Overview
========
This branch adds a view to schedule asynchronous tasks via Celery. Only one basic task is provided to import exhibits (plus one that always fails, for testing), the full set of tasks will be in a separate MP, now that the groundwork is in place. A brief video demo can be found on http://people.canonical.com/~anthony/celery-uwc.ogv

Details
=======
The current branch counts on using CELERY_RESULT_BACKEND='database', as it fetches the result for each task from the DB. Any BROKER_BACKEND can be used, I specifically tested with djkombu and amqp locally.

The TaskMeta model was registered in the admin so that results can be browsed and searched in the admin UI after a task has run. The ModelAdmin class was improved a bit after grabbing the video, you can see the final list view on http://people.canonical.com/~anthony/TaskMetaAdmin.png

The list of tasks available is put together by introspecting webcatalog.tasks, so adding new tasks available will be picked up and added automatically.

The task list template uses YUI.io to retrieve the task's status, but submits a POST form and reloads the whole page when you schedule a new task; I considered doing everything with ajax, but POST via ajax is a bit tricky, specially when csrf tokens are involved, so I went for this simpler solution that seems to be responsive enough.

Revision history for this message

James Westby (james-w) wrote on 2012-06-27:

30 -openid_launchpad_staff_teams = canonical-ca-hackers
31 +openid_launchpad_staff_teams = canonical-isd-hackers

That looks backwards?

332 + if (message.hasClass('success')) {

That looks as if it will loop forever on successful tasks?

Most of this branch looks pretty good to me.

Thanks,

James

lp://staging/~elachuni/ubuntu-webcatalog/celery updated on 2012-06-29

152. By Anthony Lenton on 2012-06-28: Added a few tasks.
153. By Anthony Lenton on 2012-06-28: Merged in latest changes from trunk.
154. By Anthony Lenton on 2012-06-29: Query TaskState model instead of TaskMeta.

Revision history for this message

Anthony Lenton (elachuni) wrote on 2012-07-02:

> 30 -openid_launchpad_staff_teams = canonical-ca-hackers
> 31 +openid_launchpad_staff_teams = canonical-isd-hackers
>
> That looks backwards?

True, for some reason staging sso *still* doesn't report me as being part of canonical-ca-hackers, so I use canonical-isd-hackers instead during dev, I've put it back now, thanks!

> 332 + if (message.hasClass('success')) {
>
> That looks as if it will loop forever on successful tasks?

Hm it shouldn't, that call is outside of check_status so it's just to trigger the first call. Maybe moving this into an function called init() or onload() would make it clearer.

lp://staging/~elachuni/ubuntu-webcatalog/celery updated on 2012-07-02

155. By Anthony Lenton on 2012-07-02: Pulled back config change per code review.
156. By Anthony Lenton on 2012-07-02: Merged in latest changes from trunk. Made all tests pass.

Revision history for this message

Danny Tamez (zematynnad) wrote on 2012-07-02:

line 111 - we don't need this any longer as we're now on 2.6
looks like line 115 is not needed
at 158, that looks like that could have stayed static but I guess it makes more sense to make it a a regular method as that's how the class will now be used...
same for 183

screencast was great! Is there any way to have something more useful than the task id to help find the task you want to look at? I'm not sure of the use cases for searching for a task so that id may be the only way that makes sense...

Approved - feel free to act on or ignore the comments.

review: Approve

lp://staging/~elachuni/ubuntu-webcatalog/celery updated on 2012-07-02