Merge lp://staging/~chipaca/snappy/lock-ness into lp://staging/~snappy-dev/snappy/snappy-moved-to-github

Thanks for this branch! It looks good (but I need to read through it again as its quite complex at first). I got a bit of a meta-question. It seems like all the locking is done in the rest-api which is fine as its the only concurrent user of the API right now. But it also means that if someone builds on top of the snappy api he/she will have to rebuild the same locking. Is it a huge amount of work to put the locks inside snappy/* itself or is there another downside that I have not considered?

Fwiw, I love the branch title :P

Remember the intention is to move away from having multiple things building against "the snappy api"; clients would use the rest api. There's a question about what that means for u-d-f, but locking shouldn't be an issue for it anyway.

The only downside to moving locking down in the call stack is that it's not clear where to lock, exactly, unless a lot of things grow a boolean "lock is already held" flag.

I can of course move the mmutex package one level down, but I don't think that's what you meant :)

On Mon, Oct 19, 2015 at 6:47 AM, John Lenton <email address hidden>
wrote:

> Remember the intention is to move away from having multiple things
> building against "the snappy api"; clients would use the rest api. There's
> a question about what that means for u-d-f, but locking shouldn't be an
> issue for it anyway.
>

 The only thing u-d-f would need to do on its own is kernel, os and gadget
snap (bootstrapping problem); the apps and frameworks should be unpacked by
firstboot.

> The only downside to moving locking down in the call stack is that it's
> not clear where to lock, exactly, unless a lot of things grow a boolean
> "lock is already held" flag.
>

/me starts to type a reply
/me thinks http://cdn.meme.am/instances/500x/65135298.jpg

>
> I can of course move the mmutex package one level down, but I don't think
> that's what you meant :)
>
>

Thanks for the branch and the explanation. One question inline and some minor stuff inline. I'm fine approving once the question is answered. In any case I read again tomorrow morning with a fresh mind, its not trivial (for me).

Gustavo Niemeyer, [21.10.15 10:16]
@chipaca I'm lacking some context for why we'd want that locked-locking-locker

John Lenton, [21.10.15 10:17]
@niemeyer you mean the mutex inside the mmutex, or you mean the whole branch?

Gustavo Niemeyer, [21.10.15 10:17]
I mean the whole thing

John Lenton, [21.10.15 10:18]
@niemeyer well, we need some kind of a lock, to avoid people doing things at the same time and putting us in an unknown state

John Lenton, [21.10.15 10:19]
@niemeyer and a single, global lock didn't seem like a good idea

Gustavo Niemeyer, [21.10.15 10:19]
@chipaca In general we try to accommodate for people doing things at the same time instead, and prevent the actual critical things that cannot be mutated at once from being mutated at once

John Lenton, [21.10.15 10:19]
@niemeyer yes, in general we do

Gustavo Niemeyer, [21.10.15 10:20]
@chipaca Right, thus why I'm trying to understand how this locking system fits

John Lenton, [21.10.15 10:24]
@niemeyer it fits in that most of snappy is still the commandline one-shot thing, which has a filesystem-based lock by necessity to prevent different invocations of it to collide

John Lenton, [21.10.15 10:25]
@niemeyer a refactor of the level needed to move it to accomplish this exclusion without explicit locks is beyond the scope of the work we can do right now

Gustavo Niemeyer, [21.10.15 10:25]
@Chipaca So keep the filesystem locks?

Gustavo Niemeyer, [21.10.15 10:26]
@chipaca Being in a hurry and doing a big complex locking system doesn't feel compatible

John Lenton, [21.10.15 10:27]
this isn't a big complex locking system, imo

John Lenton, [21.10.15 10:27]
the filesystem lock is still there

Gustavo Niemeyer, [21.10.15 10:27]
@Chipaca Rather than one big central system, it would feel much better to me to give the proper places in the code authority for their own concurrency handling

John Lenton, [21.10.15 10:28]
@niemeyer isn't that what this does?

Gustavo Niemeyer, [21.10.15 10:29]
@chipaca That often amounts to a single local mutex, if at all

Gustavo Niemeyer, [21.10.15 10:29]
@chipaca So, again, I may well not be aware of the use case, but it feels like the use case is what we should be talking about

Gustavo Niemeyer, [21.10.15 10:30]
@chipaca It feels complex to me, and it feels complex to mvo, FWIW

John Lenton, [21.10.15 10:31]
yes, but less than 200 lines of code including whitespace and comments is not "big and complex"

John Lenton, [21.10.15 10:31]
anyway, the use case

Gustavo Niemeyer, [21.10.15 10:32]
@chipaca mutex.go is 126 lines long

Gustavo Niemeyer, [21.10.15 10:32]
and a lot of it is docs :)

Gustavo Niemeyer, [21.10.15 10:33]
But I digress

John Lenton, [21.10.15 10:33]
plus 137 for rwmutex

Gustavo Niemeyer, [21.10.15 10:33]
The key here is the use case.. if we need it, so be it, but I'm not yet understanding how this would fit in that cannot be solved by something simpler

John Lenton, [21.10.15 10:34]
when somebody we need a list of packages, or services, etc, we need to avoid people installing/removing/starting/stopping things for a bit

John Lenton, [21.10.15 10:34]
when people operate on a package, we need to stop people operating on that package for a b...

Thanks for your replies, I replied inline as well.