Merge lp://staging/~danilo/juju-core/bootstrap-verify into lp://staging/~juju/juju-core/trunk
- bootstrap-verify
- Merge into trunk
Status: | Superseded |
---|---|
Proposed branch: | lp://staging/~danilo/juju-core/bootstrap-verify |
Merge into: | lp://staging/~juju/juju-core/trunk |
Diff against target: |
247 lines (+114/-2) 9 files modified
cmd/juju/bootstrap_test.go (+4/-0) environs/dummy/environs.go (+7/-1) environs/dummy/storage.go (+6/-1) environs/ec2/ec2.go (+5/-0) environs/jujutest/livetests.go (+17/-0) environs/maas/environ.go (+4/-0) environs/openstack/provider.go (+4/-0) environs/storage.go (+22/-0) environs/storage_test.go (+45/-0) |
To merge this branch: | bzr merge lp://staging/~danilo/juju-core/bootstrap-verify |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Juju Engineering | Pending | ||
Review via email: mp+166481@code.staging.launchpad.net |
This proposal has been superseded by a proposal from 2013-06-17.
Commit message
Description of the change
Verify storage writing on bootstrap
Replicates a mechanism used by python juju to test if private bucket storage is writeable by juju on bootstrap. It, however, uses a different content inside the bootstrap-verify file since we want to use the differences to detect a Python environment and fail on it (this is coming in a follow-up branch).
This has been tested with ec2, but not with openstack or maas. Any suggestions on how to best do that are welcome.
Данило Шеган (danilo) wrote : | # |
William Reade (fwereade) wrote : | # |
Since you *did* implement this for the dummy provider, you could add a
jujutest test [0] that something actually is written to environ storage.
Wouldn't fix MAAS, which doesn't use those, but that should I think be
testable directly?
[0] these (mostly?) get run against ec2, openstack, dummy. But they're
also somewhat crazy and confusing. I don't really mind how this is
tested, so long as it is :).
PS are we expecting a followup that checks there's no pyjuju
check-writable file?
https:/
File environs/
https:/
environs/
not writeable.")
FWIW, this is extending an antipattern -- the dummy provider doesn't do
the same stuff as the others anyway, and faking it up doesn't seem that
helpful -- but I'm not suggesting you change this today.
Данило Шеган (danilo) wrote : | # |
Please take a look.
Данило Шеган (danilo) wrote : | # |
On 2013/05/30 16:03:43, fwereade wrote:
> Since you *did* implement this for the dummy provider, you could add a
jujutest
> test [0] that something actually is written to environ storage.
Wouldn't fix
> MAAS, which doesn't use those, but that should I think be testable
directly?
At the very least, I've extended the dummy provider's OpPutFile to log
the filename as well and I watch out for that operation in the bootstrap
jujutest. I can add in the file contents as well, which might make
sense for the follow up branch (but if I am doing it for it, I'd rather
do it there).
I'd add a separate test for bootstrap verify, but it would be running
exactly the same code and I am uncertain if extending the test suite run
for this makes sense.
Данило Шеган (danilo) wrote : | # |
https:/
File environs/
https:/
environs/
not writeable.")
What would you suggest I do instead? Ignore the error?
Frank Mueller (themue) wrote : | # |
LGTM with some minor comments.
https:/
File environs/ec2/ec2.go (right):
https:/
environs/
writeable.")
Lowercase, "provider storage ...", and not dot at the end.
https:/
File environs/
https:/
environs/
writeable.")
Dito, lowercase and dot.
https:/
File environs/
https:/
environs/
is not writeable.")
And again lowercase and dot.
https:/
File environs/storage.go (right):
https:/
environs/
"bootstrap-verify"
Typical camel-case is used. And it only starts with an uppercase char if
it shall be exported.
John A Meinel (jameinel) wrote : | # |
I think this is reasonable, but is it ok to just write something and
look for an error code rather than also trying to read it back in (or
seeing if one already exists)?
https:/
File environs/storage.go (right):
https:/
environs/
writing verified: ok"
If you are writing it out for long-term storage, it is usually nice to
put a '\n' on the end of it.
William Reade (fwereade) wrote : | # |
On 2013/06/04 12:31:29, Danilo wrote:
> On 2013/05/30 16:03:43, fwereade wrote:
> > Since you *did* implement this for the dummy provider, you could add
a
> jujutest
> > test [0] that something actually is written to environ storage.
Wouldn't fix
> > MAAS, which doesn't use those, but that should I think be testable
directly?
> At the very least, I've extended the dummy provider's OpPutFile to log
the
> filename as well and I watch out for that operation in the bootstrap
jujutest.
> I can add in the file contents as well, which might make sense for the
follow up
> branch (but if I am doing it for it, I'd rather do it there).
> I'd add a separate test for bootstrap verify, but it would be running
exactly
> the same code and I am uncertain if extending the test suite run for
this makes
> sense.
I don't see an environs/jujutest change anywhere... if we're not syncing
the check against all providers, it doesn't help.
William Reade (fwereade) wrote : | # |
https:/
File environs/
https:/
environs/
not writeable.")
On 2013/06/04 12:31:38, Danilo wrote:
> What would you suggest I do instead? Ignore the error?
Sorry, I was commenting a bit obliquely on the fundamental issues with
the dummy provider. Implementing this functionality here is only helpful
if it's to support a jujutest test case, which would then be able to
verify that all providers (except MAAS... different discussion) fulfil
the same expectations. That is: without a jujutest case, I don't think
it's meaningful to do this.
Dimiter Naydenov (dimitern) wrote : | # |
Almost there and nice! Some minor suggestions, but big +1 on fwereade's
comments about changing jujutest.
https:/
File environs/
https:/
environs/
not writeable.")
s/Provider/
https:/
File environs/ec2/ec2.go (right):
https:/
environs/
writeable.")
On 2013/06/04 12:42:52, mue wrote:
> Lowercase, "provider storage ...", and not dot at the end.
+1
https:/
File environs/
https:/
environs/
writeable.")
On 2013/06/04 12:42:52, mue wrote:
> Dito, lowercase and dot.
+1
https:/
File environs/
https:/
environs/
is not writeable.")
On 2013/06/04 12:42:52, mue wrote:
> And again lowercase and dot.
+1
https:/
File environs/storage.go (right):
https:/
environs/
"bootstrap-verify"
Please, this is not C :)
const VerificationFil
https:/
environs/
writing verified: ok"
ditto: VerificationContent
https:/
File environs/
https:/
environs/
does it have to be exported?
https:/
environs/
length)
if you're doing the call multiline, please:
log_message := fmt.Sprintf(
"...",
filename,
content,
length,
)
https:/
environs/
again, the suite doesn't need to be exported (verifyStorageSuite or even
just storageSuite)
Данило Шеган (danilo) wrote : | # |
Please take a look.
https:/
File environs/
https:/
environs/
not writeable.")
I've added a minimal jujutest that bootstraps the environment, and then
makes sure that the bootstrap-verify file is written in. It has been
tested with at least ec2 and openstack providers.
https:/
File environs/
https:/
environs/
not writeable.")
On 2013/06/07 12:35:20, dimitern wrote:
> s/Provider/
Fixed already after review by Frank.
https:/
File environs/ec2/ec2.go (right):
https:/
environs/
writeable.")
Fixed.
https:/
File environs/
https:/
environs/
writeable.")
Fixed.
https:/
File environs/
https:/
environs/
is not writeable.")
And fixed.
https:/
File environs/storage.go (right):
https:/
environs/
"bootstrap-verify"
On 2013/06/07 12:35:20, dimitern wrote:
> Please, this is not C :)
> const VerificationFil
This is actually quite common naming pattern in Python ;) Naming
patterns are there to make it easier for us to see this in an arbitrary
place in code and understand that this is a constant. I know that Go
makes everything starting with a capital exported, but that means that
there's no good naming pattern for constants.
Anyway, changed to verificationFil
https:/
environs/
writing verified: ok"
On 2013/06/04 17:15:34, jameinel wrote:
> If you are writing it out for long-term storage, it is usually nice to
put a
> '\n' on the end of it.
Good point, fixed.
https:/
environs/
writing verified: ok"
On 2013/06/07 12:35:20, dimitern wrote:
> ditto: VerificationContent
Fixed to verificationCon
John A Meinel (jameinel) wrote : | # |
William Reade (fwereade) wrote : | # |
LGTM, just one thought.
https:/
File environs/storage.go (right):
https:/
environs/
Might be nicer to log the actual error here, and return
fmt.Errorf(
just return non-nil errors unmodified.
Данило Шеган (danilo) wrote : | # |
https:/
File environs/storage.go (right):
https:/
environs/
On 2013/06/13 16:02:22, fwereade wrote:
> Might be nicer to log the actual error here, and return
fmt.Errorf(
> storage is not writable"), so the clients can all just return non-nil
errors
> unmodified.
I've done this (had to change the unit test for failure to match on this
error now).
Reviewers: mp+166481_ code.launchpad. net,
Message:
Please take a look.
Description:
Verify storage writing on bootstrap
Replicates a mechanism used by python juju to test if private bucket
storage is writeable by juju on bootstrap. It, however, uses a
different content inside the bootstrap-verify file since we want to use
the differences to detect a Python environment and fail on it (this is
coming in a follow-up branch).
This has been tested with ec2, but not with openstack or maas. Any
suggestions on how to best do that are welcome.
https:/ /code.launchpad .net/~danilo/ juju-core/ bootstrap- verify/ +merge/ 166481
(do not edit description out of merge proposal)
Please review this at https:/ /codereview. appspot. com/9876043/
Affected files: dummy/environs. go maas/environ. go openstack/ provider. go storage_ test.go
A [revision details]
M environs/
M environs/ec2/ec2.go
M environs/
M environs/
M environs/storage.go
M environs/
Index: [revision details]
=== added file '[revision details]'
--- [revision details] 2012-01-01 00:00:00 +0000
+++ [revision details] 2012-01-01 00:00:00 +0000
@@ -0,0 +1,2 @@
+Old revision:
<email address hidden>
+New revision: <email address hidden>
Index: environs/storage.go storage. go'
=== modified file 'environs/
--- environs/storage.go 2013-05-02 15:55:42 +0000
+++ environs/storage.go 2013-05-30 10:54:02 +0000
@@ -6,6 +6,7 @@
import (
"fmt"
"io"
+ "strings"
)
// EmptyStorage holds a StorageReader object that contains no files and
@@ -14,6 +15,9 @@
type emptyStorage struct{}
+const VERIFICATION_ FILENAME string = "bootstrap-verify" CONTENT = "juju-core storage writing verified: ok" fmt.Errorf( "file %q not found", name)} storage Storage) error { NewReader( VERIFICATION_ CONTENT) Put(VERIFICATIO N_FILENAME, reader, VERIFICATION_ CONTENT) ))
+const VERIFICATION_
+
func (s emptyStorage) Get(name string) (io.ReadCloser, error) {
return nil, &NotFoundError{
}
@@ -25,3 +29,10 @@
func (s emptyStorage) List(prefix string) ([]string, error) {
return nil, nil
}
+
+func VerifyStorage(
+ reader := strings.
+ err := storage.
+ int64(len(
+ return err
+}
Index: environs/ storage_ test.go storage_ test.go' storage_ test.go 2013-05-02 15:55:42 +0000 storage_ test.go 2013-05-30 11:09:25 +0000
=== modified file 'environs/
--- environs/
+++ environs/
@@ -4,6 +4,9 @@
package environs_test
import ( net/gocheck" net/juju- core/environs" ReadAll( reader) ms.StorageReque sts, log_message) "Get('% s')", name) ms.StorageReque sts, log_message)
+ "fmt"
+ "io"
+ "io/ioutil"
. "launchpad.
"launchpad.
)
@@ -29,3 +32,46 @@
c.Assert(names, IsNil)
c.Assert(err, IsNil)
}
+
+type MockStorage struct {
+ StorageRequests []string
+}
+
+func (ms *MockStorage) Put(filename string, reader io.Reader, length
int64) error {
+ var content []byte
+ content, _ = ioutil.
+ log_message := fmt.Sprintf(
+ "Put('%s', '%s', %d)", filename, content, length)
+ ms.StorageRequests = append(
+ return nil
+}
+
+func (ms *MockStorage) Get(name string) (io.ReadCloser, error) {
+ log_message := fmt.Sprintf(
+ ms.StorageRequests = append(
+ return nil, &envir...