In go1.11, go mod vendor can modify go.sum when go.mod contains replace
directives. This behavior has been fixed in go1.12, so just work around
it for now.
Fixes#2056.
The docker container we're using has 2 vCPUs. -p=8 was working fine for
a while, but diagnosing #1832 looks like the tsi1.test process was
likely getting OOM-killed. Reducing from -p=8 to -p=4 ought to reduce
memory pressure enough to avoid OOM kills, hopefully.
Also remove a stale TODO.
test(platform): run testcontainer integration tests for nightly release
Integration tests for the vault secret service using testcontiners
should not run along with unit tests, however, they should run on some
regular schedule. This commit introduces `make test-integration` which
runs integration tests for vault using testcontainers. The command introduced
relies on docker being available on the host it is executed on.
chore(platform): make go modules tidy
chore: try to fix go mod
chore(platform): remove explicit logrus dependency
chore(platform): run go mod tidy
chore(platform): replace github.com/Sirupsen/logrus with github.com/sirupsen/logrus
chore(platform): update docker dependency
feat(vault): add vault implementation of secret service
test(platform): run testcontainer integration tests for nightly release
Integration tests for the vault secret service using testcontiners
should not run along with unit tests, however, they should run on some
regular schedule. This commit introduces `make test-integration` which
runs integration tests for vault using testcontainers. The command introduced
relies on docker being available on the host it is executed on.
chore(platform): make go modules tidy
chore: try to fix go mod
chore(platform): run go mod tidy
feat(vault): add vault implementation of secret service
chore(platform): make go modules tidy
feat(platform): add Put/Patch/Delete methods on secret service
feat(vault): add Put/Patch/Delete methods on vault secret service
feat(http): add http handler methods for secret service
feat(bolt): add Put/Delete/Patch methods to bolt secret service
feat(testing): add tests for Put/Patch/Delete methods in secret service
feat(mock): add mock secret service
feat(http): add tests for secrets endpoints
feat(http): update swagger for secrets endpoints
chore: run go mod tidy
I looked through the past few days of Circle Go failures, and they all
appeared to be real test errors, indicating that the -p=8 flag most
likely fixed the linker OOMs we were seeing.
If the race-enabled tests continue to OOM, we can try setting -p=2 to
match the number of CPUs actually available on our Circle container.
The environment Go runs in thinks there are 32 CPUs, but according to
https://circleci.com/docs/2.0/configuration-reference/#resource_class
that's a known issue, and only 2 CPUs are actually available.
We're still overallocating with -p=8, so if we continue to get OOMs we
may need to reduce that further.
The one remaining failing static check is in a pigeon-generated promql
file. It looks like megacheck is supposed to ignore generated files by
default, so it's unclear at this time if this is a bug in megacheck, or
if fixing pigeon is the only path forward.
* Simplifiy color type
* Fix type
* Introduce V2 data shape for views
* WIP Split style and parsing to separate Dygraph components
* Add basic dygraph view types
* Add Gauge to v2 view shapes
* Upgrade TypeScript to ^3.0
* Add tsc to Circle build
* Fix Dygraph component paths
* Add testURL to jest config
* Upgrade lodash types
* Remove redundant test linter stetp
* Upgrade to TypeScript ^3
* Remove TableGraph (temporarily)
This updates the dependency manager to use go modules instead of dep so
that we can remove dep as a dependency and we can begin using and giving
feedback on modules within this project. It should simplify dependency
management and make it faster to run the builds too.
The big change is updating the various Makefiles to stop relying on the
vendor directory because it no longer exists. This change creates a
`tools.go` file with the revisions of the tools we use that are currently
in the `required` section of `Gopkg.toml`. We are currently in other
discussions to modify how we handle tool dependency management, but this
change does not change that.
On a cache miss, `yarn install` via `make node_modules` takes about 42
seconds, plus about 19 seconds to upload the cache. See
https://circleci.com/gh/influxdata/platform/1872.
On a cache hit, `make node_modules` takes about 7 seconds to download
the cache plus 19 seconds to `yarn install`. See
https://circleci.com/gh/influxdata/platform/1877. Using the cache here
is a net gain of about 16 seconds on cache hit, which is the most likely
case given a random commit.
Running `dep ensure` via `make vendor`, with a clean dep cache, takes
around 30 seconds.
For run https://circleci.com/gh/influxdata/platform/1838, we can see
that `make vendor` took 33 seconds and saving the dep cache took 38
seconds. The 38 seconds to save the dep cache is paid the first time
that a particular version of Gopkg.lock is pushed. A subsequent run that
fully uses the dep cache costs 14 seconds to restore the dep cache and 5
seconds to run `make vendor`, so this saves about 10 seconds for any run
that doesn't change Gopkg.lock. See
https://circleci.com/gh/influxdata/platform/1840.
With a clean GOCACHE, `make test-go` took about 30 seconds. It's hard to
tell the exact time since `make vendor` was implicitly executed in the
same block.
Now, `make test-go` uses GOCACHE and takes about 5 seconds to restore
the cache, then about 8 seconds to run through a fully cached set of
tests. If any test is actually broken or doesn't compile, this will give
us fast feedback.
But this change also runs `go test -race -count=1`, so that we fully
exercise any possible data races. That currently takes about 19 seconds.
Then finally we save GOCACHE, which has our `go test` results, and a
populated build cache for our test files, including built with the race
detector. Saving that takes about 14 seconds.
That means we took about 30 seconds originally to just run `go test
-count=1`, and now we take (5+8+19+14)=46 seconds to run plain go test
and go test with the race detector. There's probably an argument to be
made for just running with the race detector, but running both gives us
more coverage IMO, and it does allow us to run tests that aren't enabled
on race builds.
See https://circleci.com/gh/influxdata/platform/1840 for a run with full
caching.
Mark Rushakoff
zhulongcheng
AJ Bond
Michael Desa
Brandon Farmer
Edd Robinson
Chris Goller
Jonathan A. Sternberg
Andrew Watkins
lukevmorris
Nathaniel Cook
j. Emrys Landivar (docmerlin)