Influxdata
/
influxdb
mirror of https://github.com/influxdata/influxdb.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
influxdb/.circleci/config.yml

97 lines
2.9 KiB
YAML
Raw Normal View History

Circle ci
2018-05-16 15:43:33 +00:00
version: 2
jobs:
chore(ci): parallelize js and go tests (#670) This PR teases apart the golang and js build steps so that they can be run in parallel in circleci, or locally with make test -j2. Closes #599
2018-08-27 15:34:17 +00:00
jstest:
Circle ci
2018-05-16 15:43:33 +00:00
docker:
ci: use Go 1.11 in CircleCI
2018-08-28 15:38:30 +00:00
- image: circleci/golang:1.11-node-browsers
chore(ci): parallelize js and go tests (#670) This PR teases apart the golang and js build steps so that they can be run in parallel in circleci, or locally with make test -j2. Closes #599
2018-08-27 15:34:17 +00:00
working_directory: /go/src/github.com/influxdata/platform
steps:
- checkout
ci: share yarn cache on Circle On a cache miss, `yarn install` via `make node_modules` takes about 42 seconds, plus about 19 seconds to upload the cache. See https://circleci.com/gh/influxdata/platform/1872. On a cache hit, `make node_modules` takes about 7 seconds to download the cache plus 19 seconds to `yarn install`. See https://circleci.com/gh/influxdata/platform/1877. Using the cache here is a net gain of about 16 seconds on cache hit, which is the most likely case given a random commit.
2018-08-28 21:20:46 +00:00
# Run yarn install, using Circle's cache if applicable.
- restore_cache:
name: Restore Yarn package cache
keys:
# Only cache on exact yarn.lock match, as in Circle's yarn example:
# https://circleci.com/docs/2.0/yarn/
- chronograf-yarn-packages-{{ checksum "chronograf/ui/yarn.lock" }}
- run: make node_modules
- save_cache:
name: Save Yarn package cache
key: chronograf-yarn-packages-{{ checksum "chronograf/ui/yarn.lock" }}
paths:
- ~/.cache/yarn
chore(ci): parallelize js and go tests (#670) This PR teases apart the golang and js build steps so that they can be run in parallel in circleci, or locally with make test -j2. Closes #599
2018-08-27 15:34:17 +00:00
- run: make test-js
fix(ci): fix circleci to use make file
2018-05-22 20:50:10 +00:00
chore(ci): parallelize js and go tests (#670) This PR teases apart the golang and js build steps so that they can be run in parallel in circleci, or locally with make test -j2. Closes #599
2018-08-27 15:34:17 +00:00
gotest:
docker:
ci: use Go 1.11 in CircleCI
2018-08-28 15:38:30 +00:00
- image: circleci/golang:1.11
ci: enable GOCACHE for Circle With a clean GOCACHE, `make test-go` took about 30 seconds. It's hard to tell the exact time since `make vendor` was implicitly executed in the same block. Now, `make test-go` uses GOCACHE and takes about 5 seconds to restore the cache, then about 8 seconds to run through a fully cached set of tests. If any test is actually broken or doesn't compile, this will give us fast feedback. But this change also runs `go test -race -count=1`, so that we fully exercise any possible data races. That currently takes about 19 seconds. Then finally we save GOCACHE, which has our `go test` results, and a populated build cache for our test files, including built with the race detector. Saving that takes about 14 seconds. That means we took about 30 seconds originally to just run `go test -count=1`, and now we take (5+8+19+14)=46 seconds to run plain go test and go test with the race detector. There's probably an argument to be made for just running with the race detector, but running both gives us more coverage IMO, and it does allow us to run tests that aren't enabled on race builds. See https://circleci.com/gh/influxdata/platform/1840 for a run with full caching.
2018-08-28 16:39:44 +00:00
environment:
GOCACHE: /tmp/go-cache
feat: switch to using go modules for building This updates the dependency manager to use go modules instead of dep so that we can remove dep as a dependency and we can begin using and giving feedback on modules within this project. It should simplify dependency management and make it faster to run the builds too. The big change is updating the various Makefiles to stop relying on the vendor directory because it no longer exists. This change creates a `tools.go` file with the revisions of the tools we use that are currently in the `required` section of `Gopkg.toml`. We are currently in other discussions to modify how we handle tool dependency management, but this change does not change that.
2018-08-26 18:54:24 +00:00
GOFLAGs: "-mod=readonly"
Circle ci
2018-05-16 15:43:33 +00:00
working_directory: /go/src/github.com/influxdata/platform
steps:
- checkout
ci: cache dep too Running `dep ensure` via `make vendor`, with a clean dep cache, takes around 30 seconds. For run https://circleci.com/gh/influxdata/platform/1838, we can see that `make vendor` took 33 seconds and saving the dep cache took 38 seconds. The 38 seconds to save the dep cache is paid the first time that a particular version of Gopkg.lock is pushed. A subsequent run that fully uses the dep cache costs 14 seconds to restore the dep cache and 5 seconds to run `make vendor`, so this saves about 10 seconds for any run that doesn't change Gopkg.lock. See https://circleci.com/gh/influxdata/platform/1840.
2018-08-28 18:58:55 +00:00
# Populate GOCACHE.
ci: enable GOCACHE for Circle With a clean GOCACHE, `make test-go` took about 30 seconds. It's hard to tell the exact time since `make vendor` was implicitly executed in the same block. Now, `make test-go` uses GOCACHE and takes about 5 seconds to restore the cache, then about 8 seconds to run through a fully cached set of tests. If any test is actually broken or doesn't compile, this will give us fast feedback. But this change also runs `go test -race -count=1`, so that we fully exercise any possible data races. That currently takes about 19 seconds. Then finally we save GOCACHE, which has our `go test` results, and a populated build cache for our test files, including built with the race detector. Saving that takes about 14 seconds. That means we took about 30 seconds originally to just run `go test -count=1`, and now we take (5+8+19+14)=46 seconds to run plain go test and go test with the race detector. There's probably an argument to be made for just running with the race detector, but running both gives us more coverage IMO, and it does allow us to run tests that aren't enabled on race builds. See https://circleci.com/gh/influxdata/platform/1840 for a run with full caching.
2018-08-28 16:39:44 +00:00
- restore_cache:
name: Restoring GOCACHE
keys:
- platform-gocache-{{ .Branch }}-{{ .Revision }} # Matches when retrying a single run.
- platform-gocache-{{ .Branch }}- # Matches a new commit on an existing branch.
- platform-gocache- # Matches a new branch.
ci: cache dep too Running `dep ensure` via `make vendor`, with a clean dep cache, takes around 30 seconds. For run https://circleci.com/gh/influxdata/platform/1838, we can see that `make vendor` took 33 seconds and saving the dep cache took 38 seconds. The 38 seconds to save the dep cache is paid the first time that a particular version of Gopkg.lock is pushed. A subsequent run that fully uses the dep cache costs 14 seconds to restore the dep cache and 5 seconds to run `make vendor`, so this saves about 10 seconds for any run that doesn't change Gopkg.lock. See https://circleci.com/gh/influxdata/platform/1840.
2018-08-28 18:58:55 +00:00
- run: make test-go # This uses the test cache so it may succeed or fail quickly.
add make vet
2018-05-22 22:19:04 +00:00
- run: make vet
ci: cache dep too Running `dep ensure` via `make vendor`, with a clean dep cache, takes around 30 seconds. For run https://circleci.com/gh/influxdata/platform/1838, we can see that `make vendor` took 33 seconds and saving the dep cache took 38 seconds. The 38 seconds to save the dep cache is paid the first time that a particular version of Gopkg.lock is pushed. A subsequent run that fully uses the dep cache costs 14 seconds to restore the dep cache and 5 seconds to run `make vendor`, so this saves about 10 seconds for any run that doesn't change Gopkg.lock. See https://circleci.com/gh/influxdata/platform/1840.
2018-08-28 18:58:55 +00:00
- run: make test-go-race # This doesn't use the test cache, and will not complete quickly.
fix(ci): fix circleci to use make file
2018-05-22 20:50:10 +00:00
# TODO add these checks to the Makefile
# - run: go get -v -t -d ./...
# - run: go get honnef.co/go/tools/cmd/megacheck
# - run: megacheck ./...
ci: enable GOCACHE for Circle With a clean GOCACHE, `make test-go` took about 30 seconds. It's hard to tell the exact time since `make vendor` was implicitly executed in the same block. Now, `make test-go` uses GOCACHE and takes about 5 seconds to restore the cache, then about 8 seconds to run through a fully cached set of tests. If any test is actually broken or doesn't compile, this will give us fast feedback. But this change also runs `go test -race -count=1`, so that we fully exercise any possible data races. That currently takes about 19 seconds. Then finally we save GOCACHE, which has our `go test` results, and a populated build cache for our test files, including built with the race detector. Saving that takes about 14 seconds. That means we took about 30 seconds originally to just run `go test -count=1`, and now we take (5+8+19+14)=46 seconds to run plain go test and go test with the race detector. There's probably an argument to be made for just running with the race detector, but running both gives us more coverage IMO, and it does allow us to run tests that aren't enabled on race builds. See https://circleci.com/gh/influxdata/platform/1840 for a run with full caching.
2018-08-28 16:39:44 +00:00
- save_cache:
name: Saving GOCACHE
key: platform-gocache-{{ .Branch }}-{{ .Revision }}
paths:
- /tmp/go-cache
when: always
chore(CI): Create nightly builds of ifqld The builds are really anytime a commit is made to master not nightly. Builds use goreleaser with the snapshot setting.
2018-05-22 20:42:32 +00:00
deploy:
docker:
ci: use Go 1.11 in CircleCI
2018-08-28 15:38:30 +00:00
- image: circleci/golang:1.11-node-browsers
ci: restore cache for deploy job The last few runs of deploy took 3m33s, 3m40s, and 4m27s. It should be noticeably faster after we restore the cache.
2018-08-28 21:57:17 +00:00
environment:
GOCACHE: /tmp/go-cache
feat: switch to using go modules for building This updates the dependency manager to use go modules instead of dep so that we can remove dep as a dependency and we can begin using and giving feedback on modules within this project. It should simplify dependency management and make it faster to run the builds too. The big change is updating the various Makefiles to stop relying on the vendor directory because it no longer exists. This change creates a `tools.go` file with the revisions of the tools we use that are currently in the `required` section of `Gopkg.toml`. We are currently in other discussions to modify how we handle tool dependency management, but this change does not change that.
2018-08-26 18:54:24 +00:00
GOFLAGs: "-mod=readonly"
chore(CI): Create nightly builds of ifqld The builds are really anytime a commit is made to master not nightly. Builds use goreleaser with the snapshot setting.
2018-05-22 20:42:32 +00:00
working_directory: /go/src/github.com/influxdata/platform
steps:
- checkout
ci: restore cache for deploy job The last few runs of deploy took 3m33s, 3m40s, and 4m27s. It should be noticeably faster after we restore the cache.
2018-08-28 21:57:17 +00:00
# Speed up `make nightly` by restoring caches from previous runs.
- restore_cache:
name: Restoring GOCACHE
keys:
- platform-gocache- # Just match the most recent Go cache.
- restore_cache:
name: Restore Yarn package cache
keys:
- chronograf-yarn-packages-{{ checksum "chronograf/ui/yarn.lock" }}
fix(circleci): setup remote docker so that the deploy works for nightlies
2018-07-17 22:23:39 +00:00
- setup_remote_docker
feat(docker): create a fluxd nightly docker image
2018-07-17 19:03:40 +00:00
- run: |
docker login -u "$QUAY_USER" -p $QUAY_PASS quay.io
make nightly
chore(CI): Create nightly builds of ifqld The builds are really anytime a commit is made to master not nightly. Builds use goreleaser with the snapshot setting.
2018-05-22 20:42:32 +00:00
workflows:
version: 2
build-and-deploy:
jobs:
chore(ci): parallelize js and go tests (#670) This PR teases apart the golang and js build steps so that they can be run in parallel in circleci, or locally with make test -j2. Closes #599
2018-08-27 15:34:17 +00:00
- gotest
- jstest
chore(CI): Create nightly builds of ifqld The builds are really anytime a commit is made to master not nightly. Builds use goreleaser with the snapshot setting.
2018-05-22 20:42:32 +00:00
- deploy:
requires:
chore(ci): parallelize js and go tests (#670) This PR teases apart the golang and js build steps so that they can be run in parallel in circleci, or locally with make test -j2. Closes #599
2018-08-27 15:34:17 +00:00
- gotest
- jstest
chore(CI): Create nightly builds of ifqld The builds are really anytime a commit is made to master not nightly. Builds use goreleaser with the snapshot setting.
2018-05-22 20:42:32 +00:00
filters:
branches:
only: master