chore(store) EE-1981: Refactor/store/error checking, and other refactoring (#6173)

* use the Store interface IsErrObjectNotFound() to avoid revealing internal errors

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* what happens when you extract the datastore interfaces into their own package

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* Start renaming Storage methods

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the boltdb specific code from the Portainer storage code (example, the others need the same)

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more extract bolt.Tx from datastore code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* minimise imports by putting moving the struct definition into the file that needs the Service imports

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more extraction of boltdb.Tx

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the use of bucket.SetSequence

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* almost done - just endpoint.Synchonise :/

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring'

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix test compile errors

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* test compile fixes after rebase

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* set us up to make the connection an interface

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* make the db connection a datastore interface, and separate out our datastore services from the bolt ones

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* rename methods to something less oltdb internals specific

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* these errors are not boltdb secific

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* start using the db-backend factory method too

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* export boltdb raw in case we can't export from the service layer

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add the version info by hand for now

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* actually, the export from services can be fully typed - its the import that needs to do more work

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* redo raw export, and make import capable of using it

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add DockerHub

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix go test ./... again

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* my goland wasn't setup to gofmt

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* move the two extremely dubious migration tests down into store, so they can use the test store code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* the migrator is now free of boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* reverse goland overzealous replcement of internal with boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more undo over-zealous goland internal->boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* yay, now bolt is only mentioned inside the api/database/ dir

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* and this might be the last of the boltdb references?

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add todo

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the store code into a separate module too

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* don't need the fileService in boltdb anymore

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* use IsErrObjectNotFound()

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* use a string to select what database backend we use

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* make isNew store an ephemeral bool that doesn't stay true after we've initialised it

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* undo vscode formatting html

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix app templates symbol (#6221)

* feat(webhook) EE-2125 send registry auth haeder when update swarms service via webhook (#6220)

* feat(webhook) EE-2125 add some helpers to registry utils

* feat(webhook) EE-2125 persist registryID when creating a webhook

* feat(webhook) EE-2125 send registry auth header when executing a webhook

* feat(webhook) EE-2125 send registryID to backend when creating a service with webhook

* feat(webhook) EE-2125 use the initial registry ID to create webhook on editing service screen

* feat(webhook) EE-2125 update webhook when update registry

* feat(webhook) EE-2125 add endpoint of update webhook

* feat(webhook) EE-2125 code cleanup

* feat(webhook) EE-2125 fix a typo

* feat(webhook) EE-2125 fix circle import issue with unit test

Co-authored-by: Simon Meng <simon.meng@portainer.io>

* fix(kubeconfig): show kubeconfig download button for non admin users [EE-2123] (#6204)

Co-authored-by: Simon Meng <simon.meng@portainer.io>

* fix data-cy for k8s cluster menu (#6226)

LGTM

* feat(stack): make stack created from app template editable EE-1941 (#6104)

feat(stack): make stack from app template editable

* fix(container):disable Duplicate/Edit button when the container is portainer (#6223)

* fix/ee-1909/show-pull-image-error (#6195)

Co-authored-by: sunportainer <ericsun@SG1.local>

* feat(cy): add data-cy to helm install button (#6241)

* feat(cy): add data-cy to add registry button (#6242)

* refactor(app): convert root folder files to es6 (#4159)

* refactor(app): duplicate constants as es6 exports (#4158)

* fix(docker): provide workaround to save network name variable  (#6080)

* fix/EE-1862/unable-to-stop-or-remove-stack workaround for var without default value in yaml file

* fix/EE-1862/unable-to-stop-or-remove-stack check yaml file

* fixed func and var names

* wrapper error and used bool for stringset

* UT case for createNetworkEnvFile

* UT case for %s=%s

* powerful StringSet

* wrapper error for extract network name

* wrapper all the return err

* store more env

* put to env file

* make default value None

* feat: gzip static resources (#6258)

* fix(ssl)//handle --sslcert and --sslkey ee-2106 (#6203)

* fix/ee-2106/handle-sslcert-sslkey

Co-authored-by: sunportainer <ericsun@SG1.local>

* fix(server):support disable https only ee-2068 (#6232)

* fix/ee-2068/disable-forcely-https

* feat(store): implement store tests EE-2112 (#6224)

* add store tests

* add some more tests

* Update missing helm user repo methods

* remove redundant comments

* add webhook export

* update webhooks

* use the Store interface IsErrObjectNotFound() to avoid revealing internal errors

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* what happens when you extract the datastore interfaces into their own package

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* Start renaming Storage methods

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the boltdb specific code from the Portainer storage code (example, the others need the same)

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more extract bolt.Tx from datastore code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* minimise imports by putting moving the struct definition into the file that needs the Service imports

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more extraction of boltdb.Tx

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the use of bucket.SetSequence

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* almost done - just endpoint.Synchonise :/

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring'

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix test compile errors

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* test compile fixes after rebase

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* set us up to make the connection an interface

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* make the db connection a datastore interface, and separate out our datastore services from the bolt ones

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* rename methods to something less oltdb internals specific

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* these errors are not boltdb secific

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* start using the db-backend factory method too

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* export boltdb raw in case we can't export from the service layer

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add the version info by hand for now

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* actually, the export from services can be fully typed - its the import that needs to do more work

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* redo raw export, and make import capable of using it

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add DockerHub

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix go test ./... again

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* my goland wasn't setup to gofmt

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* move the two extremely dubious migration tests down into store, so they can use the test store code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* the migrator is now free of boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* reverse goland overzealous replcement of internal with boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more undo over-zealous goland internal->boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* yay, now bolt is only mentioned inside the api/database/ dir

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* and this might be the last of the boltdb references?

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add todo

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the store code into a separate module too

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* don't need the fileService in boltdb anymore

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* use IsErrObjectNotFound()

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* use a string to select what database backend we use

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* make isNew store an ephemeral bool that doesn't stay true after we've initialised it

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* undo vscode formatting html

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* Update missing helm user repo methods

* feat(store): implement store tests EE-2112 (#6224)

* add store tests

* add some more tests

* remove redundant comments

* add webhook export

* update webhooks

* fix build issues after rebasing

* move migratorparams

* remove unneeded integer type conversions

* disable the db import/export for now

Co-authored-by: Richard Wei <54336863+WaysonWei@users.noreply.github.com>
Co-authored-by: cong meng <mcpacino@gmail.com>
Co-authored-by: Simon Meng <simon.meng@portainer.io>
Co-authored-by: Marcelo Rydel <marcelorydel26@gmail.com>
Co-authored-by: Hao Zhang <hao.zhang@portainer.io>
Co-authored-by: sunportainer <93502624+sunportainer@users.noreply.github.com>
Co-authored-by: sunportainer <ericsun@SG1.local>
Co-authored-by: wheresolivia <78844659+wheresolivia@users.noreply.github.com>
Co-authored-by: Chaim Lev-Ari <chiptus@users.noreply.github.com>
Co-authored-by: Chao Geng <93526589+chaogeng77977@users.noreply.github.com>
Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com>
Co-authored-by: Matt Hook <hookenz@gmail.com>
pull/6279/head
Sven Dowideit 2021-12-15 12:26:09 +10:00 committed by GitHub
parent b02bf0c9d7
commit f99329eb7e
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
299 changed files with 5246 additions and 5092 deletions

View File

@ -6,19 +6,20 @@ import (
"time"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices"
)
var logFatalf = log.Fatalf
type Monitor struct {
timeout time.Duration
datastore portainer.DataStore
datastore dataservices.DataStore
shutdownCtx context.Context
cancellationFunc context.CancelFunc
}
// New creates a monitor that when started will wait for the timeout duration and then shutdown the application unless it has been initialized.
func New(timeout time.Duration, datastore portainer.DataStore, shutdownCtx context.Context) *Monitor {
func New(timeout time.Duration, datastore dataservices.DataStore, shutdownCtx context.Context) *Monitor {
return &Monitor{
timeout: timeout,
datastore: datastore,

View File

@ -9,6 +9,7 @@ import (
"github.com/pkg/errors"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices"
)
const portainerAPIKeyPrefix = "ptr_"
@ -16,12 +17,12 @@ const portainerAPIKeyPrefix = "ptr_"
var ErrInvalidAPIKey = errors.New("Invalid API key")
type apiKeyService struct {
apiKeyRepository portainer.APIKeyRepository
userRepository portainer.UserService
apiKeyRepository dataservices.APIKeyRepository
userRepository dataservices.UserService
cache *apiKeyCache
}
func NewAPIKeyService(apiKeyRepository portainer.APIKeyRepository, userRepository portainer.UserService) *apiKeyService {
func NewAPIKeyService(apiKeyRepository dataservices.APIKeyRepository, userRepository dataservices.UserService) *apiKeyService {
return &apiKeyService{
apiKeyRepository: apiKeyRepository,
userRepository: userRepository,

View File

@ -8,7 +8,7 @@ import (
"time"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt"
"github.com/portainer/portainer/api/datastore"
"github.com/stretchr/testify/assert"
)
@ -20,7 +20,7 @@ func Test_SatisfiesAPIKeyServiceInterface(t *testing.T) {
func Test_GenerateApiKey(t *testing.T) {
is := assert.New(t)
store, teardown := bolt.MustNewTestStore(true)
_, store, teardown := datastore.MustNewTestStore(true)
defer teardown()
service := NewAPIKeyService(store.APIKeyRepository(), store.User())
@ -74,7 +74,7 @@ func Test_GenerateApiKey(t *testing.T) {
func Test_GetAPIKey(t *testing.T) {
is := assert.New(t)
store, teardown := bolt.MustNewTestStore(true)
_, store, teardown := datastore.MustNewTestStore(true)
defer teardown()
service := NewAPIKeyService(store.APIKeyRepository(), store.User())
@ -94,7 +94,7 @@ func Test_GetAPIKey(t *testing.T) {
func Test_GetAPIKeys(t *testing.T) {
is := assert.New(t)
store, teardown := bolt.MustNewTestStore(true)
_, store, teardown := datastore.MustNewTestStore(true)
defer teardown()
service := NewAPIKeyService(store.APIKeyRepository(), store.User())
@ -115,7 +115,7 @@ func Test_GetAPIKeys(t *testing.T) {
func Test_GetDigestUserAndKey(t *testing.T) {
is := assert.New(t)
store, teardown := bolt.MustNewTestStore(true)
_, store, teardown := datastore.MustNewTestStore(true)
defer teardown()
service := NewAPIKeyService(store.APIKeyRepository(), store.User())
@ -151,14 +151,14 @@ func Test_GetDigestUserAndKey(t *testing.T) {
func Test_UpdateAPIKey(t *testing.T) {
is := assert.New(t)
store, teardown := bolt.MustNewTestStore(true)
_, store, teardown := datastore.MustNewTestStore(true)
defer teardown()
service := NewAPIKeyService(store.APIKeyRepository(), store.User())
t.Run("Successfully updates the api-key LastUsed time", func(t *testing.T) {
user := portainer.User{ID: 1}
store.User().CreateUser(&user)
store.User().Create(&user)
_, apiKey, err := service.GenerateApiKey(user, "test-x")
is.NoError(err)
@ -199,7 +199,7 @@ func Test_UpdateAPIKey(t *testing.T) {
func Test_DeleteAPIKey(t *testing.T) {
is := assert.New(t)
store, teardown := bolt.MustNewTestStore(true)
_, store, teardown := datastore.MustNewTestStore(true)
defer teardown()
service := NewAPIKeyService(store.APIKeyRepository(), store.User())
@ -240,7 +240,7 @@ func Test_DeleteAPIKey(t *testing.T) {
func Test_InvalidateUserKeyCache(t *testing.T) {
is := assert.New(t)
store, teardown := bolt.MustNewTestStore(true)
_, store, teardown := datastore.MustNewTestStore(true)
defer teardown()
service := NewAPIKeyService(store.APIKeyRepository(), store.User())

View File

@ -3,15 +3,17 @@ package backup
import (
"fmt"
"os"
"path"
"path/filepath"
"time"
"github.com/pkg/errors"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/archive"
"github.com/portainer/portainer/api/crypto"
"github.com/portainer/portainer/api/dataservices"
"github.com/portainer/portainer/api/filesystem"
"github.com/portainer/portainer/api/http/offlinegate"
"github.com/sirupsen/logrus"
)
const rwxr__r__ os.FileMode = 0744
@ -30,7 +32,7 @@ var filesToBackup = []string{
}
// Creates a tar.gz system archive and encrypts it if password is not empty. Returns a path to the archive file.
func CreateBackupArchive(password string, gate *offlinegate.OfflineGate, datastore portainer.DataStore, filestorePath string) (string, error) {
func CreateBackupArchive(password string, gate *offlinegate.OfflineGate, datastore dataservices.DataStore, filestorePath string) (string, error) {
unlock := gate.Lock()
defer unlock()
@ -39,6 +41,18 @@ func CreateBackupArchive(password string, gate *offlinegate.OfflineGate, datasto
return "", errors.Wrap(err, "Failed to create backup dir")
}
{
// new export
exportFilename := path.Join(backupDirPath, fmt.Sprintf("export-%d.json", time.Now().Unix()))
err := datastore.Export(exportFilename)
if err != nil {
logrus.WithError(err).Debugf("failed to export to %s", exportFilename)
} else {
logrus.Debugf("exported to %s", exportFilename)
}
}
if err := backupDb(backupDirPath, datastore); err != nil {
return "", errors.Wrap(err, "Failed to backup database")
}
@ -65,7 +79,7 @@ func CreateBackupArchive(password string, gate *offlinegate.OfflineGate, datasto
return archivePath, nil
}
func backupDb(backupDirPath string, datastore portainer.DataStore) error {
func backupDb(backupDirPath string, datastore dataservices.DataStore) error {
backupWriter, err := os.Create(filepath.Join(backupDirPath, "portainer.db"))
if err != nil {
return err

View File

@ -8,9 +8,9 @@ import (
"time"
"github.com/pkg/errors"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/archive"
"github.com/portainer/portainer/api/crypto"
"github.com/portainer/portainer/api/dataservices"
"github.com/portainer/portainer/api/filesystem"
"github.com/portainer/portainer/api/http/offlinegate"
)
@ -18,7 +18,7 @@ import (
var filesToRestore = append(filesToBackup, "portainer.db")
// Restores system state from backup archive, will trigger system shutdown, when finished.
func RestoreArchive(archive io.Reader, password string, filestorePath string, gate *offlinegate.OfflineGate, datastore portainer.DataStore, shutdownTrigger context.CancelFunc) error {
func RestoreArchive(archive io.Reader, password string, filestorePath string, gate *offlinegate.OfflineGate, datastore dataservices.DataStore, shutdownTrigger context.CancelFunc) error {
var err error
if password != "" {
archive, err = decrypt(archive, password)

View File

@ -1,137 +0,0 @@
package apikeyrepository
import (
"bytes"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/errors"
"github.com/portainer/portainer/api/bolt/internal"
"github.com/boltdb/bolt"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "api_key"
)
// Service represents a service for managing api-key data.
type Service struct {
connection *internal.DbConnection
}
// NewService creates a new instance of a service.
func NewService(connection *internal.DbConnection) (*Service, error) {
err := internal.CreateBucket(connection, BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// GetAPIKeysByUserID returns a slice containing all the APIKeys a user has access to.
func (service *Service) GetAPIKeysByUserID(userID portainer.UserID) ([]portainer.APIKey, error) {
var result = make([]portainer.APIKey, 0)
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var record portainer.APIKey
err := internal.UnmarshalObject(v, &record)
if err != nil {
return err
}
if record.UserID == userID {
result = append(result, record)
}
}
return nil
})
return result, err
}
// GetAPIKeyByDigest returns the API key for the associated digest.
// Note: there is a 1-to-1 mapping of api-key and digest
func (service *Service) GetAPIKeyByDigest(digest []byte) (*portainer.APIKey, error) {
var result portainer.APIKey
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var record portainer.APIKey
err := internal.UnmarshalObject(v, &record)
if err != nil {
return err
}
if bytes.Equal(record.Digest, digest) {
result = record
return nil
}
}
return nil
})
return &result, err
}
// CreateAPIKey creates a new APIKey object.
func (service *Service) CreateAPIKey(record *portainer.APIKey) error {
return service.connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
id, _ := bucket.NextSequence()
record.ID = portainer.APIKeyID(id)
data, err := internal.MarshalObject(record)
if err != nil {
return err
}
return bucket.Put(internal.Itob(int(record.ID)), data)
})
}
// GetAPIKey retrieves an existing APIKey object by api key ID.
func (service *Service) GetAPIKey(keyID portainer.APIKeyID) (*portainer.APIKey, error) {
var apiKey *portainer.APIKey
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
item := bucket.Get(internal.Itob(int(keyID)))
if item == nil {
return errors.ErrObjectNotFound
}
err := internal.UnmarshalObject(item, &apiKey)
if err != nil {
return err
}
return nil
})
return apiKey, err
}
func (service *Service) UpdateAPIKey(key *portainer.APIKey) error {
identifier := internal.Itob(int(key.ID))
return internal.UpdateObject(service.connection, BucketName, identifier, key)
}
func (service *Service) DeleteAPIKey(ID portainer.APIKeyID) error {
return service.connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
return bucket.Delete(internal.Itob(int(ID)))
})
}

View File

@ -1,96 +0,0 @@
package customtemplate
import (
"github.com/boltdb/bolt"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/internal"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "customtemplates"
)
// Service represents a service for managing custom template data.
type Service struct {
connection *internal.DbConnection
}
// NewService creates a new instance of a service.
func NewService(connection *internal.DbConnection) (*Service, error) {
err := internal.CreateBucket(connection, BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// CustomTemplates return an array containing all the custom templates.
func (service *Service) CustomTemplates() ([]portainer.CustomTemplate, error) {
var customTemplates = make([]portainer.CustomTemplate, 0)
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var customTemplate portainer.CustomTemplate
err := internal.UnmarshalObjectWithJsoniter(v, &customTemplate)
if err != nil {
return err
}
customTemplates = append(customTemplates, customTemplate)
}
return nil
})
return customTemplates, err
}
// CustomTemplate returns an custom template by ID.
func (service *Service) CustomTemplate(ID portainer.CustomTemplateID) (*portainer.CustomTemplate, error) {
var customTemplate portainer.CustomTemplate
identifier := internal.Itob(int(ID))
err := internal.GetObject(service.connection, BucketName, identifier, &customTemplate)
if err != nil {
return nil, err
}
return &customTemplate, nil
}
// UpdateCustomTemplate updates an custom template.
func (service *Service) UpdateCustomTemplate(ID portainer.CustomTemplateID, customTemplate *portainer.CustomTemplate) error {
identifier := internal.Itob(int(ID))
return internal.UpdateObject(service.connection, BucketName, identifier, customTemplate)
}
// DeleteCustomTemplate deletes an custom template.
func (service *Service) DeleteCustomTemplate(ID portainer.CustomTemplateID) error {
identifier := internal.Itob(int(ID))
return internal.DeleteObject(service.connection, BucketName, identifier)
}
// CreateCustomTemplate assign an ID to a new custom template and saves it.
func (service *Service) CreateCustomTemplate(customTemplate *portainer.CustomTemplate) error {
return service.connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
data, err := internal.MarshalObject(customTemplate)
if err != nil {
return err
}
return bucket.Put(internal.Itob(int(customTemplate.ID)), data)
})
}
// GetNextIdentifier returns the next identifier for a custom template.
func (service *Service) GetNextIdentifier() int {
return internal.GetNextIdentifier(service.connection, BucketName)
}

View File

@ -1,156 +0,0 @@
package bolt
import (
"io"
"path"
"time"
"github.com/portainer/portainer/api/bolt/apikeyrepository"
"github.com/portainer/portainer/api/bolt/helmuserrepository"
"github.com/boltdb/bolt"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/customtemplate"
"github.com/portainer/portainer/api/bolt/dockerhub"
"github.com/portainer/portainer/api/bolt/edgegroup"
"github.com/portainer/portainer/api/bolt/edgejob"
"github.com/portainer/portainer/api/bolt/edgestack"
"github.com/portainer/portainer/api/bolt/endpoint"
"github.com/portainer/portainer/api/bolt/endpointgroup"
"github.com/portainer/portainer/api/bolt/endpointrelation"
"github.com/portainer/portainer/api/bolt/errors"
"github.com/portainer/portainer/api/bolt/extension"
"github.com/portainer/portainer/api/bolt/internal"
"github.com/portainer/portainer/api/bolt/registry"
"github.com/portainer/portainer/api/bolt/resourcecontrol"
"github.com/portainer/portainer/api/bolt/role"
"github.com/portainer/portainer/api/bolt/schedule"
"github.com/portainer/portainer/api/bolt/settings"
"github.com/portainer/portainer/api/bolt/ssl"
"github.com/portainer/portainer/api/bolt/stack"
"github.com/portainer/portainer/api/bolt/tag"
"github.com/portainer/portainer/api/bolt/team"
"github.com/portainer/portainer/api/bolt/teammembership"
"github.com/portainer/portainer/api/bolt/tunnelserver"
"github.com/portainer/portainer/api/bolt/user"
"github.com/portainer/portainer/api/bolt/version"
"github.com/portainer/portainer/api/bolt/webhook"
)
const (
databaseFileName = "portainer.db"
)
// Store defines the implementation of portainer.DataStore using
// BoltDB as the storage system.
type Store struct {
path string
connection *internal.DbConnection
isNew bool
fileService portainer.FileService
CustomTemplateService *customtemplate.Service
DockerHubService *dockerhub.Service
EdgeGroupService *edgegroup.Service
EdgeJobService *edgejob.Service
EdgeStackService *edgestack.Service
EndpointGroupService *endpointgroup.Service
EndpointService *endpoint.Service
EndpointRelationService *endpointrelation.Service
ExtensionService *extension.Service
HelmUserRepositoryService *helmuserrepository.Service
RegistryService *registry.Service
ResourceControlService *resourcecontrol.Service
RoleService *role.Service
APIKeyRepositoryService *apikeyrepository.Service
ScheduleService *schedule.Service
SettingsService *settings.Service
SSLSettingsService *ssl.Service
StackService *stack.Service
TagService *tag.Service
TeamMembershipService *teammembership.Service
TeamService *team.Service
TunnelServerService *tunnelserver.Service
UserService *user.Service
VersionService *version.Service
WebhookService *webhook.Service
}
func (store *Store) version() (int, error) {
version, err := store.VersionService.DBVersion()
if err == errors.ErrObjectNotFound {
version = 0
}
return version, err
}
func (store *Store) edition() portainer.SoftwareEdition {
edition, err := store.VersionService.Edition()
if err == errors.ErrObjectNotFound {
edition = portainer.PortainerCE
}
return edition
}
// NewStore initializes a new Store and the associated services
func NewStore(storePath string, fileService portainer.FileService) *Store {
return &Store{
path: storePath,
fileService: fileService,
isNew: true,
connection: &internal.DbConnection{},
}
}
// Open opens and initializes the BoltDB database.
func (store *Store) Open() error {
databasePath := path.Join(store.path, databaseFileName)
db, err := bolt.Open(databasePath, 0600, &bolt.Options{Timeout: 1 * time.Second})
if err != nil {
return err
}
store.connection.DB = db
err = store.initServices()
if err != nil {
return err
}
// if we have DBVersion in the database then ensure we flag this as NOT a new store
if _, err := store.VersionService.DBVersion(); err == nil {
store.isNew = false
}
return nil
}
// Close closes the BoltDB database.
// Safe to being called multiple times.
func (store *Store) Close() error {
if store.connection.DB != nil {
return store.connection.Close()
}
return nil
}
// IsNew returns true if the database was just created and false if it is re-using
// existing data.
func (store *Store) IsNew() bool {
return store.isNew
}
// BackupTo backs up db to a provided writer.
// It does hot backup and doesn't block other database reads and writes
func (store *Store) BackupTo(w io.Writer) error {
return store.connection.View(func(tx *bolt.Tx) error {
_, err := tx.WriteTo(w)
return err
})
}
// CheckCurrentEdition checks if current edition is community edition
func (store *Store) CheckCurrentEdition() error {
if store.edition() != portainer.PortainerCE {
return errors.ErrWrongDBEdition
}
return nil
}

View File

@ -1,94 +0,0 @@
package edgegroup
import (
"github.com/boltdb/bolt"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/internal"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "edgegroups"
)
// Service represents a service for managing Edge group data.
type Service struct {
connection *internal.DbConnection
}
// NewService creates a new instance of a service.
func NewService(connection *internal.DbConnection) (*Service, error) {
err := internal.CreateBucket(connection, BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// EdgeGroups return an array containing all the Edge groups.
func (service *Service) EdgeGroups() ([]portainer.EdgeGroup, error) {
var groups = make([]portainer.EdgeGroup, 0)
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var group portainer.EdgeGroup
err := internal.UnmarshalObjectWithJsoniter(v, &group)
if err != nil {
return err
}
groups = append(groups, group)
}
return nil
})
return groups, err
}
// EdgeGroup returns an Edge group by ID.
func (service *Service) EdgeGroup(ID portainer.EdgeGroupID) (*portainer.EdgeGroup, error) {
var group portainer.EdgeGroup
identifier := internal.Itob(int(ID))
err := internal.GetObject(service.connection, BucketName, identifier, &group)
if err != nil {
return nil, err
}
return &group, nil
}
// UpdateEdgeGroup updates an Edge group.
func (service *Service) UpdateEdgeGroup(ID portainer.EdgeGroupID, group *portainer.EdgeGroup) error {
identifier := internal.Itob(int(ID))
return internal.UpdateObject(service.connection, BucketName, identifier, group)
}
// DeleteEdgeGroup deletes an Edge group.
func (service *Service) DeleteEdgeGroup(ID portainer.EdgeGroupID) error {
identifier := internal.Itob(int(ID))
return internal.DeleteObject(service.connection, BucketName, identifier)
}
// CreateEdgeGroup assign an ID to a new Edge group and saves it.
func (service *Service) CreateEdgeGroup(group *portainer.EdgeGroup) error {
return service.connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
id, _ := bucket.NextSequence()
group.ID = portainer.EdgeGroupID(id)
data, err := internal.MarshalObject(group)
if err != nil {
return err
}
return bucket.Put(internal.Itob(int(group.ID)), data)
})
}

View File

@ -1,101 +0,0 @@
package edgejob
import (
"github.com/boltdb/bolt"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/internal"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "edgejobs"
)
// Service represents a service for managing edge jobs data.
type Service struct {
connection *internal.DbConnection
}
// NewService creates a new instance of a service.
func NewService(connection *internal.DbConnection) (*Service, error) {
err := internal.CreateBucket(connection, BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// EdgeJobs returns a list of Edge jobs
func (service *Service) EdgeJobs() ([]portainer.EdgeJob, error) {
var edgeJobs = make([]portainer.EdgeJob, 0)
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var edgeJob portainer.EdgeJob
err := internal.UnmarshalObject(v, &edgeJob)
if err != nil {
return err
}
edgeJobs = append(edgeJobs, edgeJob)
}
return nil
})
return edgeJobs, err
}
// EdgeJob returns an Edge job by ID
func (service *Service) EdgeJob(ID portainer.EdgeJobID) (*portainer.EdgeJob, error) {
var edgeJob portainer.EdgeJob
identifier := internal.Itob(int(ID))
err := internal.GetObject(service.connection, BucketName, identifier, &edgeJob)
if err != nil {
return nil, err
}
return &edgeJob, nil
}
// CreateEdgeJob creates a new Edge job
func (service *Service) CreateEdgeJob(edgeJob *portainer.EdgeJob) error {
return service.connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
if edgeJob.ID == 0 {
id, _ := bucket.NextSequence()
edgeJob.ID = portainer.EdgeJobID(id)
}
data, err := internal.MarshalObject(edgeJob)
if err != nil {
return err
}
return bucket.Put(internal.Itob(int(edgeJob.ID)), data)
})
}
// UpdateEdgeJob updates an Edge job by ID
func (service *Service) UpdateEdgeJob(ID portainer.EdgeJobID, edgeJob *portainer.EdgeJob) error {
identifier := internal.Itob(int(ID))
return internal.UpdateObject(service.connection, BucketName, identifier, edgeJob)
}
// DeleteEdgeJob deletes an Edge job
func (service *Service) DeleteEdgeJob(ID portainer.EdgeJobID) error {
identifier := internal.Itob(int(ID))
return internal.DeleteObject(service.connection, BucketName, identifier)
}
// GetNextIdentifier returns the next identifier for an environment(endpoint).
func (service *Service) GetNextIdentifier() int {
return internal.GetNextIdentifier(service.connection, BucketName)
}

View File

@ -1,101 +0,0 @@
package edgestack
import (
"github.com/boltdb/bolt"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/internal"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "edge_stack"
)
// Service represents a service for managing Edge stack data.
type Service struct {
connection *internal.DbConnection
}
// NewService creates a new instance of a service.
func NewService(connection *internal.DbConnection) (*Service, error) {
err := internal.CreateBucket(connection, BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// EdgeStacks returns an array containing all edge stacks
func (service *Service) EdgeStacks() ([]portainer.EdgeStack, error) {
var stacks = make([]portainer.EdgeStack, 0)
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var stack portainer.EdgeStack
err := internal.UnmarshalObject(v, &stack)
if err != nil {
return err
}
stacks = append(stacks, stack)
}
return nil
})
return stacks, err
}
// EdgeStack returns an Edge stack by ID.
func (service *Service) EdgeStack(ID portainer.EdgeStackID) (*portainer.EdgeStack, error) {
var stack portainer.EdgeStack
identifier := internal.Itob(int(ID))
err := internal.GetObject(service.connection, BucketName, identifier, &stack)
if err != nil {
return nil, err
}
return &stack, nil
}
// CreateEdgeStack assign an ID to a new Edge stack and saves it.
func (service *Service) CreateEdgeStack(edgeStack *portainer.EdgeStack) error {
return service.connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
if edgeStack.ID == 0 {
id, _ := bucket.NextSequence()
edgeStack.ID = portainer.EdgeStackID(id)
}
data, err := internal.MarshalObject(edgeStack)
if err != nil {
return err
}
return bucket.Put(internal.Itob(int(edgeStack.ID)), data)
})
}
// UpdateEdgeStack updates an Edge stack.
func (service *Service) UpdateEdgeStack(ID portainer.EdgeStackID, edgeStack *portainer.EdgeStack) error {
identifier := internal.Itob(int(ID))
return internal.UpdateObject(service.connection, BucketName, identifier, edgeStack)
}
// DeleteEdgeStack deletes an Edge stack.
func (service *Service) DeleteEdgeStack(ID portainer.EdgeStackID) error {
identifier := internal.Itob(int(ID))
return internal.DeleteObject(service.connection, BucketName, identifier)
}
// GetNextIdentifier returns the next identifier for an environment(endpoint).
func (service *Service) GetNextIdentifier() int {
return internal.GetNextIdentifier(service.connection, BucketName)
}

View File

@ -1,145 +0,0 @@
package endpoint
import (
"github.com/boltdb/bolt"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/internal"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "endpoints"
)
// Service represents a service for managing environment(endpoint) data.
type Service struct {
connection *internal.DbConnection
}
// NewService creates a new instance of a service.
func NewService(connection *internal.DbConnection) (*Service, error) {
err := internal.CreateBucket(connection, BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// Endpoint returns an environment(endpoint) by ID.
func (service *Service) Endpoint(ID portainer.EndpointID) (*portainer.Endpoint, error) {
var endpoint portainer.Endpoint
identifier := internal.Itob(int(ID))
err := internal.GetObject(service.connection, BucketName, identifier, &endpoint)
if err != nil {
return nil, err
}
return &endpoint, nil
}
// UpdateEndpoint updates an environment(endpoint).
func (service *Service) UpdateEndpoint(ID portainer.EndpointID, endpoint *portainer.Endpoint) error {
identifier := internal.Itob(int(ID))
return internal.UpdateObject(service.connection, BucketName, identifier, endpoint)
}
// DeleteEndpoint deletes an environment(endpoint).
func (service *Service) DeleteEndpoint(ID portainer.EndpointID) error {
identifier := internal.Itob(int(ID))
return internal.DeleteObject(service.connection, BucketName, identifier)
}
// Endpoints return an array containing all the environments(endpoints).
func (service *Service) Endpoints() ([]portainer.Endpoint, error) {
var endpoints = make([]portainer.Endpoint, 0)
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var endpoint portainer.Endpoint
err := internal.UnmarshalObjectWithJsoniter(v, &endpoint)
if err != nil {
return err
}
endpoints = append(endpoints, endpoint)
}
return nil
})
return endpoints, err
}
// CreateEndpoint assign an ID to a new environment(endpoint) and saves it.
func (service *Service) CreateEndpoint(endpoint *portainer.Endpoint) error {
return service.connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
// We manually manage sequences for environments(endpoints)
err := bucket.SetSequence(uint64(endpoint.ID))
if err != nil {
return err
}
data, err := internal.MarshalObject(endpoint)
if err != nil {
return err
}
return bucket.Put(internal.Itob(int(endpoint.ID)), data)
})
}
// GetNextIdentifier returns the next identifier for an environment(endpoint).
func (service *Service) GetNextIdentifier() int {
return internal.GetNextIdentifier(service.connection, BucketName)
}
// Synchronize creates, updates and deletes environments(endpoints) inside a single transaction.
func (service *Service) Synchronize(toCreate, toUpdate, toDelete []*portainer.Endpoint) error {
return service.connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
for _, endpoint := range toCreate {
id, _ := bucket.NextSequence()
endpoint.ID = portainer.EndpointID(id)
data, err := internal.MarshalObject(endpoint)
if err != nil {
return err
}
err = bucket.Put(internal.Itob(int(endpoint.ID)), data)
if err != nil {
return err
}
}
for _, endpoint := range toUpdate {
data, err := internal.MarshalObject(endpoint)
if err != nil {
return err
}
err = bucket.Put(internal.Itob(int(endpoint.ID)), data)
if err != nil {
return err
}
}
for _, endpoint := range toDelete {
err := bucket.Delete(internal.Itob(int(endpoint.ID)))
if err != nil {
return err
}
}
return nil
})
}

View File

@ -1,95 +0,0 @@
package endpointgroup
import (
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/internal"
"github.com/boltdb/bolt"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "endpoint_groups"
)
// Service represents a service for managing environment(endpoint) data.
type Service struct {
connection *internal.DbConnection
}
// NewService creates a new instance of a service.
func NewService(connection *internal.DbConnection) (*Service, error) {
err := internal.CreateBucket(connection, BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// EndpointGroup returns an environment(endpoint) group by ID.
func (service *Service) EndpointGroup(ID portainer.EndpointGroupID) (*portainer.EndpointGroup, error) {
var endpointGroup portainer.EndpointGroup
identifier := internal.Itob(int(ID))
err := internal.GetObject(service.connection, BucketName, identifier, &endpointGroup)
if err != nil {
return nil, err
}
return &endpointGroup, nil
}
// UpdateEndpointGroup updates an environment(endpoint) group.
func (service *Service) UpdateEndpointGroup(ID portainer.EndpointGroupID, endpointGroup *portainer.EndpointGroup) error {
identifier := internal.Itob(int(ID))
return internal.UpdateObject(service.connection, BucketName, identifier, endpointGroup)
}
// DeleteEndpointGroup deletes an environment(endpoint) group.
func (service *Service) DeleteEndpointGroup(ID portainer.EndpointGroupID) error {
identifier := internal.Itob(int(ID))
return internal.DeleteObject(service.connection, BucketName, identifier)
}
// EndpointGroups return an array containing all the environment(endpoint) groups.
func (service *Service) EndpointGroups() ([]portainer.EndpointGroup, error) {
var endpointGroups = make([]portainer.EndpointGroup, 0)
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var endpointGroup portainer.EndpointGroup
err := internal.UnmarshalObject(v, &endpointGroup)
if err != nil {
return err
}
endpointGroups = append(endpointGroups, endpointGroup)
}
return nil
})
return endpointGroups, err
}
// CreateEndpointGroup assign an ID to a new environment(endpoint) group and saves it.
func (service *Service) CreateEndpointGroup(endpointGroup *portainer.EndpointGroup) error {
return service.connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
id, _ := bucket.NextSequence()
endpointGroup.ID = portainer.EndpointGroupID(id)
data, err := internal.MarshalObject(endpointGroup)
if err != nil {
return err
}
return bucket.Put(internal.Itob(int(endpointGroup.ID)), data)
})
}

View File

@ -1,68 +0,0 @@
package endpointrelation
import (
"github.com/boltdb/bolt"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/internal"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "endpoint_relations"
)
// Service represents a service for managing environment(endpoint) relation data.
type Service struct {
connection *internal.DbConnection
}
// NewService creates a new instance of a service.
func NewService(connection *internal.DbConnection) (*Service, error) {
err := internal.CreateBucket(connection, BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// EndpointRelation returns a Environment(Endpoint) relation object by EndpointID
func (service *Service) EndpointRelation(endpointID portainer.EndpointID) (*portainer.EndpointRelation, error) {
var endpointRelation portainer.EndpointRelation
identifier := internal.Itob(int(endpointID))
err := internal.GetObject(service.connection, BucketName, identifier, &endpointRelation)
if err != nil {
return nil, err
}
return &endpointRelation, nil
}
// CreateEndpointRelation saves endpointRelation
func (service *Service) CreateEndpointRelation(endpointRelation *portainer.EndpointRelation) error {
return service.connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
data, err := internal.MarshalObject(endpointRelation)
if err != nil {
return err
}
return bucket.Put(internal.Itob(int(endpointRelation.EndpointID)), data)
})
}
// UpdateEndpointRelation updates an Environment(Endpoint) relation object
func (service *Service) UpdateEndpointRelation(EndpointID portainer.EndpointID, endpointRelation *portainer.EndpointRelation) error {
identifier := internal.Itob(int(EndpointID))
return internal.UpdateObject(service.connection, BucketName, identifier, endpointRelation)
}
// DeleteEndpointRelation deletes an Environment(Endpoint) relation object
func (service *Service) DeleteEndpointRelation(EndpointID portainer.EndpointID) error {
identifier := internal.Itob(int(EndpointID))
return internal.DeleteObject(service.connection, BucketName, identifier)
}

View File

@ -1,73 +0,0 @@
package helmuserrepository
import (
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/internal"
"github.com/boltdb/bolt"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "helm_user_repository"
)
// Service represents a service for managing environment(endpoint) data.
type Service struct {
connection *internal.DbConnection
}
// NewService creates a new instance of a service.
func NewService(connection *internal.DbConnection) (*Service, error) {
err := internal.CreateBucket(connection, BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// HelmUserRepositoryByUserID return an array containing all the HelmUserRepository objects where the specified userID is present.
func (service *Service) HelmUserRepositoryByUserID(userID portainer.UserID) ([]portainer.HelmUserRepository, error) {
var result = make([]portainer.HelmUserRepository, 0)
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var record portainer.HelmUserRepository
err := internal.UnmarshalObject(v, &record)
if err != nil {
return err
}
if record.UserID == userID {
result = append(result, record)
}
}
return nil
})
return result, err
}
// CreateHelmUserRepository creates a new HelmUserRepository object.
func (service *Service) CreateHelmUserRepository(record *portainer.HelmUserRepository) error {
return service.connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
id, _ := bucket.NextSequence()
record.ID = portainer.HelmUserRepositoryID(id)
data, err := internal.MarshalObject(record)
if err != nil {
return err
}
return bucket.Put(internal.Itob(int(record.ID)), data)
})
}

View File

@ -1,100 +0,0 @@
package internal
import (
"encoding/binary"
"github.com/boltdb/bolt"
"github.com/portainer/portainer/api/bolt/errors"
)
type DbConnection struct {
*bolt.DB
}
// Itob returns an 8-byte big endian representation of v.
// This function is typically used for encoding integer IDs to byte slices
// so that they can be used as BoltDB keys.
func Itob(v int) []byte {
b := make([]byte, 8)
binary.BigEndian.PutUint64(b, uint64(v))
return b
}
// CreateBucket is a generic function used to create a bucket inside a bolt database.
func CreateBucket(connection *DbConnection, bucketName string) error {
return connection.Update(func(tx *bolt.Tx) error {
_, err := tx.CreateBucketIfNotExists([]byte(bucketName))
if err != nil {
return err
}
return nil
})
}
// GetObject is a generic function used to retrieve an unmarshalled object from a bolt database.
func GetObject(connection *DbConnection, bucketName string, key []byte, object interface{}) error {
var data []byte
err := connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(bucketName))
value := bucket.Get(key)
if value == nil {
return errors.ErrObjectNotFound
}
data = make([]byte, len(value))
copy(data, value)
return nil
})
if err != nil {
return err
}
return UnmarshalObject(data, object)
}
// UpdateObject is a generic function used to update an object inside a bolt database.
func UpdateObject(connection *DbConnection, bucketName string, key []byte, object interface{}) error {
return connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(bucketName))
data, err := MarshalObject(object)
if err != nil {
return err
}
err = bucket.Put(key, data)
if err != nil {
return err
}
return nil
})
}
// DeleteObject is a generic function used to delete an object inside a bolt database.
func DeleteObject(connection *DbConnection, bucketName string, key []byte) error {
return connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(bucketName))
return bucket.Delete(key)
})
}
// GetNextIdentifier is a generic function that returns the specified bucket identifier incremented by 1.
func GetNextIdentifier(connection *DbConnection, bucketName string) int {
var identifier int
connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(bucketName))
id, err := bucket.NextSequence()
if err != nil {
return err
}
identifier = int(id)
return nil
})
return identifier
}

View File

@ -1 +0,0 @@
package log

View File

@ -1,37 +0,0 @@
package migrator
import (
"github.com/boltdb/bolt"
"github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/errors"
"github.com/portainer/portainer/api/bolt/user"
)
func (m *Migrator) updateAdminUserToDBVersion1() error {
u, err := m.userService.UserByUsername("admin")
if err == nil {
admin := &portainer.User{
Username: "admin",
Password: u.Password,
Role: portainer.AdministratorRole,
}
err = m.userService.CreateUser(admin)
if err != nil {
return err
}
err = m.removeLegacyAdminUser()
if err != nil {
return err
}
} else if err != nil && err != errors.ErrObjectNotFound {
return err
}
return nil
}
func (m *Migrator) removeLegacyAdminUser() error {
return m.db.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(user.BucketName))
return bucket.Delete([]byte("admin"))
})
}

View File

@ -1,103 +0,0 @@
package migrator
import (
"github.com/boltdb/bolt"
"github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/internal"
)
func (m *Migrator) updateResourceControlsToDBVersion2() error {
legacyResourceControls, err := m.retrieveLegacyResourceControls()
if err != nil {
return err
}
for _, resourceControl := range legacyResourceControls {
resourceControl.SubResourceIDs = []string{}
resourceControl.TeamAccesses = []portainer.TeamResourceAccess{}
owner, err := m.userService.User(resourceControl.OwnerID)
if err != nil {
return err
}
if owner.Role == portainer.AdministratorRole {
resourceControl.AdministratorsOnly = true
resourceControl.UserAccesses = []portainer.UserResourceAccess{}
} else {
resourceControl.AdministratorsOnly = false
userAccess := portainer.UserResourceAccess{
UserID: resourceControl.OwnerID,
AccessLevel: portainer.ReadWriteAccessLevel,
}
resourceControl.UserAccesses = []portainer.UserResourceAccess{userAccess}
}
err = m.resourceControlService.CreateResourceControl(&resourceControl)
if err != nil {
return err
}
}
return nil
}
func (m *Migrator) updateEndpointsToDBVersion2() error {
legacyEndpoints, err := m.endpointService.Endpoints()
if err != nil {
return err
}
for _, endpoint := range legacyEndpoints {
endpoint.AuthorizedTeams = []portainer.TeamID{}
err = m.endpointService.UpdateEndpoint(endpoint.ID, &endpoint)
if err != nil {
return err
}
}
return nil
}
func (m *Migrator) retrieveLegacyResourceControls() ([]portainer.ResourceControl, error) {
legacyResourceControls := make([]portainer.ResourceControl, 0)
err := m.db.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte("containerResourceControl"))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var resourceControl portainer.ResourceControl
err := internal.UnmarshalObject(v, &resourceControl)
if err != nil {
return err
}
resourceControl.Type = portainer.ContainerResourceControl
legacyResourceControls = append(legacyResourceControls, resourceControl)
}
bucket = tx.Bucket([]byte("serviceResourceControl"))
cursor = bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var resourceControl portainer.ResourceControl
err := internal.UnmarshalObject(v, &resourceControl)
if err != nil {
return err
}
resourceControl.Type = portainer.ServiceResourceControl
legacyResourceControls = append(legacyResourceControls, resourceControl)
}
bucket = tx.Bucket([]byte("volumeResourceControl"))
cursor = bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var resourceControl portainer.ResourceControl
err := internal.UnmarshalObject(v, &resourceControl)
if err != nil {
return err
}
resourceControl.Type = portainer.VolumeResourceControl
legacyResourceControls = append(legacyResourceControls, resourceControl)
}
return nil
})
return legacyResourceControls, err
}

View File

@ -1,28 +0,0 @@
package migrator
import "github.com/portainer/portainer/api"
func (m *Migrator) updateEndpointsToVersion11() error {
legacyEndpoints, err := m.endpointService.Endpoints()
if err != nil {
return err
}
for _, endpoint := range legacyEndpoints {
if endpoint.Type == portainer.AgentOnDockerEnvironment {
endpoint.TLSConfig.TLS = true
endpoint.TLSConfig.TLSSkipVerify = true
} else {
if endpoint.TLSConfig.TLSSkipVerify && !endpoint.TLSConfig.TLS {
endpoint.TLSConfig.TLSSkipVerify = false
}
}
err = m.endpointService.UpdateEndpoint(endpoint.ID, &endpoint)
if err != nil {
return err
}
}
return nil
}

View File

@ -1,127 +0,0 @@
package migrator
import (
"strconv"
"strings"
"github.com/boltdb/bolt"
"github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/internal"
"github.com/portainer/portainer/api/bolt/stack"
)
func (m *Migrator) updateEndpointsToVersion12() error {
legacyEndpoints, err := m.endpointService.Endpoints()
if err != nil {
return err
}
for _, endpoint := range legacyEndpoints {
endpoint.Tags = []string{}
err = m.endpointService.UpdateEndpoint(endpoint.ID, &endpoint)
if err != nil {
return err
}
}
return nil
}
func (m *Migrator) updateEndpointGroupsToVersion12() error {
legacyEndpointGroups, err := m.endpointGroupService.EndpointGroups()
if err != nil {
return err
}
for _, group := range legacyEndpointGroups {
group.Tags = []string{}
err = m.endpointGroupService.UpdateEndpointGroup(group.ID, &group)
if err != nil {
return err
}
}
return nil
}
type legacyStack struct {
ID string `json:"Id"`
Name string `json:"Name"`
EndpointID portainer.EndpointID `json:"EndpointId"`
SwarmID string `json:"SwarmId"`
EntryPoint string `json:"EntryPoint"`
Env []portainer.Pair `json:"Env"`
ProjectPath string
}
func (m *Migrator) updateStacksToVersion12() error {
legacyStacks, err := m.retrieveLegacyStacks()
if err != nil {
return err
}
for _, legacyStack := range legacyStacks {
err := m.convertLegacyStack(&legacyStack)
if err != nil {
return err
}
}
return nil
}
func (m *Migrator) convertLegacyStack(s *legacyStack) error {
stackID := m.stackService.GetNextIdentifier()
stack := &portainer.Stack{
ID: portainer.StackID(stackID),
Name: s.Name,
Type: portainer.DockerSwarmStack,
SwarmID: s.SwarmID,
EndpointID: 0,
EntryPoint: s.EntryPoint,
Env: s.Env,
}
stack.ProjectPath = strings.Replace(s.ProjectPath, s.ID, strconv.Itoa(stackID), 1)
err := m.fileService.Rename(s.ProjectPath, stack.ProjectPath)
if err != nil {
return err
}
err = m.deleteLegacyStack(s.ID)
if err != nil {
return err
}
return m.stackService.CreateStack(stack)
}
func (m *Migrator) deleteLegacyStack(legacyID string) error {
return m.db.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(stack.BucketName))
return bucket.Delete([]byte(legacyID))
})
}
func (m *Migrator) retrieveLegacyStacks() ([]legacyStack, error) {
var legacyStacks = make([]legacyStack, 0)
err := m.db.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(stack.BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var stack legacyStack
err := internal.UnmarshalObject(v, &stack)
if err != nil {
return err
}
legacyStacks = append(legacyStacks, stack)
}
return nil
})
return legacyStacks, err
}

View File

@ -1,17 +0,0 @@
package migrator
import "github.com/portainer/portainer/api"
func (m *Migrator) updateSettingsToVersion13() error {
legacySettings, err := m.settingsService.Settings()
if err != nil {
return err
}
legacySettings.LDAPSettings.AutoCreateUsers = false
legacySettings.LDAPSettings.GroupSearchSettings = []portainer.LDAPGroupSearchSettings{
portainer.LDAPGroupSearchSettings{},
}
return m.settingsService.UpdateSettings(legacySettings)
}

View File

@ -1,19 +0,0 @@
package migrator
func (m *Migrator) updateResourceControlsToDBVersion14() error {
resourceControls, err := m.resourceControlService.ResourceControls()
if err != nil {
return err
}
for _, resourceControl := range resourceControls {
if resourceControl.AdministratorsOnly == true {
err = m.resourceControlService.DeleteResourceControl(resourceControl.ID)
if err != nil {
return err
}
}
}
return nil
}

View File

@ -1,16 +0,0 @@
package migrator
func (m *Migrator) updateSettingsToDBVersion15() error {
legacySettings, err := m.settingsService.Settings()
if err != nil {
return err
}
legacySettings.EnableHostManagementFeatures = false
return m.settingsService.UpdateSettings(legacySettings)
}
func (m *Migrator) updateTemplatesToVersion15() error {
// Removed with the entire template management layer, part of https://github.com/portainer/portainer/issues/3707
return nil
}

View File

@ -1,14 +0,0 @@
package migrator
func (m *Migrator) updateSettingsToDBVersion16() error {
legacySettings, err := m.settingsService.Settings()
if err != nil {
return err
}
if legacySettings.SnapshotInterval == "" {
legacySettings.SnapshotInterval = "5m"
}
return m.settingsService.UpdateSettings(legacySettings)
}

View File

@ -1,19 +0,0 @@
package migrator
func (m *Migrator) updateExtensionsToDBVersion17() error {
legacyExtensions, err := m.extensionService.Extensions()
if err != nil {
return err
}
for _, extension := range legacyExtensions {
extension.License.Valid = true
err = m.extensionService.Persist(&extension)
if err != nil {
return err
}
}
return nil
}

View File

@ -1,20 +0,0 @@
package migrator
import "github.com/portainer/portainer/api"
func (m *Migrator) updateSettingsToDBVersion3() error {
legacySettings, err := m.settingsService.Settings()
if err != nil {
return err
}
legacySettings.AuthenticationMethod = portainer.AuthenticationInternal
legacySettings.LDAPSettings = portainer.LDAPSettings{
TLSConfig: portainer.TLSConfiguration{},
SearchSettings: []portainer.LDAPSearchSettings{
portainer.LDAPSearchSettings{},
},
}
return m.settingsService.UpdateSettings(legacySettings)
}

View File

@ -1,95 +0,0 @@
package migrator
import (
"os"
"path"
"testing"
"time"
"github.com/boltdb/bolt"
"github.com/portainer/portainer/api/bolt/internal"
"github.com/portainer/portainer/api/bolt/settings"
)
var (
testingDBStorePath string
testingDBFileName string
dummyLogoURL string
dbConn *bolt.DB
settingsService *settings.Service
)
// initTestingDBConn creates a raw bolt DB connection
// for unit testing usage only since using NewStore will cause cycle import inside migrator pkg
func initTestingDBConn(storePath, fileName string) (*bolt.DB, error) {
databasePath := path.Join(storePath, fileName)
dbConn, err := bolt.Open(databasePath, 0600, &bolt.Options{Timeout: 1 * time.Second})
if err != nil {
return nil, err
}
return dbConn, nil
}
// initTestingDBConn creates a settings service with raw bolt DB connection
// for unit testing usage only since using NewStore will cause cycle import inside migrator pkg
func initTestingSettingsService(dbConn *bolt.DB, preSetObj map[string]interface{}) (*settings.Service, error) {
internalDBConn := &internal.DbConnection{
DB: dbConn,
}
settingsService, err := settings.NewService(internalDBConn)
if err != nil {
return nil, err
}
//insert a obj
if err := internal.UpdateObject(internalDBConn, "settings", []byte("SETTINGS"), preSetObj); err != nil {
return nil, err
}
return settingsService, nil
}
func setup() error {
testingDBStorePath, _ = os.Getwd()
testingDBFileName = "portainer-ee-mig-30.db"
dummyLogoURL = "example.com"
var err error
dbConn, err = initTestingDBConn(testingDBStorePath, testingDBFileName)
if err != nil {
return err
}
dummySettingsObj := map[string]interface{}{
"LogoURL": dummyLogoURL,
}
settingsService, err = initTestingSettingsService(dbConn, dummySettingsObj)
if err != nil {
return err
}
return nil
}
func TestMigrateSettings(t *testing.T) {
if err := setup(); err != nil {
t.Errorf("failed to complete testing setups, err: %v", err)
}
defer dbConn.Close()
defer os.Remove(testingDBFileName)
m := &Migrator{
db: dbConn,
settingsService: settingsService,
}
if err := m.migrateSettingsToDB30(); err != nil {
t.Errorf("failed to update settings: %v", err)
}
updatedSettings, err := m.settingsService.Settings()
if err != nil {
t.Errorf("failed to retrieve the updated settings: %v", err)
}
if updatedSettings.LogoURL != dummyLogoURL {
t.Errorf("unexpected value changes in the updated settings, want LogoURL value: %s, got LogoURL value: %s", dummyLogoURL, updatedSettings.LogoURL)
}
if updatedSettings.OAuthSettings.SSO != false {
t.Errorf("unexpected default OAuth SSO setting, want: false, got: %t", updatedSettings.OAuthSettings.SSO)
}
if updatedSettings.OAuthSettings.LogoutURI != "" {
t.Errorf("unexpected default OAuth HideInternalAuth setting, want:, got: %s", updatedSettings.OAuthSettings.LogoutURI)
}
}

View File

@ -1,28 +0,0 @@
package migrator
import "github.com/portainer/portainer/api"
func (m *Migrator) updateEndpointsToDBVersion4() error {
legacyEndpoints, err := m.endpointService.Endpoints()
if err != nil {
return err
}
for _, endpoint := range legacyEndpoints {
endpoint.TLSConfig = portainer.TLSConfiguration{}
if endpoint.TLS {
endpoint.TLSConfig.TLS = true
endpoint.TLSConfig.TLSSkipVerify = false
endpoint.TLSConfig.TLSCACertPath = endpoint.TLSCACertPath
endpoint.TLSConfig.TLSCertPath = endpoint.TLSCertPath
endpoint.TLSConfig.TLSKeyPath = endpoint.TLSKeyPath
}
err = m.endpointService.UpdateEndpoint(endpoint.ID, &endpoint)
if err != nil {
return err
}
}
return nil
}

View File

@ -1,108 +0,0 @@
package migrator
import (
"os"
"path"
"testing"
"time"
"github.com/boltdb/bolt"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/dockerhub"
"github.com/portainer/portainer/api/bolt/endpoint"
"github.com/portainer/portainer/api/bolt/internal"
"github.com/portainer/portainer/api/bolt/registry"
"github.com/stretchr/testify/assert"
)
const (
db35TestFile = "portainer-mig-35.db"
username = "portainer"
password = "password"
)
func setupDB35Test(t *testing.T) *Migrator {
is := assert.New(t)
dbConn, err := bolt.Open(path.Join(t.TempDir(), db35TestFile), 0600, &bolt.Options{Timeout: 1 * time.Second})
is.NoError(err, "failed to init testing DB connection")
// Create an old style dockerhub authenticated account
dockerhubService, err := dockerhub.NewService(&internal.DbConnection{DB: dbConn})
is.NoError(err, "failed to init testing registry service")
err = dockerhubService.UpdateDockerHub(&portainer.DockerHub{true, username, password})
is.NoError(err, "failed to create dockerhub account")
registryService, err := registry.NewService(&internal.DbConnection{DB: dbConn})
is.NoError(err, "failed to init testing registry service")
endpointService, err := endpoint.NewService(&internal.DbConnection{DB: dbConn})
is.NoError(err, "failed to init endpoint service")
m := &Migrator{
db: dbConn,
dockerhubService: dockerhubService,
registryService: registryService,
endpointService: endpointService,
}
return m
}
// TestUpdateDockerhubToDB32 tests a normal upgrade
func TestUpdateDockerhubToDB32(t *testing.T) {
is := assert.New(t)
m := setupDB35Test(t)
defer m.db.Close()
defer os.Remove(db35TestFile)
if err := m.updateDockerhubToDB32(); err != nil {
t.Errorf("failed to update settings: %v", err)
}
// Verify we have a single registry were created
registries, err := m.registryService.Registries()
is.NoError(err, "failed to read registries from the RegistryService")
is.Equal(len(registries), 1, "only one migrated registry expected")
}
// TestUpdateDockerhubToDB32_with_duplicate_migrations tests an upgrade where in earlier versions a broken migration
// created a large number of duplicate "dockerhub migrated" registry entries.
func TestUpdateDockerhubToDB32_with_duplicate_migrations(t *testing.T) {
is := assert.New(t)
m := setupDB35Test(t)
defer m.db.Close()
defer os.Remove(db35TestFile)
// Create lots of duplicate entries...
registry := &portainer.Registry{
Type: portainer.DockerHubRegistry,
Name: "Dockerhub (authenticated - migrated)",
URL: "docker.io",
Authentication: true,
Username: "portainer",
Password: "password",
RegistryAccesses: portainer.RegistryAccesses{},
}
for i := 1; i < 150; i++ {
err := m.registryService.CreateRegistry(registry)
assert.NoError(t, err, "create registry failed")
}
// Verify they were created
registries, err := m.registryService.Registries()
is.NoError(err, "failed to read registries from the RegistryService")
is.Condition(func() bool {
return len(registries) > 1
}, "expected multiple duplicate registry entries")
// Now run the migrator
if err := m.updateDockerhubToDB32(); err != nil {
t.Errorf("failed to update settings: %v", err)
}
// Verify we have a single registry were created
registries, err = m.registryService.Registries()
is.NoError(err, "failed to read registries from the RegistryService")
is.Equal(len(registries), 1, "only one migrated registry expected")
}

View File

@ -1,11 +0,0 @@
package migrator
func (m *Migrator) updateSettingsToVersion5() error {
legacySettings, err := m.settingsService.Settings()
if err != nil {
return err
}
legacySettings.AllowBindMountsForRegularUsers = true
return m.settingsService.UpdateSettings(legacySettings)
}

View File

@ -1,11 +0,0 @@
package migrator
func (m *Migrator) updateSettingsToVersion6() error {
legacySettings, err := m.settingsService.Settings()
if err != nil {
return err
}
legacySettings.AllowPrivilegedModeForRegularUsers = true
return m.settingsService.UpdateSettings(legacySettings)
}

View File

@ -1,11 +0,0 @@
package migrator
func (m *Migrator) updateSettingsToVersion7() error {
legacySettings, err := m.settingsService.Settings()
if err != nil {
return err
}
legacySettings.DisplayDonationHeader = true
return m.settingsService.UpdateSettings(legacySettings)
}

View File

@ -1,20 +0,0 @@
package migrator
import "github.com/portainer/portainer/api"
func (m *Migrator) updateEndpointsToVersion8() error {
legacyEndpoints, err := m.endpointService.Endpoints()
if err != nil {
return err
}
for _, endpoint := range legacyEndpoints {
endpoint.Extensions = []portainer.EndpointExtension{}
err = m.endpointService.UpdateEndpoint(endpoint.ID, &endpoint)
if err != nil {
return err
}
}
return nil
}

View File

@ -1,20 +0,0 @@
package migrator
import "github.com/portainer/portainer/api"
func (m *Migrator) updateEndpointsToVersion9() error {
legacyEndpoints, err := m.endpointService.Endpoints()
if err != nil {
return err
}
for _, endpoint := range legacyEndpoints {
endpoint.GroupID = portainer.EndpointGroupID(1)
err = m.endpointService.UpdateEndpoint(endpoint.ID, &endpoint)
if err != nil {
return err
}
}
return nil
}

View File

@ -1,20 +0,0 @@
package migrator
import "github.com/portainer/portainer/api"
func (m *Migrator) updateEndpointsToVersion10() error {
legacyEndpoints, err := m.endpointService.Endpoints()
if err != nil {
return err
}
for _, endpoint := range legacyEndpoints {
endpoint.Type = portainer.DockerEnvironment
err = m.endpointService.UpdateEndpoint(endpoint.ID, &endpoint)
if err != nil {
return err
}
}
return nil
}

View File

@ -1,95 +0,0 @@
package registry
import (
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/internal"
"github.com/boltdb/bolt"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "registries"
)
// Service represents a service for managing environment(endpoint) data.
type Service struct {
connection *internal.DbConnection
}
// NewService creates a new instance of a service.
func NewService(connection *internal.DbConnection) (*Service, error) {
err := internal.CreateBucket(connection, BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// Registry returns an registry by ID.
func (service *Service) Registry(ID portainer.RegistryID) (*portainer.Registry, error) {
var registry portainer.Registry
identifier := internal.Itob(int(ID))
err := internal.GetObject(service.connection, BucketName, identifier, &registry)
if err != nil {
return nil, err
}
return &registry, nil
}
// Registries returns an array containing all the registries.
func (service *Service) Registries() ([]portainer.Registry, error) {
var registries = make([]portainer.Registry, 0)
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var registry portainer.Registry
err := internal.UnmarshalObject(v, &registry)
if err != nil {
return err
}
registries = append(registries, registry)
}
return nil
})
return registries, err
}
// CreateRegistry creates a new registry.
func (service *Service) CreateRegistry(registry *portainer.Registry) error {
return service.connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
id, _ := bucket.NextSequence()
registry.ID = portainer.RegistryID(id)
data, err := internal.MarshalObject(registry)
if err != nil {
return err
}
return bucket.Put(internal.Itob(int(registry.ID)), data)
})
}
// UpdateRegistry updates an registry.
func (service *Service) UpdateRegistry(ID portainer.RegistryID, registry *portainer.Registry) error {
identifier := internal.Itob(int(ID))
return internal.UpdateObject(service.connection, BucketName, identifier, registry)
}
// DeleteRegistry deletes an registry.
func (service *Service) DeleteRegistry(ID portainer.RegistryID) error {
identifier := internal.Itob(int(ID))
return internal.DeleteObject(service.connection, BucketName, identifier)
}

View File

@ -1,131 +0,0 @@
package resourcecontrol
import (
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/internal"
"github.com/boltdb/bolt"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "resource_control"
)
// Service represents a service for managing environment(endpoint) data.
type Service struct {
connection *internal.DbConnection
}
// NewService creates a new instance of a service.
func NewService(connection *internal.DbConnection) (*Service, error) {
err := internal.CreateBucket(connection, BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// ResourceControl returns a ResourceControl object by ID
func (service *Service) ResourceControl(ID portainer.ResourceControlID) (*portainer.ResourceControl, error) {
var resourceControl portainer.ResourceControl
identifier := internal.Itob(int(ID))
err := internal.GetObject(service.connection, BucketName, identifier, &resourceControl)
if err != nil {
return nil, err
}
return &resourceControl, nil
}
// ResourceControlByResourceIDAndType returns a ResourceControl object by checking if the resourceID is equal
// to the main ResourceID or in SubResourceIDs. It also performs a check on the resource type. Return nil
// if no ResourceControl was found.
func (service *Service) ResourceControlByResourceIDAndType(resourceID string, resourceType portainer.ResourceControlType) (*portainer.ResourceControl, error) {
var resourceControl *portainer.ResourceControl
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var rc portainer.ResourceControl
err := internal.UnmarshalObject(v, &rc)
if err != nil {
return err
}
if rc.ResourceID == resourceID && rc.Type == resourceType {
resourceControl = &rc
break
}
for _, subResourceID := range rc.SubResourceIDs {
if subResourceID == resourceID {
resourceControl = &rc
break
}
}
}
return nil
})
return resourceControl, err
}
// ResourceControls returns all the ResourceControl objects
func (service *Service) ResourceControls() ([]portainer.ResourceControl, error) {
var rcs = make([]portainer.ResourceControl, 0)
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var resourceControl portainer.ResourceControl
err := internal.UnmarshalObject(v, &resourceControl)
if err != nil {
return err
}
rcs = append(rcs, resourceControl)
}
return nil
})
return rcs, err
}
// CreateResourceControl creates a new ResourceControl object
func (service *Service) CreateResourceControl(resourceControl *portainer.ResourceControl) error {
return service.connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
id, _ := bucket.NextSequence()
resourceControl.ID = portainer.ResourceControlID(id)
data, err := internal.MarshalObject(resourceControl)
if err != nil {
return err
}
return bucket.Put(internal.Itob(int(resourceControl.ID)), data)
})
}
// UpdateResourceControl saves a ResourceControl object.
func (service *Service) UpdateResourceControl(ID portainer.ResourceControlID, resourceControl *portainer.ResourceControl) error {
identifier := internal.Itob(int(ID))
return internal.UpdateObject(service.connection, BucketName, identifier, resourceControl)
}
// DeleteResourceControl deletes a ResourceControl object by ID
func (service *Service) DeleteResourceControl(ID portainer.ResourceControlID) error {
identifier := internal.Itob(int(ID))
return internal.DeleteObject(service.connection, BucketName, identifier)
}

View File

@ -1,89 +0,0 @@
package role
import (
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/internal"
"github.com/boltdb/bolt"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "roles"
)
// Service represents a service for managing environment(endpoint) data.
type Service struct {
connection *internal.DbConnection
}
// NewService creates a new instance of a service.
func NewService(connection *internal.DbConnection) (*Service, error) {
err := internal.CreateBucket(connection, BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// Role returns a Role by ID
func (service *Service) Role(ID portainer.RoleID) (*portainer.Role, error) {
var set portainer.Role
identifier := internal.Itob(int(ID))
err := internal.GetObject(service.connection, BucketName, identifier, &set)
if err != nil {
return nil, err
}
return &set, nil
}
// Roles return an array containing all the sets.
func (service *Service) Roles() ([]portainer.Role, error) {
var sets = make([]portainer.Role, 0)
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var set portainer.Role
err := internal.UnmarshalObject(v, &set)
if err != nil {
return err
}
sets = append(sets, set)
}
return nil
})
return sets, err
}
// CreateRole creates a new Role.
func (service *Service) CreateRole(role *portainer.Role) error {
return service.connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
id, _ := bucket.NextSequence()
role.ID = portainer.RoleID(id)
data, err := internal.MarshalObject(role)
if err != nil {
return err
}
return bucket.Put(internal.Itob(int(role.ID)), data)
})
}
// UpdateRole updates a role.
func (service *Service) UpdateRole(ID portainer.RoleID, role *portainer.Role) error {
identifier := internal.Itob(int(ID))
return internal.UpdateObject(service.connection, BucketName, identifier, role)
}

View File

@ -1,129 +0,0 @@
package schedule
import (
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/internal"
"github.com/boltdb/bolt"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "schedules"
)
// Service represents a service for managing schedule data.
type Service struct {
connection *internal.DbConnection
}
// NewService creates a new instance of a service.
func NewService(connection *internal.DbConnection) (*Service, error) {
err := internal.CreateBucket(connection, BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// Schedule returns a schedule by ID.
func (service *Service) Schedule(ID portainer.ScheduleID) (*portainer.Schedule, error) {
var schedule portainer.Schedule
identifier := internal.Itob(int(ID))
err := internal.GetObject(service.connection, BucketName, identifier, &schedule)
if err != nil {
return nil, err
}
return &schedule, nil
}
// UpdateSchedule updates a schedule.
func (service *Service) UpdateSchedule(ID portainer.ScheduleID, schedule *portainer.Schedule) error {
identifier := internal.Itob(int(ID))
return internal.UpdateObject(service.connection, BucketName, identifier, schedule)
}
// DeleteSchedule deletes a schedule.
func (service *Service) DeleteSchedule(ID portainer.ScheduleID) error {
identifier := internal.Itob(int(ID))
return internal.DeleteObject(service.connection, BucketName, identifier)
}
// Schedules return a array containing all the schedules.
func (service *Service) Schedules() ([]portainer.Schedule, error) {
var schedules = make([]portainer.Schedule, 0)
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var schedule portainer.Schedule
err := internal.UnmarshalObject(v, &schedule)
if err != nil {
return err
}
schedules = append(schedules, schedule)
}
return nil
})
return schedules, err
}
// SchedulesByJobType return a array containing all the schedules
// with the specified JobType.
func (service *Service) SchedulesByJobType(jobType portainer.JobType) ([]portainer.Schedule, error) {
var schedules = make([]portainer.Schedule, 0)
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var schedule portainer.Schedule
err := internal.UnmarshalObject(v, &schedule)
if err != nil {
return err
}
if schedule.JobType == jobType {
schedules = append(schedules, schedule)
}
}
return nil
})
return schedules, err
}
// CreateSchedule assign an ID to a new schedule and saves it.
func (service *Service) CreateSchedule(schedule *portainer.Schedule) error {
return service.connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
// We manually manage sequences for schedules
err := bucket.SetSequence(uint64(schedule.ID))
if err != nil {
return err
}
data, err := internal.MarshalObject(schedule)
if err != nil {
return err
}
return bucket.Put(internal.Itob(int(schedule.ID)), data)
})
}
// GetNextIdentifier returns the next identifier for a schedule.
func (service *Service) GetNextIdentifier() int {
return internal.GetNextIdentifier(service.connection, BucketName)
}

View File

@ -1,294 +0,0 @@
package bolt
import (
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/apikeyrepository"
"github.com/portainer/portainer/api/bolt/customtemplate"
"github.com/portainer/portainer/api/bolt/dockerhub"
"github.com/portainer/portainer/api/bolt/edgegroup"
"github.com/portainer/portainer/api/bolt/edgejob"
"github.com/portainer/portainer/api/bolt/edgestack"
"github.com/portainer/portainer/api/bolt/endpoint"
"github.com/portainer/portainer/api/bolt/endpointgroup"
"github.com/portainer/portainer/api/bolt/endpointrelation"
"github.com/portainer/portainer/api/bolt/extension"
"github.com/portainer/portainer/api/bolt/helmuserrepository"
"github.com/portainer/portainer/api/bolt/registry"
"github.com/portainer/portainer/api/bolt/resourcecontrol"
"github.com/portainer/portainer/api/bolt/role"
"github.com/portainer/portainer/api/bolt/schedule"
"github.com/portainer/portainer/api/bolt/settings"
"github.com/portainer/portainer/api/bolt/ssl"
"github.com/portainer/portainer/api/bolt/stack"
"github.com/portainer/portainer/api/bolt/tag"
"github.com/portainer/portainer/api/bolt/team"
"github.com/portainer/portainer/api/bolt/teammembership"
"github.com/portainer/portainer/api/bolt/tunnelserver"
"github.com/portainer/portainer/api/bolt/user"
"github.com/portainer/portainer/api/bolt/version"
"github.com/portainer/portainer/api/bolt/webhook"
)
func (store *Store) initServices() error {
authorizationsetService, err := role.NewService(store.connection)
if err != nil {
return err
}
store.RoleService = authorizationsetService
customTemplateService, err := customtemplate.NewService(store.connection)
if err != nil {
return err
}
store.CustomTemplateService = customTemplateService
dockerhubService, err := dockerhub.NewService(store.connection)
if err != nil {
return err
}
store.DockerHubService = dockerhubService
edgeStackService, err := edgestack.NewService(store.connection)
if err != nil {
return err
}
store.EdgeStackService = edgeStackService
edgeGroupService, err := edgegroup.NewService(store.connection)
if err != nil {
return err
}
store.EdgeGroupService = edgeGroupService
edgeJobService, err := edgejob.NewService(store.connection)
if err != nil {
return err
}
store.EdgeJobService = edgeJobService
endpointgroupService, err := endpointgroup.NewService(store.connection)
if err != nil {
return err
}
store.EndpointGroupService = endpointgroupService
endpointService, err := endpoint.NewService(store.connection)
if err != nil {
return err
}
store.EndpointService = endpointService
endpointRelationService, err := endpointrelation.NewService(store.connection)
if err != nil {
return err
}
store.EndpointRelationService = endpointRelationService
extensionService, err := extension.NewService(store.connection)
if err != nil {
return err
}
store.ExtensionService = extensionService
helmUserRepositoryService, err := helmuserrepository.NewService(store.connection)
if err != nil {
return err
}
store.HelmUserRepositoryService = helmUserRepositoryService
registryService, err := registry.NewService(store.connection)
if err != nil {
return err
}
store.RegistryService = registryService
resourcecontrolService, err := resourcecontrol.NewService(store.connection)
if err != nil {
return err
}
store.ResourceControlService = resourcecontrolService
settingsService, err := settings.NewService(store.connection)
if err != nil {
return err
}
store.SettingsService = settingsService
sslSettingsService, err := ssl.NewService(store.connection)
if err != nil {
return err
}
store.SSLSettingsService = sslSettingsService
stackService, err := stack.NewService(store.connection)
if err != nil {
return err
}
store.StackService = stackService
tagService, err := tag.NewService(store.connection)
if err != nil {
return err
}
store.TagService = tagService
teammembershipService, err := teammembership.NewService(store.connection)
if err != nil {
return err
}
store.TeamMembershipService = teammembershipService
teamService, err := team.NewService(store.connection)
if err != nil {
return err
}
store.TeamService = teamService
tunnelServerService, err := tunnelserver.NewService(store.connection)
if err != nil {
return err
}
store.TunnelServerService = tunnelServerService
userService, err := user.NewService(store.connection)
if err != nil {
return err
}
store.UserService = userService
apiKeyService, err := apikeyrepository.NewService(store.connection)
if err != nil {
return err
}
store.APIKeyRepositoryService = apiKeyService
versionService, err := version.NewService(store.connection)
if err != nil {
return err
}
store.VersionService = versionService
webhookService, err := webhook.NewService(store.connection)
if err != nil {
return err
}
store.WebhookService = webhookService
scheduleService, err := schedule.NewService(store.connection)
if err != nil {
return err
}
store.ScheduleService = scheduleService
return nil
}
// CustomTemplate gives access to the CustomTemplate data management layer
func (store *Store) CustomTemplate() portainer.CustomTemplateService {
return store.CustomTemplateService
}
// EdgeGroup gives access to the EdgeGroup data management layer
func (store *Store) EdgeGroup() portainer.EdgeGroupService {
return store.EdgeGroupService
}
// EdgeJob gives access to the EdgeJob data management layer
func (store *Store) EdgeJob() portainer.EdgeJobService {
return store.EdgeJobService
}
// EdgeStack gives access to the EdgeStack data management layer
func (store *Store) EdgeStack() portainer.EdgeStackService {
return store.EdgeStackService
}
// Environment(Endpoint) gives access to the Environment(Endpoint) data management layer
func (store *Store) Endpoint() portainer.EndpointService {
return store.EndpointService
}
// EndpointGroup gives access to the EndpointGroup data management layer
func (store *Store) EndpointGroup() portainer.EndpointGroupService {
return store.EndpointGroupService
}
// EndpointRelation gives access to the EndpointRelation data management layer
func (store *Store) EndpointRelation() portainer.EndpointRelationService {
return store.EndpointRelationService
}
// HelmUserRepository access the helm user repository settings
func (store *Store) HelmUserRepository() portainer.HelmUserRepositoryService {
return store.HelmUserRepositoryService
}
// Registry gives access to the Registry data management layer
func (store *Store) Registry() portainer.RegistryService {
return store.RegistryService
}
// ResourceControl gives access to the ResourceControl data management layer
func (store *Store) ResourceControl() portainer.ResourceControlService {
return store.ResourceControlService
}
// Role gives access to the Role data management layer
func (store *Store) Role() portainer.RoleService {
return store.RoleService
}
// APIKeyRepository gives access to the api-key data management layer
func (store *Store) APIKeyRepository() portainer.APIKeyRepository {
return store.APIKeyRepositoryService
}
// Settings gives access to the Settings data management layer
func (store *Store) Settings() portainer.SettingsService {
return store.SettingsService
}
// SSLSettings gives access to the SSL Settings data management layer
func (store *Store) SSLSettings() portainer.SSLSettingsService {
return store.SSLSettingsService
}
// Stack gives access to the Stack data management layer
func (store *Store) Stack() portainer.StackService {
return store.StackService
}
// Tag gives access to the Tag data management layer
func (store *Store) Tag() portainer.TagService {
return store.TagService
}
// TeamMembership gives access to the TeamMembership data management layer
func (store *Store) TeamMembership() portainer.TeamMembershipService {
return store.TeamMembershipService
}
// Team gives access to the Team data management layer
func (store *Store) Team() portainer.TeamService {
return store.TeamService
}
// TunnelServer gives access to the TunnelServer data management layer
func (store *Store) TunnelServer() portainer.TunnelServerService {
return store.TunnelServerService
}
// User gives access to the User data management layer
func (store *Store) User() portainer.UserService {
return store.UserService
}
// Version gives access to the Version data management layer
func (store *Store) Version() portainer.VersionService {
return store.VersionService
}
// Webhook gives access to the Webhook data management layer
func (store *Store) Webhook() portainer.WebhookService {
return store.WebhookService
}

View File

@ -1,236 +0,0 @@
package stack
import (
"strings"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/errors"
"github.com/portainer/portainer/api/bolt/internal"
"github.com/boltdb/bolt"
pkgerrors "github.com/pkg/errors"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "stacks"
)
// Service represents a service for managing environment(endpoint) data.
type Service struct {
connection *internal.DbConnection
}
// NewService creates a new instance of a service.
func NewService(connection *internal.DbConnection) (*Service, error) {
err := internal.CreateBucket(connection, BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// Stack returns a stack object by ID.
func (service *Service) Stack(ID portainer.StackID) (*portainer.Stack, error) {
var stack portainer.Stack
identifier := internal.Itob(int(ID))
err := internal.GetObject(service.connection, BucketName, identifier, &stack)
if err != nil {
return nil, err
}
return &stack, nil
}
// StackByName returns a stack object by name.
func (service *Service) StackByName(name string) (*portainer.Stack, error) {
var stack *portainer.Stack
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var t portainer.Stack
err := internal.UnmarshalObject(v, &t)
if err != nil {
return err
}
if t.Name == name {
stack = &t
break
}
}
if stack == nil {
return errors.ErrObjectNotFound
}
return nil
})
return stack, err
}
// Stacks returns an array containing all the stacks with same name
func (service *Service) StacksByName(name string) ([]portainer.Stack, error) {
var stacks = make([]portainer.Stack, 0)
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var t portainer.Stack
err := internal.UnmarshalObject(v, &t)
if err != nil {
return err
}
if t.Name == name {
stacks = append(stacks, t)
}
}
return nil
})
return stacks, err
}
// Stacks returns an array containing all the stacks.
func (service *Service) Stacks() ([]portainer.Stack, error) {
var stacks = make([]portainer.Stack, 0)
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var stack portainer.Stack
err := internal.UnmarshalObject(v, &stack)
if err != nil {
return err
}
stacks = append(stacks, stack)
}
return nil
})
return stacks, err
}
// GetNextIdentifier returns the next identifier for a stack.
func (service *Service) GetNextIdentifier() int {
return internal.GetNextIdentifier(service.connection, BucketName)
}
// CreateStack creates a new stack.
func (service *Service) CreateStack(stack *portainer.Stack) error {
return service.connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
// We manually manage sequences for stacks
err := bucket.SetSequence(uint64(stack.ID))
if err != nil {
return err
}
data, err := internal.MarshalObject(stack)
if err != nil {
return err
}
return bucket.Put(internal.Itob(int(stack.ID)), data)
})
}
// UpdateStack updates a stack.
func (service *Service) UpdateStack(ID portainer.StackID, stack *portainer.Stack) error {
identifier := internal.Itob(int(ID))
return internal.UpdateObject(service.connection, BucketName, identifier, stack)
}
// DeleteStack deletes a stack.
func (service *Service) DeleteStack(ID portainer.StackID) error {
identifier := internal.Itob(int(ID))
return internal.DeleteObject(service.connection, BucketName, identifier)
}
// StackByWebhookID returns a pointer to a stack object by webhook ID.
// It returns nil, errors.ErrObjectNotFound if there's no stack associated with the webhook ID.
func (service *Service) StackByWebhookID(id string) (*portainer.Stack, error) {
if id == "" {
return nil, pkgerrors.New("webhook ID can't be empty string")
}
var stack portainer.Stack
found := false
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var t struct {
AutoUpdate *struct {
WebhookID string `json:"Webhook"`
} `json:"AutoUpdate"`
}
err := internal.UnmarshalObject(v, &t)
if err != nil {
return err
}
if t.AutoUpdate != nil && strings.EqualFold(t.AutoUpdate.WebhookID, id) {
found = true
err := internal.UnmarshalObject(v, &stack)
if err != nil {
return err
}
break
}
}
return nil
})
if err != nil {
return nil, err
}
if !found {
return nil, errors.ErrObjectNotFound
}
return &stack, nil
}
// RefreshableStacks returns stacks that are configured for a periodic update
func (service *Service) RefreshableStacks() ([]portainer.Stack, error) {
stacks := make([]portainer.Stack, 0)
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
stack := portainer.Stack{}
err := internal.UnmarshalObject(v, &stack)
if err != nil {
return err
}
if stack.AutoUpdate != nil && stack.AutoUpdate.Interval != "" {
stacks = append(stacks, stack)
}
}
return nil
})
return stacks, err
}

View File

@ -1,95 +0,0 @@
package tag
import (
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/internal"
"github.com/boltdb/bolt"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "tags"
)
// Service represents a service for managing environment(endpoint) data.
type Service struct {
connection *internal.DbConnection
}
// NewService creates a new instance of a service.
func NewService(connection *internal.DbConnection) (*Service, error) {
err := internal.CreateBucket(connection, BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// Tags return an array containing all the tags.
func (service *Service) Tags() ([]portainer.Tag, error) {
var tags = make([]portainer.Tag, 0)
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var tag portainer.Tag
err := internal.UnmarshalObject(v, &tag)
if err != nil {
return err
}
tags = append(tags, tag)
}
return nil
})
return tags, err
}
// Tag returns a tag by ID.
func (service *Service) Tag(ID portainer.TagID) (*portainer.Tag, error) {
var tag portainer.Tag
identifier := internal.Itob(int(ID))
err := internal.GetObject(service.connection, BucketName, identifier, &tag)
if err != nil {
return nil, err
}
return &tag, nil
}
// CreateTag creates a new tag.
func (service *Service) CreateTag(tag *portainer.Tag) error {
return service.connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
id, _ := bucket.NextSequence()
tag.ID = portainer.TagID(id)
data, err := internal.MarshalObject(tag)
if err != nil {
return err
}
return bucket.Put(internal.Itob(int(tag.ID)), data)
})
}
// UpdateTag updates a tag.
func (service *Service) UpdateTag(ID portainer.TagID, tag *portainer.Tag) error {
identifier := internal.Itob(int(ID))
return internal.UpdateObject(service.connection, BucketName, identifier, tag)
}
// DeleteTag deletes a tag.
func (service *Service) DeleteTag(ID portainer.TagID) error {
identifier := internal.Itob(int(ID))
return internal.DeleteObject(service.connection, BucketName, identifier)
}

View File

@ -1,129 +0,0 @@
package team
import (
"strings"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/errors"
"github.com/portainer/portainer/api/bolt/internal"
"github.com/boltdb/bolt"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "teams"
)
// Service represents a service for managing environment(endpoint) data.
type Service struct {
connection *internal.DbConnection
}
// NewService creates a new instance of a service.
func NewService(connection *internal.DbConnection) (*Service, error) {
err := internal.CreateBucket(connection, BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// Team returns a Team by ID
func (service *Service) Team(ID portainer.TeamID) (*portainer.Team, error) {
var team portainer.Team
identifier := internal.Itob(int(ID))
err := internal.GetObject(service.connection, BucketName, identifier, &team)
if err != nil {
return nil, err
}
return &team, nil
}
// TeamByName returns a team by name.
func (service *Service) TeamByName(name string) (*portainer.Team, error) {
var team *portainer.Team
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var t portainer.Team
err := internal.UnmarshalObject(v, &t)
if err != nil {
return err
}
if strings.EqualFold(t.Name, name) {
team = &t
break
}
}
if team == nil {
return errors.ErrObjectNotFound
}
return nil
})
return team, err
}
// Teams return an array containing all the teams.
func (service *Service) Teams() ([]portainer.Team, error) {
var teams = make([]portainer.Team, 0)
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var team portainer.Team
err := internal.UnmarshalObject(v, &team)
if err != nil {
return err
}
teams = append(teams, team)
}
return nil
})
return teams, err
}
// UpdateTeam saves a Team.
func (service *Service) UpdateTeam(ID portainer.TeamID, team *portainer.Team) error {
identifier := internal.Itob(int(ID))
return internal.UpdateObject(service.connection, BucketName, identifier, team)
}
// CreateTeam creates a new Team.
func (service *Service) CreateTeam(team *portainer.Team) error {
return service.connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
id, _ := bucket.NextSequence()
team.ID = portainer.TeamID(id)
data, err := internal.MarshalObject(team)
if err != nil {
return err
}
return bucket.Put(internal.Itob(int(team.ID)), data)
})
}
// DeleteTeam deletes a Team.
func (service *Service) DeleteTeam(ID portainer.TeamID) error {
identifier := internal.Itob(int(ID))
return internal.DeleteObject(service.connection, BucketName, identifier)
}

View File

@ -1,197 +0,0 @@
package teammembership
import (
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/internal"
"github.com/boltdb/bolt"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "team_membership"
)
// Service represents a service for managing environment(endpoint) data.
type Service struct {
connection *internal.DbConnection
}
// NewService creates a new instance of a service.
func NewService(connection *internal.DbConnection) (*Service, error) {
err := internal.CreateBucket(connection, BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// TeamMembership returns a TeamMembership object by ID
func (service *Service) TeamMembership(ID portainer.TeamMembershipID) (*portainer.TeamMembership, error) {
var membership portainer.TeamMembership
identifier := internal.Itob(int(ID))
err := internal.GetObject(service.connection, BucketName, identifier, &membership)
if err != nil {
return nil, err
}
return &membership, nil
}
// TeamMemberships return an array containing all the TeamMembership objects.
func (service *Service) TeamMemberships() ([]portainer.TeamMembership, error) {
var memberships = make([]portainer.TeamMembership, 0)
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var membership portainer.TeamMembership
err := internal.UnmarshalObject(v, &membership)
if err != nil {
return err
}
memberships = append(memberships, membership)
}
return nil
})
return memberships, err
}
// TeamMembershipsByUserID return an array containing all the TeamMembership objects where the specified userID is present.
func (service *Service) TeamMembershipsByUserID(userID portainer.UserID) ([]portainer.TeamMembership, error) {
var memberships = make([]portainer.TeamMembership, 0)
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var membership portainer.TeamMembership
err := internal.UnmarshalObject(v, &membership)
if err != nil {
return err
}
if membership.UserID == userID {
memberships = append(memberships, membership)
}
}
return nil
})
return memberships, err
}
// TeamMembershipsByTeamID return an array containing all the TeamMembership objects where the specified teamID is present.
func (service *Service) TeamMembershipsByTeamID(teamID portainer.TeamID) ([]portainer.TeamMembership, error) {
var memberships = make([]portainer.TeamMembership, 0)
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var membership portainer.TeamMembership
err := internal.UnmarshalObject(v, &membership)
if err != nil {
return err
}
if membership.TeamID == teamID {
memberships = append(memberships, membership)
}
}
return nil
})
return memberships, err
}
// UpdateTeamMembership saves a TeamMembership object.
func (service *Service) UpdateTeamMembership(ID portainer.TeamMembershipID, membership *portainer.TeamMembership) error {
identifier := internal.Itob(int(ID))
return internal.UpdateObject(service.connection, BucketName, identifier, membership)
}
// CreateTeamMembership creates a new TeamMembership object.
func (service *Service) CreateTeamMembership(membership *portainer.TeamMembership) error {
return service.connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
id, _ := bucket.NextSequence()
membership.ID = portainer.TeamMembershipID(id)
data, err := internal.MarshalObject(membership)
if err != nil {
return err
}
return bucket.Put(internal.Itob(int(membership.ID)), data)
})
}
// DeleteTeamMembership deletes a TeamMembership object.
func (service *Service) DeleteTeamMembership(ID portainer.TeamMembershipID) error {
identifier := internal.Itob(int(ID))
return internal.DeleteObject(service.connection, BucketName, identifier)
}
// DeleteTeamMembershipByUserID deletes all the TeamMembership object associated to a UserID.
func (service *Service) DeleteTeamMembershipByUserID(userID portainer.UserID) error {
return service.connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var membership portainer.TeamMembership
err := internal.UnmarshalObject(v, &membership)
if err != nil {
return err
}
if membership.UserID == userID {
err := bucket.Delete(internal.Itob(int(membership.ID)))
if err != nil {
return err
}
}
}
return nil
})
}
// DeleteTeamMembershipByTeamID deletes all the TeamMembership object associated to a TeamID.
func (service *Service) DeleteTeamMembershipByTeamID(teamID portainer.TeamID) error {
return service.connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var membership portainer.TeamMembership
err := internal.UnmarshalObject(v, &membership)
if err != nil {
return err
}
if membership.TeamID == teamID {
err := bucket.Delete(internal.Itob(int(membership.ID)))
if err != nil {
return err
}
}
}
return nil
})
}

View File

@ -1,68 +0,0 @@
package bolt
import (
"io/ioutil"
"log"
"os"
"github.com/pkg/errors"
"github.com/portainer/portainer/api/filesystem"
)
var errTempDir = errors.New("can't create a temp dir")
func MustNewTestStore(init bool) (*Store, func()) {
store, teardown, err := NewTestStore(init)
if err != nil {
if !errors.Is(err, errTempDir) {
teardown()
}
log.Fatal(err)
}
return store, teardown
}
func NewTestStore(init bool) (*Store, func(), error) {
// Creates unique temp directory in a concurrency friendly manner.
dataStorePath, err := ioutil.TempDir("", "boltdb")
if err != nil {
return nil, nil, errors.Wrap(errTempDir, err.Error())
}
fileService, err := filesystem.NewService(dataStorePath, "")
if err != nil {
return nil, nil, err
}
store := NewStore(dataStorePath, fileService)
err = store.Open()
if err != nil {
return nil, nil, err
}
if init {
err = store.Init()
if err != nil {
return nil, nil, err
}
}
teardown := func() {
teardown(store, dataStorePath)
}
return store, teardown, nil
}
func teardown(store *Store, dataStorePath string) {
err := store.Close()
if err != nil {
log.Fatalln(err)
}
err = os.RemoveAll(dataStorePath)
if err != nil {
log.Fatalln(err)
}
}

View File

@ -1,156 +0,0 @@
package user
import (
"strings"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/errors"
"github.com/portainer/portainer/api/bolt/internal"
"github.com/boltdb/bolt"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "users"
)
// Service represents a service for managing environment(endpoint) data.
type Service struct {
connection *internal.DbConnection
}
// NewService creates a new instance of a service.
func NewService(connection *internal.DbConnection) (*Service, error) {
err := internal.CreateBucket(connection, BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// User returns a user by ID
func (service *Service) User(ID portainer.UserID) (*portainer.User, error) {
var user portainer.User
identifier := internal.Itob(int(ID))
err := internal.GetObject(service.connection, BucketName, identifier, &user)
if err != nil {
return nil, err
}
return &user, nil
}
// UserByUsername returns a user by username.
func (service *Service) UserByUsername(username string) (*portainer.User, error) {
var user *portainer.User
username = strings.ToLower(username)
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var u portainer.User
err := internal.UnmarshalObject(v, &u)
if err != nil {
return err
}
if strings.EqualFold(u.Username, username) {
user = &u
break
}
}
if user == nil {
return errors.ErrObjectNotFound
}
return nil
})
return user, err
}
// Users return an array containing all the users.
func (service *Service) Users() ([]portainer.User, error) {
var users = make([]portainer.User, 0)
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var user portainer.User
err := internal.UnmarshalObject(v, &user)
if err != nil {
return err
}
users = append(users, user)
}
return nil
})
return users, err
}
// UsersByRole return an array containing all the users with the specified role.
func (service *Service) UsersByRole(role portainer.UserRole) ([]portainer.User, error) {
var users = make([]portainer.User, 0)
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var user portainer.User
err := internal.UnmarshalObject(v, &user)
if err != nil {
return err
}
if user.Role == role {
users = append(users, user)
}
}
return nil
})
return users, err
}
// UpdateUser saves a user.
func (service *Service) UpdateUser(ID portainer.UserID, user *portainer.User) error {
identifier := internal.Itob(int(ID))
user.Username = strings.ToLower(user.Username)
return internal.UpdateObject(service.connection, BucketName, identifier, user)
}
// CreateUser creates a new user.
func (service *Service) CreateUser(user *portainer.User) error {
return service.connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
id, _ := bucket.NextSequence()
user.ID = portainer.UserID(id)
user.Username = strings.ToLower(user.Username)
data, err := internal.MarshalObject(user)
if err != nil {
return err
}
return bucket.Put(internal.Itob(int(user.ID)), data)
})
}
// DeleteUser deletes a user.
func (service *Service) DeleteUser(ID portainer.UserID) error {
identifier := internal.Itob(int(ID))
return internal.DeleteObject(service.connection, BucketName, identifier)
}

View File

@ -1,167 +0,0 @@
package version
import (
"strconv"
"github.com/boltdb/bolt"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/errors"
"github.com/portainer/portainer/api/bolt/internal"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "version"
versionKey = "DB_VERSION"
instanceKey = "INSTANCE_ID"
editionKey = "EDITION"
updatingKey = "DB_UPDATING"
)
// Service represents a service to manage stored versions.
type Service struct {
connection *internal.DbConnection
}
// NewService creates a new instance of a service.
func NewService(connection *internal.DbConnection) (*Service, error) {
err := internal.CreateBucket(connection, BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// DBVersion retrieves the stored database version.
func (service *Service) DBVersion() (int, error) {
var data []byte
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
value := bucket.Get([]byte(versionKey))
if value == nil {
return errors.ErrObjectNotFound
}
data = make([]byte, len(value))
copy(data, value)
return nil
})
if err != nil {
return 0, err
}
return strconv.Atoi(string(data))
}
// Edition retrieves the stored portainer edition.
func (service *Service) Edition() (portainer.SoftwareEdition, error) {
editionData, err := service.getKey(editionKey)
if err != nil {
return 0, err
}
edition, err := strconv.Atoi(string(editionData))
if err != nil {
return 0, err
}
return portainer.SoftwareEdition(edition), nil
}
// StoreDBVersion store the database version.
func (service *Service) StoreDBVersion(version int) error {
return service.connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
data := []byte(strconv.Itoa(version))
return bucket.Put([]byte(versionKey), data)
})
}
// IsUpdating retrieves the database updating status.
func (service *Service) IsUpdating() (bool, error) {
isUpdating, err := service.getKey(updatingKey)
if err != nil {
return false, err
}
return strconv.ParseBool(string(isUpdating))
}
// StoreIsUpdating store the database updating status.
func (service *Service) StoreIsUpdating(isUpdating bool) error {
return service.setKey(updatingKey, strconv.FormatBool(isUpdating))
}
// InstanceID retrieves the stored instance ID.
func (service *Service) InstanceID() (string, error) {
var data []byte
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
value := bucket.Get([]byte(instanceKey))
if value == nil {
return errors.ErrObjectNotFound
}
data = make([]byte, len(value))
copy(data, value)
return nil
})
if err != nil {
return "", err
}
return string(data), nil
}
// StoreInstanceID store the instance ID.
func (service *Service) StoreInstanceID(ID string) error {
return service.connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
data := []byte(ID)
return bucket.Put([]byte(instanceKey), data)
})
}
func (service *Service) getKey(key string) ([]byte, error) {
var data []byte
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
value := bucket.Get([]byte(key))
if value == nil {
return errors.ErrObjectNotFound
}
data = make([]byte, len(value))
copy(data, value)
return nil
})
if err != nil {
return nil, err
}
return data, nil
}
func (service *Service) setKey(key string, value string) error {
return service.connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
data := []byte(value)
return bucket.Put([]byte(key), data)
})
}

View File

@ -1,158 +0,0 @@
package webhook
import (
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/errors"
"github.com/portainer/portainer/api/bolt/internal"
"github.com/boltdb/bolt"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "webhooks"
)
// Service represents a service for managing webhook data.
type Service struct {
connection *internal.DbConnection
}
// NewService creates a new instance of a service.
func NewService(connection *internal.DbConnection) (*Service, error) {
err := internal.CreateBucket(connection, BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
//Webhooks returns an array of all webhooks
func (service *Service) Webhooks() ([]portainer.Webhook, error) {
var webhooks = make([]portainer.Webhook, 0)
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var webhook portainer.Webhook
err := internal.UnmarshalObject(v, &webhook)
if err != nil {
return err
}
webhooks = append(webhooks, webhook)
}
return nil
})
return webhooks, err
}
// Webhook returns a webhook by ID.
func (service *Service) Webhook(ID portainer.WebhookID) (*portainer.Webhook, error) {
var webhook portainer.Webhook
identifier := internal.Itob(int(ID))
err := internal.GetObject(service.connection, BucketName, identifier, &webhook)
if err != nil {
return nil, err
}
return &webhook, nil
}
// WebhookByResourceID returns a webhook by the ResourceID it is associated with.
func (service *Service) WebhookByResourceID(ID string) (*portainer.Webhook, error) {
var webhook *portainer.Webhook
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var w portainer.Webhook
err := internal.UnmarshalObject(v, &w)
if err != nil {
return err
}
if w.ResourceID == ID {
webhook = &w
break
}
}
if webhook == nil {
return errors.ErrObjectNotFound
}
return nil
})
return webhook, err
}
// WebhookByToken returns a webhook by the random token it is associated with.
func (service *Service) WebhookByToken(token string) (*portainer.Webhook, error) {
var webhook *portainer.Webhook
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var w portainer.Webhook
err := internal.UnmarshalObject(v, &w)
if err != nil {
return err
}
if w.Token == token {
webhook = &w
break
}
}
if webhook == nil {
return errors.ErrObjectNotFound
}
return nil
})
return webhook, err
}
// DeleteWebhook deletes a webhook.
func (service *Service) DeleteWebhook(ID portainer.WebhookID) error {
identifier := internal.Itob(int(ID))
return internal.DeleteObject(service.connection, BucketName, identifier)
}
// CreateWebhook assign an ID to a new webhook and saves it.
func (service *Service) CreateWebhook(webhook *portainer.Webhook) error {
return service.connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
id, _ := bucket.NextSequence()
webhook.ID = portainer.WebhookID(id)
data, err := internal.MarshalObject(webhook)
if err != nil {
return err
}
return bucket.Put(internal.Itob(int(webhook.ID)), data)
})
}
// UpdateWebhook update a webhook.
func (service *Service) UpdateWebhook(ID portainer.WebhookID, webhook *portainer.Webhook) error {
identifier := internal.Itob(int(ID))
return internal.UpdateObject(service.connection, BucketName, identifier, webhook)
}

View File

@ -13,7 +13,7 @@ import (
chserver "github.com/jpillora/chisel/server"
cmap "github.com/orcaman/concurrent-map"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/errors"
"github.com/portainer/portainer/api/dataservices"
)
const (
@ -29,7 +29,7 @@ type Service struct {
serverFingerprint string
serverPort string
tunnelDetailsMap cmap.ConcurrentMap
dataStore portainer.DataStore
dataStore dataservices.DataStore
snapshotService portainer.SnapshotService
chiselServer *chserver.Server
shutdownCtx context.Context
@ -37,7 +37,7 @@ type Service struct {
}
// NewService returns a pointer to a new instance of Service
func NewService(dataStore portainer.DataStore, shutdownCtx context.Context) *Service {
func NewService(dataStore dataservices.DataStore, shutdownCtx context.Context) *Service {
return &Service{
tunnelDetailsMap: cmap.New(),
dataStore: dataStore,
@ -46,7 +46,7 @@ func NewService(dataStore portainer.DataStore, shutdownCtx context.Context) *Ser
}
// pingAgent ping the given agent so that the agent can keep the tunnel alive
func (service *Service) pingAgent(endpointID portainer.EndpointID) error{
func (service *Service) pingAgent(endpointID portainer.EndpointID) error {
tunnel := service.GetTunnelDetails(endpointID)
requestURL := fmt.Sprintf("http://127.0.0.1:%d/ping", tunnel.Port)
req, err := http.NewRequest(http.MethodHead, requestURL, nil)
@ -147,7 +147,7 @@ func (service *Service) retrievePrivateKeySeed() (string, error) {
var serverInfo *portainer.TunnelServerInfo
serverInfo, err := service.dataStore.TunnelServer().Info()
if err == errors.ErrObjectNotFound {
if service.dataStore.IsErrObjectNotFound(err) {
keySeed := uniuri.NewLen(16)
serverInfo = &portainer.TunnelServerInfo{

View File

@ -1,3 +1,4 @@
//go:build !windows
// +build !windows
package cli

View File

@ -0,0 +1,29 @@
package main
import (
"log"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/datastore"
"github.com/sirupsen/logrus"
)
func importFromJson(fileService portainer.FileService, store *datastore.Store) {
// EXPERIMENTAL - if used with an incomplete json file, it will fail, as we don't have a way to default the model values
importFile := "/data/import.json"
if exists, _ := fileService.FileExists(importFile); exists {
if err := store.Import(importFile); err != nil {
logrus.WithError(err).Debugf("import %s failed", importFile)
// TODO: should really rollback on failure, but then we have nothing.
} else {
logrus.Printf("Successfully imported %s to new portainer database", importFile)
}
// TODO: this is bad - its to ensure that any defaults that were broken in import, or migrations get set back to what we want
// I also suspect that everything from "Init to Init" is potentially a migration
err := store.Init()
if err != nil {
log.Fatalf("failed initializing data store: %v", err)
}
}
}

View File

@ -5,16 +5,22 @@ import (
"fmt"
"log"
"os"
"path"
"strconv"
"strings"
"time"
"github.com/sirupsen/logrus"
"github.com/portainer/libhelm"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/apikey"
"github.com/portainer/portainer/api/bolt"
"github.com/portainer/portainer/api/chisel"
"github.com/portainer/portainer/api/cli"
"github.com/portainer/portainer/api/crypto"
"github.com/portainer/portainer/api/database"
"github.com/portainer/portainer/api/dataservices"
"github.com/portainer/portainer/api/datastore"
"github.com/portainer/portainer/api/docker"
"github.com/portainer/portainer/api/exec"
"github.com/portainer/portainer/api/filesystem"
@ -59,14 +65,18 @@ func initFileService(dataStorePath string) portainer.FileService {
return fileService
}
func initDataStore(dataStorePath string, rollback bool, fileService portainer.FileService, shutdownCtx context.Context) portainer.DataStore {
store := bolt.NewStore(dataStorePath, fileService)
err := store.Open()
func initDataStore(flags *portainer.CLIFlags, fileService portainer.FileService, shutdownCtx context.Context) dataservices.DataStore {
connection, err := database.NewDatabase("boltdb", *flags.Data)
if err != nil {
panic(err)
}
store := datastore.NewStore(*flags.Data, fileService, connection)
isNew, err := store.Open()
if err != nil {
log.Fatalf("failed opening store: %v", err)
}
if rollback {
if *flags.Rollback {
err := store.Rollback(false)
if err != nil {
log.Fatalf("failed rolling back: %s", err)
@ -77,23 +87,53 @@ func initDataStore(dataStorePath string, rollback bool, fileService portainer.Fi
return nil
}
// Init sets some defaults - its basically a migration
err = store.Init()
if err != nil {
log.Fatalf("failed initializing data store: %v", err)
}
err = store.MigrateData(false)
if err != nil {
log.Fatalf("failed migration: %v", err)
if isNew {
// from MigrateData
store.VersionService.StoreDBVersion(portainer.DBVersion)
// Disabled for now. Can't use feature flags due to the way that works
// EXPERIMENTAL, will only activate if `/data/import.json` exists
//importFromJson(fileService, store)
err := updateSettingsFromFlags(store, flags)
if err != nil {
log.Fatalf("failed updating settings from flags: %v", err)
}
}
go shutdownDatastore(shutdownCtx, store)
return store
}
storedVersion, err := store.VersionService.DBVersion()
if err != nil {
log.Fatalf("Something failed during creation of new database: %v", err)
}
if storedVersion != portainer.DBVersion {
err = store.MigrateData()
if err != nil {
log.Fatalf("failed migration: %v", err)
}
}
func shutdownDatastore(shutdownCtx context.Context, datastore portainer.DataStore) {
<-shutdownCtx.Done()
datastore.Close()
// this is for the db restore functionality - needs more tests.
go func() {
<-shutdownCtx.Done()
defer connection.Close()
exportFilename := path.Join(*flags.Data, fmt.Sprintf("export-%d.json", time.Now().Unix()))
err := store.Export(exportFilename)
if err != nil {
logrus.WithError(err).Debugf("failed to export to %s", exportFilename)
} else {
logrus.Debugf("exported to %s", exportFilename)
}
connection.Close()
}()
return store
}
func initComposeStackManager(assetsPath string, configPath string, reverseTunnelService portainer.ReverseTunnelService, proxyManager *proxy.Manager) portainer.ComposeStackManager {
@ -111,12 +151,12 @@ func initSwarmStackManager(
signatureService portainer.DigitalSignatureService,
fileService portainer.FileService,
reverseTunnelService portainer.ReverseTunnelService,
dataStore portainer.DataStore,
dataStore dataservices.DataStore,
) (portainer.SwarmStackManager, error) {
return exec.NewSwarmStackManager(assetsPath, configPath, signatureService, fileService, reverseTunnelService, dataStore)
}
func initKubernetesDeployer(kubernetesTokenCacheManager *kubeproxy.TokenCacheManager, kubernetesClientFactory *kubecli.ClientFactory, dataStore portainer.DataStore, reverseTunnelService portainer.ReverseTunnelService, signatureService portainer.DigitalSignatureService, proxyManager *proxy.Manager, assetsPath string) portainer.KubernetesDeployer {
func initKubernetesDeployer(kubernetesTokenCacheManager *kubeproxy.TokenCacheManager, kubernetesClientFactory *kubecli.ClientFactory, dataStore dataservices.DataStore, reverseTunnelService portainer.ReverseTunnelService, signatureService portainer.DigitalSignatureService, proxyManager *proxy.Manager, assetsPath string) portainer.KubernetesDeployer {
return exec.NewKubernetesDeployer(kubernetesTokenCacheManager, kubernetesClientFactory, dataStore, reverseTunnelService, signatureService, proxyManager, assetsPath)
}
@ -124,21 +164,12 @@ func initHelmPackageManager(assetsPath string) (libhelm.HelmPackageManager, erro
return libhelm.NewHelmPackageManager(libhelm.HelmConfig{BinaryPath: assetsPath})
}
func initAPIKeyService(datastore portainer.DataStore) apikey.APIKeyService {
func initAPIKeyService(datastore dataservices.DataStore) apikey.APIKeyService {
return apikey.NewAPIKeyService(datastore.APIKeyRepository(), datastore.User())
}
func initJWTService(dataStore portainer.DataStore) (portainer.JWTService, error) {
settings, err := dataStore.Settings().Settings()
if err != nil {
return nil, err
}
if settings.UserSessionTimeout == "" {
settings.UserSessionTimeout = portainer.DefaultUserSessionTimeout
dataStore.Settings().UpdateSettings(settings)
}
jwtService, err := jwt.NewService(settings.UserSessionTimeout, dataStore)
func initJWTService(userSessionTimeout string, dataStore dataservices.DataStore) (dataservices.JWTService, error) {
jwtService, err := jwt.NewService(userSessionTimeout, dataStore)
if err != nil {
return nil, err
}
@ -165,7 +196,7 @@ func initGitService() portainer.GitService {
return git.NewService()
}
func initSSLService(addr, dataPath, certPath, keyPath string, fileService portainer.FileService, dataStore portainer.DataStore, shutdownTrigger context.CancelFunc) (*ssl.Service, error) {
func initSSLService(addr, dataPath, certPath, keyPath string, fileService portainer.FileService, dataStore dataservices.DataStore, shutdownTrigger context.CancelFunc) (*ssl.Service, error) {
slices := strings.Split(addr, ":")
host := slices[0]
if host == "" {
@ -186,11 +217,11 @@ func initDockerClientFactory(signatureService portainer.DigitalSignatureService,
return docker.NewClientFactory(signatureService, reverseTunnelService)
}
func initKubernetesClientFactory(signatureService portainer.DigitalSignatureService, reverseTunnelService portainer.ReverseTunnelService, instanceID string, dataStore portainer.DataStore) *kubecli.ClientFactory {
func initKubernetesClientFactory(signatureService portainer.DigitalSignatureService, reverseTunnelService portainer.ReverseTunnelService, instanceID string, dataStore dataservices.DataStore) *kubecli.ClientFactory {
return kubecli.NewClientFactory(signatureService, reverseTunnelService, instanceID, dataStore)
}
func initSnapshotService(snapshotInterval string, dataStore portainer.DataStore, dockerClientFactory *docker.ClientFactory, kubernetesClientFactory *kubecli.ClientFactory, shutdownCtx context.Context) (portainer.SnapshotService, error) {
func initSnapshotService(snapshotInterval string, dataStore dataservices.DataStore, dockerClientFactory *docker.ClientFactory, kubernetesClientFactory *kubecli.ClientFactory, shutdownCtx context.Context) (portainer.SnapshotService, error) {
dockerSnapshotter := docker.NewSnapshotter(dockerClientFactory)
kubernetesSnapshotter := kubernetes.NewSnapshotter(kubernetesClientFactory)
@ -209,7 +240,7 @@ func initStatus(instanceID string) *portainer.Status {
}
}
func updateSettingsFromFlags(dataStore portainer.DataStore, flags *portainer.CLIFlags) error {
func updateSettingsFromFlags(dataStore dataservices.DataStore, flags *portainer.CLIFlags) error {
settings, err := dataStore.Settings().Settings()
if err != nil {
return err
@ -256,7 +287,7 @@ func updateSettingsFromFlags(dataStore portainer.DataStore, flags *portainer.CLI
// enableFeaturesFromFlags turns on or off feature flags
// e.g. portainer --feat open-amt --feat fdo=true ... (defaults to true)
// note, settings are persisted to the DB. To turn off `--feat open-amt=false`
func enableFeaturesFromFlags(dataStore portainer.DataStore, flags *portainer.CLIFlags) error {
func enableFeaturesFromFlags(dataStore dataservices.DataStore, flags *portainer.CLIFlags) error {
settings, err := dataStore.Settings().Settings()
if err != nil {
return err
@ -325,7 +356,7 @@ func initKeyPair(fileService portainer.FileService, signatureService portainer.D
return generateAndStoreKeyPair(fileService, signatureService)
}
func createTLSSecuredEndpoint(flags *portainer.CLIFlags, dataStore portainer.DataStore, snapshotService portainer.SnapshotService) error {
func createTLSSecuredEndpoint(flags *portainer.CLIFlags, dataStore dataservices.DataStore, snapshotService portainer.SnapshotService) error {
tlsConfiguration := portainer.TLSConfiguration{
TLS: *flags.TLS,
TLSSkipVerify: *flags.TLSSkipVerify,
@ -390,10 +421,10 @@ func createTLSSecuredEndpoint(flags *portainer.CLIFlags, dataStore portainer.Dat
log.Printf("http error: environment snapshot error (environment=%s, URL=%s) (err=%s)\n", endpoint.Name, endpoint.URL, err)
}
return dataStore.Endpoint().CreateEndpoint(endpoint)
return dataStore.Endpoint().Create(endpoint)
}
func createUnsecuredEndpoint(endpointURL string, dataStore portainer.DataStore, snapshotService portainer.SnapshotService) error {
func createUnsecuredEndpoint(endpointURL string, dataStore dataservices.DataStore, snapshotService portainer.SnapshotService) error {
if strings.HasPrefix(endpointURL, "tcp://") {
_, err := client.ExecutePingOperation(endpointURL, nil)
if err != nil {
@ -436,10 +467,10 @@ func createUnsecuredEndpoint(endpointURL string, dataStore portainer.DataStore,
log.Printf("http error: environment snapshot error (environment=%s, URL=%s) (err=%s)\n", endpoint.Name, endpoint.URL, err)
}
return dataStore.Endpoint().CreateEndpoint(endpoint)
return dataStore.Endpoint().Create(endpoint)
}
func initEndpoint(flags *portainer.CLIFlags, dataStore portainer.DataStore, snapshotService portainer.SnapshotService) error {
func initEndpoint(flags *portainer.CLIFlags, dataStore dataservices.DataStore, snapshotService portainer.SnapshotService) error {
if *flags.EndpointURL == "" {
return nil
}
@ -465,15 +496,23 @@ func buildServer(flags *portainer.CLIFlags) portainer.Server {
fileService := initFileService(*flags.Data)
dataStore := initDataStore(*flags.Data, *flags.Rollback, fileService, shutdownCtx)
dataStore := initDataStore(flags, fileService, shutdownCtx)
if err := dataStore.CheckCurrentEdition(); err != nil {
log.Fatal(err)
}
instanceID, err := dataStore.Version().InstanceID()
if err != nil {
log.Fatalf("failed getting instance id: %v", err)
}
apiKeyService := initAPIKeyService(dataStore)
jwtService, err := initJWTService(dataStore)
settings, err := dataStore.Settings().Settings()
if err != nil {
log.Fatal(err)
}
jwtService, err := initJWTService(settings.UserSessionTimeout, dataStore)
if err != nil {
log.Fatalf("failed initializing JWT service: %v", err)
}
@ -484,15 +523,12 @@ func buildServer(flags *portainer.CLIFlags) portainer.Server {
}
ldapService := initLDAPService()
oauthService := initOAuthService()
gitService := initGitService()
openAMTService := openamt.NewService(dataStore)
cryptoService := initCryptoService()
digitalSignatureService := initDigitalSignatureService()
sslService, err := initSSLService(*flags.AddrHTTPS, *flags.Data, *flags.SSLCert, *flags.SSLKey, fileService, dataStore, shutdownTrigger)
@ -507,16 +543,11 @@ func buildServer(flags *portainer.CLIFlags) portainer.Server {
err = initKeyPair(fileService, digitalSignatureService)
if err != nil {
log.Fatalf("failed initializing key pai: %v", err)
log.Fatalf("failed initializing key pair: %v", err)
}
reverseTunnelService := chisel.NewService(dataStore, shutdownCtx)
instanceID, err := dataStore.Version().InstanceID()
if err != nil {
log.Fatalf("failed getting instance id: %v", err)
}
dockerClientFactory := initDockerClientFactory(digitalSignatureService, reverseTunnelService)
kubernetesClientFactory := initKubernetesClientFactory(digitalSignatureService, reverseTunnelService, instanceID, dataStore)
@ -553,11 +584,6 @@ func buildServer(flags *portainer.CLIFlags) portainer.Server {
log.Fatalf("failed initializing helm package manager: %s", err)
}
err = updateSettingsFromFlags(dataStore, flags)
if err != nil {
log.Fatalf("failed updating settings from flags: %v", err)
}
err = edge.LoadEdgeJobs(dataStore, reverseTunnelService)
if err != nil {
log.Fatalf("failed loading edge jobs from database: %v", err)
@ -597,7 +623,7 @@ func buildServer(flags *portainer.CLIFlags) portainer.Server {
Role: portainer.AdministratorRole,
Password: adminPasswordHash,
}
err := dataStore.User().CreateUser(user)
err := dataStore.User().Create(user)
if err != nil {
log.Fatalf("failed creating admin user: %v", err)
}

View File

@ -5,8 +5,9 @@ import (
"testing"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt"
"github.com/portainer/portainer/api/cli"
"github.com/portainer/portainer/api/dataservices"
"github.com/portainer/portainer/api/datastore"
"github.com/stretchr/testify/assert"
"gopkg.in/alecthomas/kingpin.v2"
)
@ -20,7 +21,7 @@ func (m mockKingpinSetting) SetValue(value kingpin.Value) {
func Test_enableFeaturesFromFlags(t *testing.T) {
is := assert.New(t)
store, teardown := bolt.MustNewTestStore(true)
_, store, teardown := datastore.MustNewTestStore(true)
defer teardown()
tests := []struct {
@ -59,7 +60,7 @@ func Test_enableFeaturesFromFlags(t *testing.T) {
const FeatTest portainer.Feature = "optional-test"
func optionalFunc(dataStore portainer.DataStore) string {
func optionalFunc(dataStore dataservices.DataStore) string {
// TODO: this is a code smell - finding out if a feature flag is enabled should not require having access to the store, and asking for a settings obj.
// ideally, the `if` should look more like:
@ -80,7 +81,7 @@ func Test_optionalFeature(t *testing.T) {
is := assert.New(t)
store, teardown := bolt.MustNewTestStore(true)
_, store, teardown := datastore.MustNewTestStore(true)
defer teardown()
// Enable the test feature

34
api/connection.go Normal file
View File

@ -0,0 +1,34 @@
package portainer
import (
"io"
)
type Connection interface {
Open() error
Close() error
// write the db contents to filename as json (the schema needs defining)
ExportRaw(filename string) error
//Rollback(force bool) error
//MigrateData(migratorParams *database.MigratorParameters, force bool) error
// TODO: this one is very database specific atm
BackupTo(w io.Writer) error
GetDatabaseFilename() string
GetStorePath() string
SetServiceName(bucketName string) error
GetObject(bucketName string, key []byte, object interface{}) error
UpdateObject(bucketName string, key []byte, object interface{}) error
DeleteObject(bucketName string, key []byte) error
DeleteAllObjects(bucketName string, matching func(o interface{}) (id int, ok bool)) error
GetNextIdentifier(bucketName string) int
CreateObject(bucketName string, fn func(uint64) (int, interface{})) error
CreateObjectWithId(bucketName string, id int, obj interface{}) error
CreateObjectWithSetSequence(bucketName string, id int, obj interface{}) error
GetAll(bucketName string, obj interface{}, append func(o interface{}) (interface{}, error)) error
GetAllWithJsoniter(bucketName string, obj interface{}, append func(o interface{}) (interface{}, error)) error
ConvertToKey(v int) []byte
}

293
api/database/boltdb/db.go Normal file
View File

@ -0,0 +1,293 @@
package boltdb
import (
"encoding/binary"
"fmt"
"io"
"io/ioutil"
"os"
"path"
"time"
"github.com/boltdb/bolt"
"github.com/portainer/portainer/api/dataservices/errors"
)
const (
DatabaseFileName = "portainer.db"
)
type DbConnection struct {
Path string
*bolt.DB
}
func (connection *DbConnection) GetDatabaseFilename() string {
return DatabaseFileName
}
func (connection *DbConnection) GetStorePath() string {
return connection.Path
}
// Open opens and initializes the BoltDB database.
func (connection *DbConnection) Open() error {
// Disabled for now. Can't use feature flags due to the way that works
// databaseExportPath := path.Join(connection.Path, fmt.Sprintf("raw-%s-%d.json", DatabaseFileName, time.Now().Unix()))
// if err := connection.ExportRaw(databaseExportPath); err != nil {
// log.Printf("raw export to %s error: %s", databaseExportPath, err)
// } else {
// log.Printf("raw export to %s success", databaseExportPath)
// }
databasePath := path.Join(connection.Path, DatabaseFileName)
db, err := bolt.Open(databasePath, 0600, &bolt.Options{Timeout: 1 * time.Second})
if err != nil {
return err
}
connection.DB = db
return nil
}
// Close closes the BoltDB database.
// Safe to being called multiple times.
func (connection *DbConnection) Close() error {
if connection.DB != nil {
return connection.DB.Close()
}
return nil
}
// BackupTo backs up db to a provided writer.
// It does hot backup and doesn't block other database reads and writes
func (connection *DbConnection) BackupTo(w io.Writer) error {
return connection.View(func(tx *bolt.Tx) error {
_, err := tx.WriteTo(w)
return err
})
}
func (connection *DbConnection) ExportRaw(filename string) error {
databasePath := path.Join(connection.Path, DatabaseFileName)
if _, err := os.Stat(databasePath); err != nil {
return fmt.Errorf("stat on %s failed: %s", databasePath, err)
}
b, err := exportJson(databasePath)
if err != nil {
return err
}
return ioutil.WriteFile(filename, b, 0600)
}
// ConvertToKey returns an 8-byte big endian representation of v.
// This function is typically used for encoding integer IDs to byte slices
// so that they can be used as BoltDB keys.
func (connection *DbConnection) ConvertToKey(v int) []byte {
b := make([]byte, 8)
binary.BigEndian.PutUint64(b, uint64(v))
return b
}
// CreateBucket is a generic function used to create a bucket inside a database database.
func (connection *DbConnection) SetServiceName(bucketName string) error {
return connection.Update(func(tx *bolt.Tx) error {
_, err := tx.CreateBucketIfNotExists([]byte(bucketName))
if err != nil {
return err
}
return nil
})
}
// GetObject is a generic function used to retrieve an unmarshalled object from a database database.
func (connection *DbConnection) GetObject(bucketName string, key []byte, object interface{}) error {
var data []byte
err := connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(bucketName))
value := bucket.Get(key)
if value == nil {
return errors.ErrObjectNotFound
}
data = make([]byte, len(value))
copy(data, value)
return nil
})
if err != nil {
return err
}
return UnmarshalObject(data, object)
}
// UpdateObject is a generic function used to update an object inside a database database.
func (connection *DbConnection) UpdateObject(bucketName string, key []byte, object interface{}) error {
return connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(bucketName))
data, err := MarshalObject(object)
if err != nil {
return err
}
err = bucket.Put(key, data)
if err != nil {
return err
}
return nil
})
}
// DeleteObject is a generic function used to delete an object inside a database database.
func (connection *DbConnection) DeleteObject(bucketName string, key []byte) error {
return connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(bucketName))
return bucket.Delete(key)
})
}
// DeleteAllObjects delete all objects where matching() returns (id, ok).
// TODO: think about how to return the error inside (maybe change ok to type err, and use "notfound"?
func (connection *DbConnection) DeleteAllObjects(bucketName string, matching func(o interface{}) (id int, ok bool)) error {
return connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(bucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var obj interface{}
err := UnmarshalObject(v, &obj)
if err != nil {
return err
}
if id, ok := matching(obj); ok {
err := bucket.Delete(connection.ConvertToKey(id))
if err != nil {
return err
}
}
}
return nil
})
}
// GetNextIdentifier is a generic function that returns the specified bucket identifier incremented by 1.
func (connection *DbConnection) GetNextIdentifier(bucketName string) int {
var identifier int
connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(bucketName))
id, err := bucket.NextSequence()
if err != nil {
return err
}
identifier = int(id)
return nil
})
return identifier
}
// CreateObject creates a new object in the bucket, using the next bucket sequence id
func (connection *DbConnection) CreateObject(bucketName string, fn func(uint64) (int, interface{})) error {
return connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(bucketName))
seqId, _ := bucket.NextSequence()
id, obj := fn(seqId)
data, err := MarshalObject(obj)
if err != nil {
return err
}
return bucket.Put(connection.ConvertToKey(int(id)), data)
})
}
// CreateObjectWithId creates a new object in the bucket, using the specified id
func (connection *DbConnection) CreateObjectWithId(bucketName string, id int, obj interface{}) error {
return connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(bucketName))
data, err := MarshalObject(obj)
if err != nil {
return err
}
return bucket.Put(connection.ConvertToKey(id), data)
})
}
// CreateObjectWithSetSequence creates a new object in the bucket, using the specified id, and sets the bucket sequence
// avoid this :)
func (connection *DbConnection) CreateObjectWithSetSequence(bucketName string, id int, obj interface{}) error {
return connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(bucketName))
// We manually manage sequences for schedules
err := bucket.SetSequence(uint64(id))
if err != nil {
return err
}
data, err := MarshalObject(obj)
if err != nil {
return err
}
return bucket.Put(connection.ConvertToKey(id), data)
})
}
func (connection *DbConnection) GetAll(bucketName string, obj interface{}, append func(o interface{}) (interface{}, error)) error {
err := connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(bucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
err := UnmarshalObject(v, obj)
if err != nil {
return err
}
obj, err = append(obj)
if err != nil {
return err
}
}
return nil
})
return err
}
// TODO: decide which Unmarshal to use, and use one...
func (connection *DbConnection) GetAllWithJsoniter(bucketName string, obj interface{}, append func(o interface{}) (interface{}, error)) error {
err := connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(bucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
err := UnmarshalObjectWithJsoniter(v, obj)
if err != nil {
return err
}
obj, err = append(obj)
if err != nil {
return err
}
}
return nil
})
return err
}

View File

@ -0,0 +1,67 @@
package boltdb
import (
"encoding/json"
"time"
"github.com/boltdb/bolt"
"github.com/sirupsen/logrus"
)
// inspired by github.com/konoui/boltdb-exporter (which has no license)
// but very much simplified, based on how we use boltdb
func exportJson(databasePath string) ([]byte, error) {
connection, err := bolt.Open(databasePath, 0600, &bolt.Options{Timeout: 1 * time.Second, ReadOnly: true})
if err != nil {
return []byte("{}"), err
}
defer connection.Close()
backup := make(map[string]interface{})
err = connection.View(func(tx *bolt.Tx) error {
err = tx.ForEach(func(name []byte, bucket *bolt.Bucket) error {
bucketName := string(name)
var list []interface{}
version := make(map[string]string)
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
if v == nil {
continue
}
var obj interface{}
err := UnmarshalObject(v, &obj)
if err != nil {
logrus.WithError(err).Errorf("Failed to unmarshal (bucket %s): %v", bucketName, string(v))
obj = v
}
if bucketName == "version" {
version[string(k)] = string(v)
} else {
list = append(list, obj)
}
}
if bucketName == "version" {
backup[bucketName] = version
}
if len(list) > 0 {
if bucketName == "ssl" ||
bucketName == "settings" ||
bucketName == "tunnel_server" {
backup[bucketName] = list[0]
return nil
}
backup[bucketName] = list
}
return nil
})
return err
})
if err != nil {
return []byte("{}"), err
}
return json.MarshalIndent(backup, "", " ")
}

View File

@ -1,4 +1,4 @@
package internal
package boltdb
import (
"encoding/json"

16
api/database/database.go Normal file
View File

@ -0,0 +1,16 @@
package database
import (
"fmt"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/database/boltdb"
)
// NewDatabase should use config options to return a connection to the requested database
func NewDatabase(storeType, storePath string) (connection portainer.Connection, err error) {
switch storeType {
case "boltdb":
return &boltdb.DbConnection{Path: storePath}, nil
}
return nil, fmt.Errorf("Unknown storage database: %s", storeType)
}

View File

@ -0,0 +1,119 @@
package apikeyrepository
import (
"bytes"
"fmt"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices/errors"
"github.com/sirupsen/logrus"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "api_key"
)
// Service represents a service for managing api-key data.
type Service struct {
connection portainer.Connection
}
// NewService creates a new instance of a service.
func NewService(connection portainer.Connection) (*Service, error) {
err := connection.SetServiceName(BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// GetAPIKeysByUserID returns a slice containing all the APIKeys a user has access to.
func (service *Service) GetAPIKeysByUserID(userID portainer.UserID) ([]portainer.APIKey, error) {
var result = make([]portainer.APIKey, 0)
err := service.connection.GetAll(
BucketName,
&portainer.APIKey{},
func(obj interface{}) (interface{}, error) {
record, ok := obj.(*portainer.APIKey)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to APIKey object")
return nil, fmt.Errorf("Failed to convert to APIKey object: %s", obj)
}
if record.UserID == userID {
result = append(result, *record)
}
return &portainer.APIKey{}, nil
})
return result, err
}
// GetAPIKeyByDigest returns the API key for the associated digest.
// Note: there is a 1-to-1 mapping of api-key and digest
func (service *Service) GetAPIKeyByDigest(digest []byte) (*portainer.APIKey, error) {
var k *portainer.APIKey
stop := fmt.Errorf("ok")
err := service.connection.GetAll(
BucketName,
&portainer.APIKey{},
func(obj interface{}) (interface{}, error) {
key, ok := obj.(*portainer.APIKey)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to APIKey object")
return nil, fmt.Errorf("Failed to convert to APIKey object: %s", obj)
}
if bytes.Equal(key.Digest, digest) {
k = key
return nil, stop
}
return &portainer.APIKey{}, nil
})
if err == stop {
return k, nil
}
if err == nil {
return nil, errors.ErrObjectNotFound
}
return nil, err
}
// CreateAPIKey creates a new APIKey object.
func (service *Service) CreateAPIKey(record *portainer.APIKey) error {
return service.connection.CreateObject(
BucketName,
func(id uint64) (int, interface{}) {
record.ID = portainer.APIKeyID(id)
return int(record.ID), record
},
)
}
// GetAPIKey retrieves an existing APIKey object by api key ID.
func (service *Service) GetAPIKey(keyID portainer.APIKeyID) (*portainer.APIKey, error) {
var key portainer.APIKey
identifier := service.connection.ConvertToKey(int(keyID))
err := service.connection.GetObject(BucketName, identifier, &key)
if err != nil {
return nil, err
}
return &key, nil
}
func (service *Service) UpdateAPIKey(key *portainer.APIKey) error {
identifier := service.connection.ConvertToKey(int(key.ID))
return service.connection.UpdateObject(BucketName, identifier, key)
}
func (service *Service) DeleteAPIKey(ID portainer.APIKeyID) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.DeleteObject(BucketName, identifier)
}

View File

@ -0,0 +1,91 @@
package customtemplate
import (
"fmt"
portainer "github.com/portainer/portainer/api"
"github.com/sirupsen/logrus"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "customtemplates"
)
// Service represents a service for managing custom template data.
type Service struct {
connection portainer.Connection
}
func (service *Service) BucketName() string {
return BucketName
}
// NewService creates a new instance of a service.
func NewService(connection portainer.Connection) (*Service, error) {
err := connection.SetServiceName(BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// CustomTemplates return an array containing all the custom templates.
func (service *Service) CustomTemplates() ([]portainer.CustomTemplate, error) {
var customTemplates = make([]portainer.CustomTemplate, 0)
err := service.connection.GetAll(
BucketName,
&portainer.CustomTemplate{},
func(obj interface{}) (interface{}, error) {
//var tag portainer.Tag
customTemplate, ok := obj.(*portainer.CustomTemplate)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to CustomTemplate object")
return nil, fmt.Errorf("Failed to convert to CustomTemplate object: %s", obj)
}
customTemplates = append(customTemplates, *customTemplate)
return &portainer.CustomTemplate{}, nil
})
return customTemplates, err
}
// CustomTemplate returns an custom template by ID.
func (service *Service) CustomTemplate(ID portainer.CustomTemplateID) (*portainer.CustomTemplate, error) {
var customTemplate portainer.CustomTemplate
identifier := service.connection.ConvertToKey(int(ID))
err := service.connection.GetObject(BucketName, identifier, &customTemplate)
if err != nil {
return nil, err
}
return &customTemplate, nil
}
// UpdateCustomTemplate updates an custom template.
func (service *Service) UpdateCustomTemplate(ID portainer.CustomTemplateID, customTemplate *portainer.CustomTemplate) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.UpdateObject(BucketName, identifier, customTemplate)
}
// DeleteCustomTemplate deletes an custom template.
func (service *Service) DeleteCustomTemplate(ID portainer.CustomTemplateID) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.DeleteObject(BucketName, identifier)
}
// CreateCustomTemplate uses the existing id and saves it.
// TODO: where does the ID come from, and is it safe?
func (service *Service) Create(customTemplate *portainer.CustomTemplate) error {
return service.connection.CreateObjectWithId(BucketName, int(customTemplate.ID), customTemplate)
}
// GetNextIdentifier returns the next identifier for a custom template.
func (service *Service) GetNextIdentifier() int {
return service.connection.GetNextIdentifier(BucketName)
}

View File

@ -2,7 +2,6 @@ package dockerhub
import (
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/internal"
)
const (
@ -13,12 +12,16 @@ const (
// Service represents a service for managing Dockerhub data.
type Service struct {
connection *internal.DbConnection
connection portainer.Connection
}
func (service *Service) BucketName() string {
return BucketName
}
// NewService creates a new instance of a service.
func NewService(connection *internal.DbConnection) (*Service, error) {
err := internal.CreateBucket(connection, BucketName)
func NewService(connection portainer.Connection) (*Service, error) {
err := connection.SetServiceName(BucketName)
if err != nil {
return nil, err
}
@ -32,7 +35,7 @@ func NewService(connection *internal.DbConnection) (*Service, error) {
func (service *Service) DockerHub() (*portainer.DockerHub, error) {
var dockerhub portainer.DockerHub
err := internal.GetObject(service.connection, BucketName, []byte(dockerHubKey), &dockerhub)
err := service.connection.GetObject(BucketName, []byte(dockerHubKey), &dockerhub)
if err != nil {
return nil, err
}
@ -42,5 +45,5 @@ func (service *Service) DockerHub() (*portainer.DockerHub, error) {
// UpdateDockerHub updates a DockerHub object.
func (service *Service) UpdateDockerHub(dockerhub *portainer.DockerHub) error {
return internal.UpdateObject(service.connection, BucketName, []byte(dockerHubKey), dockerhub)
return service.connection.UpdateObject(BucketName, []byte(dockerHubKey), dockerhub)
}

View File

@ -0,0 +1,90 @@
package edgegroup
import (
"fmt"
portainer "github.com/portainer/portainer/api"
"github.com/sirupsen/logrus"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "edgegroups"
)
// Service represents a service for managing Edge group data.
type Service struct {
connection portainer.Connection
}
func (service *Service) BucketName() string {
return BucketName
}
// NewService creates a new instance of a service.
func NewService(connection portainer.Connection) (*Service, error) {
err := connection.SetServiceName(BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// EdgeGroups return an array containing all the Edge groups.
func (service *Service) EdgeGroups() ([]portainer.EdgeGroup, error) {
var groups = make([]portainer.EdgeGroup, 0)
err := service.connection.GetAllWithJsoniter(
BucketName,
&portainer.EdgeGroup{},
func(obj interface{}) (interface{}, error) {
group, ok := obj.(*portainer.EdgeGroup)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to EdgeGroup object")
return nil, fmt.Errorf("Failed to convert to EdgeGroup object: %s", obj)
}
groups = append(groups, *group)
return &portainer.EdgeGroup{}, nil
})
return groups, err
}
// EdgeGroup returns an Edge group by ID.
func (service *Service) EdgeGroup(ID portainer.EdgeGroupID) (*portainer.EdgeGroup, error) {
var group portainer.EdgeGroup
identifier := service.connection.ConvertToKey(int(ID))
err := service.connection.GetObject(BucketName, identifier, &group)
if err != nil {
return nil, err
}
return &group, nil
}
// UpdateEdgeGroup updates an Edge group.
func (service *Service) UpdateEdgeGroup(ID portainer.EdgeGroupID, group *portainer.EdgeGroup) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.UpdateObject(BucketName, identifier, group)
}
// DeleteEdgeGroup deletes an Edge group.
func (service *Service) DeleteEdgeGroup(ID portainer.EdgeGroupID) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.DeleteObject(BucketName, identifier)
}
// CreateEdgeGroup assign an ID to a new Edge group and saves it.
func (service *Service) Create(group *portainer.EdgeGroup) error {
return service.connection.CreateObject(
BucketName,
func(id uint64) (int, interface{}) {
group.ID = portainer.EdgeGroupID(id)
return int(group.ID), group
},
)
}

View File

@ -0,0 +1,96 @@
package edgejob
import (
"fmt"
portainer "github.com/portainer/portainer/api"
"github.com/sirupsen/logrus"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "edgejobs"
)
// Service represents a service for managing edge jobs data.
type Service struct {
connection portainer.Connection
}
func (service *Service) BucketName() string {
return BucketName
}
// NewService creates a new instance of a service.
func NewService(connection portainer.Connection) (*Service, error) {
err := connection.SetServiceName(BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// EdgeJobs returns a list of Edge jobs
func (service *Service) EdgeJobs() ([]portainer.EdgeJob, error) {
var edgeJobs = make([]portainer.EdgeJob, 0)
err := service.connection.GetAll(
BucketName,
&portainer.EdgeJob{},
func(obj interface{}) (interface{}, error) {
//var tag portainer.Tag
job, ok := obj.(*portainer.EdgeJob)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to EdgeJob object")
return nil, fmt.Errorf("Failed to convert to EdgeJob object: %s", obj)
}
edgeJobs = append(edgeJobs, *job)
return &portainer.EdgeJob{}, nil
})
return edgeJobs, err
}
// EdgeJob returns an Edge job by ID
func (service *Service) EdgeJob(ID portainer.EdgeJobID) (*portainer.EdgeJob, error) {
var edgeJob portainer.EdgeJob
identifier := service.connection.ConvertToKey(int(ID))
err := service.connection.GetObject(BucketName, identifier, &edgeJob)
if err != nil {
return nil, err
}
return &edgeJob, nil
}
// CreateEdgeJob creates a new Edge job
func (service *Service) Create(edgeJob *portainer.EdgeJob) error {
return service.connection.CreateObject(
BucketName,
func(id uint64) (int, interface{}) {
edgeJob.ID = portainer.EdgeJobID(id)
return int(edgeJob.ID), edgeJob
},
)
}
// UpdateEdgeJob updates an Edge job by ID
func (service *Service) UpdateEdgeJob(ID portainer.EdgeJobID, edgeJob *portainer.EdgeJob) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.UpdateObject(BucketName, identifier, edgeJob)
}
// DeleteEdgeJob deletes an Edge job
func (service *Service) DeleteEdgeJob(ID portainer.EdgeJobID) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.DeleteObject(BucketName, identifier)
}
// GetNextIdentifier returns the next identifier for an environment(endpoint).
func (service *Service) GetNextIdentifier() int {
return service.connection.GetNextIdentifier(BucketName)
}

View File

@ -0,0 +1,96 @@
package edgestack
import (
"fmt"
portainer "github.com/portainer/portainer/api"
"github.com/sirupsen/logrus"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "edge_stack"
)
// Service represents a service for managing Edge stack data.
type Service struct {
connection portainer.Connection
}
func (service *Service) BucketName() string {
return BucketName
}
// NewService creates a new instance of a service.
func NewService(connection portainer.Connection) (*Service, error) {
err := connection.SetServiceName(BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// EdgeStacks returns an array containing all edge stacks
func (service *Service) EdgeStacks() ([]portainer.EdgeStack, error) {
var stacks = make([]portainer.EdgeStack, 0)
err := service.connection.GetAll(
BucketName,
&portainer.EdgeStack{},
func(obj interface{}) (interface{}, error) {
//var tag portainer.Tag
stack, ok := obj.(*portainer.EdgeStack)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to EdgeStack object")
return nil, fmt.Errorf("Failed to convert to EdgeStack object: %s", obj)
}
stacks = append(stacks, *stack)
return &portainer.EdgeStack{}, nil
})
return stacks, err
}
// EdgeStack returns an Edge stack by ID.
func (service *Service) EdgeStack(ID portainer.EdgeStackID) (*portainer.EdgeStack, error) {
var stack portainer.EdgeStack
identifier := service.connection.ConvertToKey(int(ID))
err := service.connection.GetObject(BucketName, identifier, &stack)
if err != nil {
return nil, err
}
return &stack, nil
}
// CreateEdgeStack assign an ID to a new Edge stack and saves it.
func (service *Service) Create(edgeStack *portainer.EdgeStack) error {
return service.connection.CreateObject(
BucketName,
func(id uint64) (int, interface{}) {
edgeStack.ID = portainer.EdgeStackID(id)
return int(edgeStack.ID), edgeStack
},
)
}
// UpdateEdgeStack updates an Edge stack.
func (service *Service) UpdateEdgeStack(ID portainer.EdgeStackID, edgeStack *portainer.EdgeStack) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.UpdateObject(BucketName, identifier, edgeStack)
}
// DeleteEdgeStack deletes an Edge stack.
func (service *Service) DeleteEdgeStack(ID portainer.EdgeStackID) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.DeleteObject(BucketName, identifier)
}
// GetNextIdentifier returns the next identifier for an environment(endpoint).
func (service *Service) GetNextIdentifier() int {
return service.connection.GetNextIdentifier(BucketName)
}

View File

@ -0,0 +1,89 @@
package endpoint
import (
"fmt"
portainer "github.com/portainer/portainer/api"
"github.com/sirupsen/logrus"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "endpoints"
)
// Service represents a service for managing environment(endpoint) data.
type Service struct {
connection portainer.Connection
}
func (service *Service) BucketName() string {
return BucketName
}
// NewService creates a new instance of a service.
func NewService(connection portainer.Connection) (*Service, error) {
err := connection.SetServiceName(BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// Endpoint returns an environment(endpoint) by ID.
func (service *Service) Endpoint(ID portainer.EndpointID) (*portainer.Endpoint, error) {
var endpoint portainer.Endpoint
identifier := service.connection.ConvertToKey(int(ID))
err := service.connection.GetObject(BucketName, identifier, &endpoint)
if err != nil {
return nil, err
}
return &endpoint, nil
}
// UpdateEndpoint updates an environment(endpoint).
func (service *Service) UpdateEndpoint(ID portainer.EndpointID, endpoint *portainer.Endpoint) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.UpdateObject(BucketName, identifier, endpoint)
}
// DeleteEndpoint deletes an environment(endpoint).
func (service *Service) DeleteEndpoint(ID portainer.EndpointID) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.DeleteObject(BucketName, identifier)
}
// Endpoints return an array containing all the environments(endpoints).
func (service *Service) Endpoints() ([]portainer.Endpoint, error) {
var endpoints = make([]portainer.Endpoint, 0)
err := service.connection.GetAllWithJsoniter(
BucketName,
&portainer.Endpoint{},
func(obj interface{}) (interface{}, error) {
endpoint, ok := obj.(*portainer.Endpoint)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to Endpoint object")
return nil, fmt.Errorf("Failed to convert to Endpoint object: %s", obj)
}
endpoints = append(endpoints, *endpoint)
return &portainer.Endpoint{}, nil
})
return endpoints, err
}
// CreateEndpoint assign an ID to a new environment(endpoint) and saves it.
func (service *Service) Create(endpoint *portainer.Endpoint) error {
return service.connection.CreateObjectWithSetSequence(BucketName, int(endpoint.ID), endpoint)
}
// GetNextIdentifier returns the next identifier for an environment(endpoint).
func (service *Service) GetNextIdentifier() int {
return service.connection.GetNextIdentifier(BucketName)
}

View File

@ -0,0 +1,91 @@
package endpointgroup
import (
"fmt"
portainer "github.com/portainer/portainer/api"
"github.com/sirupsen/logrus"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "endpoint_groups"
)
// Service represents a service for managing environment(endpoint) data.
type Service struct {
connection portainer.Connection
}
func (service *Service) BucketName() string {
return BucketName
}
// NewService creates a new instance of a service.
func NewService(connection portainer.Connection) (*Service, error) {
err := connection.SetServiceName(BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// EndpointGroup returns an environment(endpoint) group by ID.
func (service *Service) EndpointGroup(ID portainer.EndpointGroupID) (*portainer.EndpointGroup, error) {
var endpointGroup portainer.EndpointGroup
identifier := service.connection.ConvertToKey(int(ID))
err := service.connection.GetObject(BucketName, identifier, &endpointGroup)
if err != nil {
return nil, err
}
return &endpointGroup, nil
}
// UpdateEndpointGroup updates an environment(endpoint) group.
func (service *Service) UpdateEndpointGroup(ID portainer.EndpointGroupID, endpointGroup *portainer.EndpointGroup) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.UpdateObject(BucketName, identifier, endpointGroup)
}
// DeleteEndpointGroup deletes an environment(endpoint) group.
func (service *Service) DeleteEndpointGroup(ID portainer.EndpointGroupID) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.DeleteObject(BucketName, identifier)
}
// EndpointGroups return an array containing all the environment(endpoint) groups.
func (service *Service) EndpointGroups() ([]portainer.EndpointGroup, error) {
var endpointGroups = make([]portainer.EndpointGroup, 0)
err := service.connection.GetAll(
BucketName,
&portainer.EndpointGroup{},
func(obj interface{}) (interface{}, error) {
//var tag portainer.Tag
endpointGroup, ok := obj.(*portainer.EndpointGroup)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to EndpointGroup object")
return nil, fmt.Errorf("Failed to convert to EndpointGroup object: %s", obj)
}
endpointGroups = append(endpointGroups, *endpointGroup)
return &portainer.EndpointGroup{}, nil
})
return endpointGroups, err
}
// CreateEndpointGroup assign an ID to a new environment(endpoint) group and saves it.
func (service *Service) Create(endpointGroup *portainer.EndpointGroup) error {
return service.connection.CreateObject(
BucketName,
func(id uint64) (int, interface{}) {
endpointGroup.ID = portainer.EndpointGroupID(id)
return int(endpointGroup.ID), endpointGroup
},
)
}

View File

@ -0,0 +1,84 @@
package endpointrelation
import (
"fmt"
portainer "github.com/portainer/portainer/api"
"github.com/sirupsen/logrus"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "endpoint_relations"
)
// Service represents a service for managing environment(endpoint) relation data.
type Service struct {
connection portainer.Connection
}
func (service *Service) BucketName() string {
return BucketName
}
// NewService creates a new instance of a service.
func NewService(connection portainer.Connection) (*Service, error) {
err := connection.SetServiceName(BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
//EndpointRelations returns an array of all EndpointRelations
func (service *Service) EndpointRelations() ([]portainer.EndpointRelation, error) {
var all = make([]portainer.EndpointRelation, 0)
err := service.connection.GetAll(
BucketName,
&portainer.EndpointRelation{},
func(obj interface{}) (interface{}, error) {
r, ok := obj.(*portainer.EndpointRelation)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to EndpointRelation object")
return nil, fmt.Errorf("Failed to convert to EndpointRelation object: %s", obj)
}
all = append(all, *r)
return &portainer.EndpointRelation{}, nil
})
return all, err
}
// EndpointRelation returns a Environment(Endpoint) relation object by EndpointID
func (service *Service) EndpointRelation(endpointID portainer.EndpointID) (*portainer.EndpointRelation, error) {
var endpointRelation portainer.EndpointRelation
identifier := service.connection.ConvertToKey(int(endpointID))
err := service.connection.GetObject(BucketName, identifier, &endpointRelation)
if err != nil {
return nil, err
}
return &endpointRelation, nil
}
// CreateEndpointRelation saves endpointRelation
func (service *Service) Create(endpointRelation *portainer.EndpointRelation) error {
return service.connection.CreateObjectWithId(BucketName, int(endpointRelation.EndpointID), endpointRelation)
}
// UpdateEndpointRelation updates an Environment(Endpoint) relation object
func (service *Service) UpdateEndpointRelation(EndpointID portainer.EndpointID, endpointRelation *portainer.EndpointRelation) error {
identifier := service.connection.ConvertToKey(int(EndpointID))
return service.connection.UpdateObject(BucketName, identifier, endpointRelation)
}
// DeleteEndpointRelation deletes an Environment(Endpoint) relation object
func (service *Service) DeleteEndpointRelation(EndpointID portainer.EndpointID) error {
identifier := service.connection.ConvertToKey(int(EndpointID))
return service.connection.DeleteObject(BucketName, identifier)
}

View File

@ -3,6 +3,7 @@ package errors
import "errors"
var (
// TODO: i'm pretty sure this needs wrapping at several levels
ErrObjectNotFound = errors.New("Object not found inside the database")
ErrWrongDBEdition = errors.New("The Portainer database is set for Portainer Business Edition, please follow the instructions in our documentation to downgrade it: https://documentation.portainer.io/v2.0-be/downgrade/be-to-ce/")
)

View File

@ -1,10 +1,10 @@
package extension
import (
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/internal"
"fmt"
"github.com/boltdb/bolt"
portainer "github.com/portainer/portainer/api"
"github.com/sirupsen/logrus"
)
const (
@ -14,12 +14,16 @@ const (
// Service represents a service for managing environment(endpoint) data.
type Service struct {
connection *internal.DbConnection
connection portainer.Connection
}
func (service *Service) BucketName() string {
return BucketName
}
// NewService creates a new instance of a service.
func NewService(connection *internal.DbConnection) (*Service, error) {
err := internal.CreateBucket(connection, BucketName)
func NewService(connection portainer.Connection) (*Service, error) {
err := connection.SetServiceName(BucketName)
if err != nil {
return nil, err
}
@ -32,9 +36,9 @@ func NewService(connection *internal.DbConnection) (*Service, error) {
// Extension returns a extension by ID
func (service *Service) Extension(ID portainer.ExtensionID) (*portainer.Extension, error) {
var extension portainer.Extension
identifier := internal.Itob(int(ID))
identifier := service.connection.ConvertToKey(int(ID))
err := internal.GetObject(service.connection, BucketName, identifier, &extension)
err := service.connection.GetObject(BucketName, identifier, &extension)
if err != nil {
return nil, err
}
@ -46,41 +50,29 @@ func (service *Service) Extension(ID portainer.ExtensionID) (*portainer.Extensio
func (service *Service) Extensions() ([]portainer.Extension, error) {
var extensions = make([]portainer.Extension, 0)
err := service.connection.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
cursor := bucket.Cursor()
for k, v := cursor.First(); k != nil; k, v = cursor.Next() {
var extension portainer.Extension
err := internal.UnmarshalObject(v, &extension)
if err != nil {
return err
err := service.connection.GetAll(
BucketName,
&portainer.Extension{},
func(obj interface{}) (interface{}, error) {
extension, ok := obj.(*portainer.Extension)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to Extension object")
return nil, fmt.Errorf("Failed to convert to Extension object: %s", obj)
}
extensions = append(extensions, extension)
}
return nil
})
extensions = append(extensions, *extension)
return &portainer.Extension{}, nil
})
return extensions, err
}
// Persist persists a extension inside the database.
func (service *Service) Persist(extension *portainer.Extension) error {
return service.connection.Update(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(BucketName))
data, err := internal.MarshalObject(extension)
if err != nil {
return err
}
return bucket.Put(internal.Itob(int(extension.ID)), data)
})
return service.connection.CreateObjectWithId(BucketName, int(extension.ID), extension)
}
// DeleteExtension deletes a Extension.
func (service *Service) DeleteExtension(ID portainer.ExtensionID) error {
identifier := internal.Itob(int(ID))
return internal.DeleteObject(service.connection, BucketName, identifier)
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.DeleteObject(BucketName, identifier)
}

View File

@ -0,0 +1,99 @@
package helmuserrepository
import (
"fmt"
portainer "github.com/portainer/portainer/api"
"github.com/sirupsen/logrus"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "helm_user_repository"
)
// Service represents a service for managing environment(endpoint) data.
type Service struct {
connection portainer.Connection
}
func (service *Service) BucketName() string {
return BucketName
}
// NewService creates a new instance of a service.
func NewService(connection portainer.Connection) (*Service, error) {
err := connection.SetServiceName(BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
//HelmUserRepository returns an array of all HelmUserRepository
func (service *Service) HelmUserRepositorys() ([]portainer.HelmUserRepository, error) {
var repos = make([]portainer.HelmUserRepository, 0)
err := service.connection.GetAll(
BucketName,
&portainer.HelmUserRepository{},
func(obj interface{}) (interface{}, error) {
r, ok := obj.(*portainer.HelmUserRepository)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to HelmUserRepository object")
return nil, fmt.Errorf("Failed to convert to HelmUserRepository object: %s", obj)
}
repos = append(repos, *r)
return &portainer.HelmUserRepository{}, nil
})
return repos, err
}
// HelmUserRepositoryByUserID return an array containing all the HelmUserRepository objects where the specified userID is present.
func (service *Service) HelmUserRepositoryByUserID(userID portainer.UserID) ([]portainer.HelmUserRepository, error) {
var result = make([]portainer.HelmUserRepository, 0)
err := service.connection.GetAll(
BucketName,
&portainer.HelmUserRepository{},
func(obj interface{}) (interface{}, error) {
record, ok := obj.(*portainer.HelmUserRepository)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to HelmUserRepository object")
return nil, fmt.Errorf("Failed to convert to HelmUserRepository object: %s", obj)
}
if record.UserID == userID {
result = append(result, *record)
}
return &portainer.HelmUserRepository{}, nil
})
return result, err
}
// CreateHelmUserRepository creates a new HelmUserRepository object.
func (service *Service) Create(record *portainer.HelmUserRepository) error {
return service.connection.CreateObject(
BucketName,
func(id uint64) (int, interface{}) {
record.ID = portainer.HelmUserRepositoryID(id)
return int(record.ID), record
},
)
}
// UpdateHelmUserRepostory updates an registry.
func (service *Service) UpdateHelmUserRepository(ID portainer.HelmUserRepositoryID, registry *portainer.HelmUserRepository) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.UpdateObject(BucketName, identifier, registry)
}
// DeleteHelmUserRepository deletes an registry.
func (service *Service) DeleteHelmUserRepository(ID portainer.HelmUserRepositoryID) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.DeleteObject(BucketName, identifier)
}

View File

@ -0,0 +1,293 @@
package dataservices
// "github.com/portainer/portainer/api/dataservices"
import (
"io"
"time"
"github.com/portainer/portainer/api/dataservices/errors"
portainer "github.com/portainer/portainer/api"
)
type (
// DataStore defines the interface to manage the data
DataStore interface {
Open() (newStore bool, err error)
Init() error
Close() error
MigrateData() error
Rollback(force bool) error
CheckCurrentEdition() error
BackupTo(w io.Writer) error
Export(filename string) (err error)
IsErrObjectNotFound(err error) bool
CustomTemplate() CustomTemplateService
EdgeGroup() EdgeGroupService
EdgeJob() EdgeJobService
EdgeStack() EdgeStackService
Endpoint() EndpointService
EndpointGroup() EndpointGroupService
EndpointRelation() EndpointRelationService
HelmUserRepository() HelmUserRepositoryService
Registry() RegistryService
ResourceControl() ResourceControlService
Role() RoleService
APIKeyRepository() APIKeyRepository
Settings() SettingsService
SSLSettings() SSLSettingsService
Stack() StackService
Tag() TagService
TeamMembership() TeamMembershipService
Team() TeamService
TunnelServer() TunnelServerService
User() UserService
Version() VersionService
Webhook() WebhookService
}
// CustomTemplateService represents a service to manage custom templates
CustomTemplateService interface {
GetNextIdentifier() int
CustomTemplates() ([]portainer.CustomTemplate, error)
CustomTemplate(ID portainer.CustomTemplateID) (*portainer.CustomTemplate, error)
Create(customTemplate *portainer.CustomTemplate) error
UpdateCustomTemplate(ID portainer.CustomTemplateID, customTemplate *portainer.CustomTemplate) error
DeleteCustomTemplate(ID portainer.CustomTemplateID) error
BucketName() string
}
// EdgeGroupService represents a service to manage Edge groups
EdgeGroupService interface {
EdgeGroups() ([]portainer.EdgeGroup, error)
EdgeGroup(ID portainer.EdgeGroupID) (*portainer.EdgeGroup, error)
Create(group *portainer.EdgeGroup) error
UpdateEdgeGroup(ID portainer.EdgeGroupID, group *portainer.EdgeGroup) error
DeleteEdgeGroup(ID portainer.EdgeGroupID) error
BucketName() string
}
// EdgeJobService represents a service to manage Edge jobs
EdgeJobService interface {
EdgeJobs() ([]portainer.EdgeJob, error)
EdgeJob(ID portainer.EdgeJobID) (*portainer.EdgeJob, error)
Create(edgeJob *portainer.EdgeJob) error
UpdateEdgeJob(ID portainer.EdgeJobID, edgeJob *portainer.EdgeJob) error
DeleteEdgeJob(ID portainer.EdgeJobID) error
GetNextIdentifier() int
BucketName() string
}
// EdgeStackService represents a service to manage Edge stacks
EdgeStackService interface {
EdgeStacks() ([]portainer.EdgeStack, error)
EdgeStack(ID portainer.EdgeStackID) (*portainer.EdgeStack, error)
Create(edgeStack *portainer.EdgeStack) error
UpdateEdgeStack(ID portainer.EdgeStackID, edgeStack *portainer.EdgeStack) error
DeleteEdgeStack(ID portainer.EdgeStackID) error
GetNextIdentifier() int
BucketName() string
}
// EndpointService represents a service for managing environment(endpoint) data
EndpointService interface {
Endpoint(ID portainer.EndpointID) (*portainer.Endpoint, error)
Endpoints() ([]portainer.Endpoint, error)
Create(endpoint *portainer.Endpoint) error
UpdateEndpoint(ID portainer.EndpointID, endpoint *portainer.Endpoint) error
DeleteEndpoint(ID portainer.EndpointID) error
GetNextIdentifier() int
BucketName() string
}
// EndpointGroupService represents a service for managing environment(endpoint) group data
EndpointGroupService interface {
EndpointGroup(ID portainer.EndpointGroupID) (*portainer.EndpointGroup, error)
EndpointGroups() ([]portainer.EndpointGroup, error)
Create(group *portainer.EndpointGroup) error
UpdateEndpointGroup(ID portainer.EndpointGroupID, group *portainer.EndpointGroup) error
DeleteEndpointGroup(ID portainer.EndpointGroupID) error
BucketName() string
}
// EndpointRelationService represents a service for managing environment(endpoint) relations data
EndpointRelationService interface {
EndpointRelations() ([]portainer.EndpointRelation, error)
EndpointRelation(EndpointID portainer.EndpointID) (*portainer.EndpointRelation, error)
Create(endpointRelation *portainer.EndpointRelation) error
UpdateEndpointRelation(EndpointID portainer.EndpointID, endpointRelation *portainer.EndpointRelation) error
DeleteEndpointRelation(EndpointID portainer.EndpointID) error
BucketName() string
}
// HelmUserRepositoryService represents a service to manage HelmUserRepositories
HelmUserRepositoryService interface {
HelmUserRepositorys() ([]portainer.HelmUserRepository, error)
HelmUserRepositoryByUserID(userID portainer.UserID) ([]portainer.HelmUserRepository, error)
Create(record *portainer.HelmUserRepository) error
UpdateHelmUserRepository(ID portainer.HelmUserRepositoryID, repository *portainer.HelmUserRepository) error
DeleteHelmUserRepository(ID portainer.HelmUserRepositoryID) error
BucketName() string
}
// JWTService represents a service for managing JWT tokens
JWTService interface {
GenerateToken(data *portainer.TokenData) (string, error)
GenerateTokenForOAuth(data *portainer.TokenData, expiryTime *time.Time) (string, error)
GenerateTokenForKubeconfig(data *portainer.TokenData) (string, error)
ParseAndVerifyToken(token string) (*portainer.TokenData, error)
SetUserSessionDuration(userSessionDuration time.Duration)
}
// RegistryService represents a service for managing registry data
RegistryService interface {
Registry(ID portainer.RegistryID) (*portainer.Registry, error)
Registries() ([]portainer.Registry, error)
Create(registry *portainer.Registry) error
UpdateRegistry(ID portainer.RegistryID, registry *portainer.Registry) error
DeleteRegistry(ID portainer.RegistryID) error
BucketName() string
}
// ResourceControlService represents a service for managing resource control data
ResourceControlService interface {
ResourceControl(ID portainer.ResourceControlID) (*portainer.ResourceControl, error)
ResourceControlByResourceIDAndType(resourceID string, resourceType portainer.ResourceControlType) (*portainer.ResourceControl, error)
ResourceControls() ([]portainer.ResourceControl, error)
Create(rc *portainer.ResourceControl) error
UpdateResourceControl(ID portainer.ResourceControlID, resourceControl *portainer.ResourceControl) error
DeleteResourceControl(ID portainer.ResourceControlID) error
BucketName() string
}
// RoleService represents a service for managing user roles
RoleService interface {
Role(ID portainer.RoleID) (*portainer.Role, error)
Roles() ([]portainer.Role, error)
Create(role *portainer.Role) error
UpdateRole(ID portainer.RoleID, role *portainer.Role) error
BucketName() string
}
// APIKeyRepositoryService
APIKeyRepository interface {
CreateAPIKey(key *portainer.APIKey) error
GetAPIKey(keyID portainer.APIKeyID) (*portainer.APIKey, error)
UpdateAPIKey(key *portainer.APIKey) error
DeleteAPIKey(ID portainer.APIKeyID) error
GetAPIKeysByUserID(userID portainer.UserID) ([]portainer.APIKey, error)
GetAPIKeyByDigest(digest []byte) (*portainer.APIKey, error)
}
// SettingsService represents a service for managing application settings
SettingsService interface {
Settings() (*portainer.Settings, error)
UpdateSettings(settings *portainer.Settings) error
IsFeatureFlagEnabled(feature portainer.Feature) bool
BucketName() string
}
// SSLSettingsService represents a service for managing application settings
SSLSettingsService interface {
Settings() (*portainer.SSLSettings, error)
UpdateSettings(settings *portainer.SSLSettings) error
BucketName() string
}
// StackService represents a service for managing stack data
StackService interface {
Stack(ID portainer.StackID) (*portainer.Stack, error)
StackByName(name string) (*portainer.Stack, error)
StacksByName(name string) ([]portainer.Stack, error)
Stacks() ([]portainer.Stack, error)
Create(stack *portainer.Stack) error
UpdateStack(ID portainer.StackID, stack *portainer.Stack) error
DeleteStack(ID portainer.StackID) error
GetNextIdentifier() int
StackByWebhookID(ID string) (*portainer.Stack, error)
RefreshableStacks() ([]portainer.Stack, error)
BucketName() string
}
// TagService represents a service for managing tag data
TagService interface {
Tags() ([]portainer.Tag, error)
Tag(ID portainer.TagID) (*portainer.Tag, error)
Create(tag *portainer.Tag) error
UpdateTag(ID portainer.TagID, tag *portainer.Tag) error
DeleteTag(ID portainer.TagID) error
BucketName() string
}
// TeamService represents a service for managing user data
TeamService interface {
Team(ID portainer.TeamID) (*portainer.Team, error)
TeamByName(name string) (*portainer.Team, error)
Teams() ([]portainer.Team, error)
Create(team *portainer.Team) error
UpdateTeam(ID portainer.TeamID, team *portainer.Team) error
DeleteTeam(ID portainer.TeamID) error
BucketName() string
}
// TeamMembershipService represents a service for managing team membership data
TeamMembershipService interface {
TeamMembership(ID portainer.TeamMembershipID) (*portainer.TeamMembership, error)
TeamMemberships() ([]portainer.TeamMembership, error)
TeamMembershipsByUserID(userID portainer.UserID) ([]portainer.TeamMembership, error)
TeamMembershipsByTeamID(teamID portainer.TeamID) ([]portainer.TeamMembership, error)
Create(membership *portainer.TeamMembership) error
UpdateTeamMembership(ID portainer.TeamMembershipID, membership *portainer.TeamMembership) error
DeleteTeamMembership(ID portainer.TeamMembershipID) error
DeleteTeamMembershipByUserID(userID portainer.UserID) error
DeleteTeamMembershipByTeamID(teamID portainer.TeamID) error
BucketName() string
}
// TunnelServerService represents a service for managing data associated to the tunnel server
TunnelServerService interface {
Info() (*portainer.TunnelServerInfo, error)
UpdateInfo(info *portainer.TunnelServerInfo) error
BucketName() string
}
// UserService represents a service for managing user data
UserService interface {
User(ID portainer.UserID) (*portainer.User, error)
UserByUsername(username string) (*portainer.User, error)
Users() ([]portainer.User, error)
UsersByRole(role portainer.UserRole) ([]portainer.User, error)
Create(user *portainer.User) error
UpdateUser(ID portainer.UserID, user *portainer.User) error
DeleteUser(ID portainer.UserID) error
BucketName() string
}
// VersionService represents a service for managing version data
VersionService interface {
DBVersion() (int, error)
Edition() (portainer.SoftwareEdition, error)
InstanceID() (string, error)
StoreDBVersion(version int) error
StoreInstanceID(ID string) error
BucketName() string
}
// WebhookService represents a service for managing webhook data.
WebhookService interface {
Webhooks() ([]portainer.Webhook, error)
Webhook(ID portainer.WebhookID) (*portainer.Webhook, error)
Create(portainer *portainer.Webhook) error
UpdateWebhook(ID portainer.WebhookID, webhook *portainer.Webhook) error
WebhookByResourceID(resourceID string) (*portainer.Webhook, error)
WebhookByToken(token string) (*portainer.Webhook, error)
DeleteWebhook(ID portainer.WebhookID) error
BucketName() string
}
)
func IsErrObjectNotFound(e error) bool {
return e == errors.ErrObjectNotFound
}

View File

@ -0,0 +1,90 @@
package registry
import (
"fmt"
portainer "github.com/portainer/portainer/api"
"github.com/sirupsen/logrus"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "registries"
)
// Service represents a service for managing environment(endpoint) data.
type Service struct {
connection portainer.Connection
}
func (service *Service) BucketName() string {
return BucketName
}
// NewService creates a new instance of a service.
func NewService(connection portainer.Connection) (*Service, error) {
err := connection.SetServiceName(BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// Registry returns an registry by ID.
func (service *Service) Registry(ID portainer.RegistryID) (*portainer.Registry, error) {
var registry portainer.Registry
identifier := service.connection.ConvertToKey(int(ID))
err := service.connection.GetObject(BucketName, identifier, &registry)
if err != nil {
return nil, err
}
return &registry, nil
}
// Registries returns an array containing all the registries.
func (service *Service) Registries() ([]portainer.Registry, error) {
var registries = make([]portainer.Registry, 0)
err := service.connection.GetAll(
BucketName,
&portainer.Registry{},
func(obj interface{}) (interface{}, error) {
registry, ok := obj.(*portainer.Registry)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to Registry object")
return nil, fmt.Errorf("Failed to convert to Registry object: %s", obj)
}
registries = append(registries, *registry)
return &portainer.Registry{}, nil
})
return registries, err
}
// CreateRegistry creates a new registry.
func (service *Service) Create(registry *portainer.Registry) error {
return service.connection.CreateObject(
BucketName,
func(id uint64) (int, interface{}) {
registry.ID = portainer.RegistryID(id)
return int(registry.ID), registry
},
)
}
// UpdateRegistry updates an registry.
func (service *Service) UpdateRegistry(ID portainer.RegistryID, registry *portainer.Registry) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.UpdateObject(BucketName, identifier, registry)
}
// DeleteRegistry deletes an registry.
func (service *Service) DeleteRegistry(ID portainer.RegistryID) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.DeleteObject(BucketName, identifier)
}

View File

@ -0,0 +1,130 @@
package resourcecontrol
import (
"fmt"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices/errors"
"github.com/sirupsen/logrus"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "resource_control"
)
// Service represents a service for managing environment(endpoint) data.
type Service struct {
connection portainer.Connection
}
func (service *Service) BucketName() string {
return BucketName
}
// NewService creates a new instance of a service.
func NewService(connection portainer.Connection) (*Service, error) {
err := connection.SetServiceName(BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// ResourceControl returns a ResourceControl object by ID
func (service *Service) ResourceControl(ID portainer.ResourceControlID) (*portainer.ResourceControl, error) {
var resourceControl portainer.ResourceControl
identifier := service.connection.ConvertToKey(int(ID))
err := service.connection.GetObject(BucketName, identifier, &resourceControl)
if err != nil {
return nil, err
}
return &resourceControl, nil
}
// ResourceControlByResourceIDAndType returns a ResourceControl object by checking if the resourceID is equal
// to the main ResourceID or in SubResourceIDs. It also performs a check on the resource type. Return nil
// if no ResourceControl was found.
func (service *Service) ResourceControlByResourceIDAndType(resourceID string, resourceType portainer.ResourceControlType) (*portainer.ResourceControl, error) {
var resourceControl *portainer.ResourceControl
stop := fmt.Errorf("ok")
err := service.connection.GetAll(
BucketName,
&portainer.ResourceControl{},
func(obj interface{}) (interface{}, error) {
rc, ok := obj.(*portainer.ResourceControl)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to ResourceControl object")
return nil, fmt.Errorf("Failed to convert to ResourceControl object: %s", obj)
}
if rc.ResourceID == resourceID && rc.Type == resourceType {
resourceControl = rc
return nil, stop
}
for _, subResourceID := range rc.SubResourceIDs {
if subResourceID == resourceID {
resourceControl = rc
return nil, stop
}
}
return &portainer.ResourceControl{}, nil
})
if err == stop {
return resourceControl, nil
}
if err == nil {
return nil, errors.ErrObjectNotFound
}
return nil, err
}
// ResourceControls returns all the ResourceControl objects
func (service *Service) ResourceControls() ([]portainer.ResourceControl, error) {
var rcs = make([]portainer.ResourceControl, 0)
err := service.connection.GetAll(
BucketName,
&portainer.ResourceControl{},
func(obj interface{}) (interface{}, error) {
rc, ok := obj.(*portainer.ResourceControl)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to ResourceControl object")
return nil, fmt.Errorf("Failed to convert to ResourceControl object: %s", obj)
}
rcs = append(rcs, *rc)
return &portainer.ResourceControl{}, nil
})
return rcs, err
}
// CreateResourceControl creates a new ResourceControl object
func (service *Service) Create(resourceControl *portainer.ResourceControl) error {
return service.connection.CreateObject(
BucketName,
func(id uint64) (int, interface{}) {
resourceControl.ID = portainer.ResourceControlID(id)
return int(resourceControl.ID), resourceControl
},
)
}
// UpdateResourceControl saves a ResourceControl object.
func (service *Service) UpdateResourceControl(ID portainer.ResourceControlID, resourceControl *portainer.ResourceControl) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.UpdateObject(BucketName, identifier, resourceControl)
}
// DeleteResourceControl deletes a ResourceControl object by ID
func (service *Service) DeleteResourceControl(ID portainer.ResourceControlID) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.DeleteObject(BucketName, identifier)
}

View File

@ -0,0 +1,84 @@
package role
import (
"fmt"
portainer "github.com/portainer/portainer/api"
"github.com/sirupsen/logrus"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "roles"
)
// Service represents a service for managing environment(endpoint) data.
type Service struct {
connection portainer.Connection
}
func (service *Service) BucketName() string {
return BucketName
}
// NewService creates a new instance of a service.
func NewService(connection portainer.Connection) (*Service, error) {
err := connection.SetServiceName(BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// Role returns a Role by ID
func (service *Service) Role(ID portainer.RoleID) (*portainer.Role, error) {
var set portainer.Role
identifier := service.connection.ConvertToKey(int(ID))
err := service.connection.GetObject(BucketName, identifier, &set)
if err != nil {
return nil, err
}
return &set, nil
}
// Roles return an array containing all the sets.
func (service *Service) Roles() ([]portainer.Role, error) {
var sets = make([]portainer.Role, 0)
err := service.connection.GetAll(
BucketName,
&portainer.Role{},
func(obj interface{}) (interface{}, error) {
set, ok := obj.(*portainer.Role)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to Role object")
return nil, fmt.Errorf("Failed to convert to Role object: %s", obj)
}
sets = append(sets, *set)
return &portainer.Role{}, nil
})
return sets, err
}
// CreateRole creates a new Role.
func (service *Service) Create(role *portainer.Role) error {
return service.connection.CreateObject(
BucketName,
func(id uint64) (int, interface{}) {
role.ID = portainer.RoleID(id)
return int(role.ID), role
},
)
}
// UpdateRole updates a role.
func (service *Service) UpdateRole(ID portainer.RoleID, role *portainer.Role) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.UpdateObject(BucketName, identifier, role)
}

View File

@ -0,0 +1,112 @@
package schedule
import (
"fmt"
portainer "github.com/portainer/portainer/api"
"github.com/sirupsen/logrus"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "schedules"
)
// Service represents a service for managing schedule data.
type Service struct {
connection portainer.Connection
}
func (service *Service) BucketName() string {
return BucketName
}
// NewService creates a new instance of a service.
func NewService(connection portainer.Connection) (*Service, error) {
err := connection.SetServiceName(BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// Schedule returns a schedule by ID.
func (service *Service) Schedule(ID portainer.ScheduleID) (*portainer.Schedule, error) {
var schedule portainer.Schedule
identifier := service.connection.ConvertToKey(int(ID))
err := service.connection.GetObject(BucketName, identifier, &schedule)
if err != nil {
return nil, err
}
return &schedule, nil
}
// UpdateSchedule updates a schedule.
func (service *Service) UpdateSchedule(ID portainer.ScheduleID, schedule *portainer.Schedule) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.UpdateObject(BucketName, identifier, schedule)
}
// DeleteSchedule deletes a schedule.
func (service *Service) DeleteSchedule(ID portainer.ScheduleID) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.DeleteObject(BucketName, identifier)
}
// Schedules return a array containing all the schedules.
func (service *Service) Schedules() ([]portainer.Schedule, error) {
var schedules = make([]portainer.Schedule, 0)
err := service.connection.GetAll(
BucketName,
&portainer.Schedule{},
func(obj interface{}) (interface{}, error) {
schedule, ok := obj.(*portainer.Schedule)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to Schedule object")
return nil, fmt.Errorf("Failed to convert to Schedule object: %s", obj)
}
schedules = append(schedules, *schedule)
return &portainer.Schedule{}, nil
})
return schedules, err
}
// SchedulesByJobType return a array containing all the schedules
// with the specified JobType.
func (service *Service) SchedulesByJobType(jobType portainer.JobType) ([]portainer.Schedule, error) {
var schedules = make([]portainer.Schedule, 0)
err := service.connection.GetAll(
BucketName,
&portainer.Schedule{},
func(obj interface{}) (interface{}, error) {
schedule, ok := obj.(*portainer.Schedule)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to Schedule object")
return nil, fmt.Errorf("Failed to convert to Schedule object: %s", obj)
}
if schedule.JobType == jobType {
schedules = append(schedules, *schedule)
}
return &portainer.Schedule{}, nil
})
return schedules, err
}
// Create assign an ID to a new schedule and saves it.
func (service *Service) CreateSchedule(schedule *portainer.Schedule) error {
return service.connection.CreateObjectWithSetSequence(BucketName, int(schedule.ID), schedule)
}
// GetNextIdentifier returns the next identifier for a schedule.
func (service *Service) GetNextIdentifier() int {
return service.connection.GetNextIdentifier(BucketName)
}

View File

@ -2,7 +2,6 @@ package settings
import (
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/internal"
)
const (
@ -13,12 +12,16 @@ const (
// Service represents a service for managing environment(endpoint) data.
type Service struct {
connection *internal.DbConnection
connection portainer.Connection
}
func (service *Service) BucketName() string {
return BucketName
}
// NewService creates a new instance of a service.
func NewService(connection *internal.DbConnection) (*Service, error) {
err := internal.CreateBucket(connection, BucketName)
func NewService(connection portainer.Connection) (*Service, error) {
err := connection.SetServiceName(BucketName)
if err != nil {
return nil, err
}
@ -32,7 +35,7 @@ func NewService(connection *internal.DbConnection) (*Service, error) {
func (service *Service) Settings() (*portainer.Settings, error) {
var settings portainer.Settings
err := internal.GetObject(service.connection, BucketName, []byte(settingsKey), &settings)
err := service.connection.GetObject(BucketName, []byte(settingsKey), &settings)
if err != nil {
return nil, err
}
@ -42,7 +45,7 @@ func (service *Service) Settings() (*portainer.Settings, error) {
// UpdateSettings persists a Settings object.
func (service *Service) UpdateSettings(settings *portainer.Settings) error {
return internal.UpdateObject(service.connection, BucketName, []byte(settingsKey), settings)
return service.connection.UpdateObject(BucketName, []byte(settingsKey), settings)
}
func (service *Service) IsFeatureFlagEnabled(feature portainer.Feature) bool {

View File

@ -2,7 +2,6 @@ package ssl
import (
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/internal"
)
const (
@ -13,12 +12,16 @@ const (
// Service represents a service for managing ssl data.
type Service struct {
connection *internal.DbConnection
connection portainer.Connection
}
func (service *Service) BucketName() string {
return BucketName
}
// NewService creates a new instance of a service.
func NewService(connection *internal.DbConnection) (*Service, error) {
err := internal.CreateBucket(connection, BucketName)
func NewService(connection portainer.Connection) (*Service, error) {
err := connection.SetServiceName(BucketName)
if err != nil {
return nil, err
}
@ -32,7 +35,7 @@ func NewService(connection *internal.DbConnection) (*Service, error) {
func (service *Service) Settings() (*portainer.SSLSettings, error) {
var settings portainer.SSLSettings
err := internal.GetObject(service.connection, BucketName, []byte(key), &settings)
err := service.connection.GetObject(BucketName, []byte(key), &settings)
if err != nil {
return nil, err
}
@ -42,5 +45,5 @@ func (service *Service) Settings() (*portainer.SSLSettings, error) {
// UpdateSettings persists a SSLSettings object.
func (service *Service) UpdateSettings(settings *portainer.SSLSettings) error {
return internal.UpdateObject(service.connection, BucketName, []byte(key), settings)
return service.connection.UpdateObject(BucketName, []byte(key), settings)
}

View File

@ -0,0 +1,199 @@
package stack
import (
"fmt"
"strings"
"github.com/sirupsen/logrus"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices/errors"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "stacks"
)
// Service represents a service for managing environment(endpoint) data.
type Service struct {
connection portainer.Connection
}
func (service *Service) BucketName() string {
return BucketName
}
// NewService creates a new instance of a service.
func NewService(connection portainer.Connection) (*Service, error) {
err := connection.SetServiceName(BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// Stack returns a stack object by ID.
func (service *Service) Stack(ID portainer.StackID) (*portainer.Stack, error) {
var stack portainer.Stack
identifier := service.connection.ConvertToKey(int(ID))
err := service.connection.GetObject(BucketName, identifier, &stack)
if err != nil {
return nil, err
}
return &stack, nil
}
// StackByName returns a stack object by name.
func (service *Service) StackByName(name string) (*portainer.Stack, error) {
var s *portainer.Stack
stop := fmt.Errorf("ok")
err := service.connection.GetAll(
BucketName,
&portainer.Stack{},
func(obj interface{}) (interface{}, error) {
stack, ok := obj.(*portainer.Stack)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to Stack object")
return nil, fmt.Errorf("Failed to convert to Stack object: %s", obj)
}
if stack.Name == name {
s = stack
return nil, stop
}
return &portainer.Stack{}, nil
})
if err == stop {
return s, nil
}
if err == nil {
return nil, errors.ErrObjectNotFound
}
return nil, err
}
// Stacks returns an array containing all the stacks with same name
func (service *Service) StacksByName(name string) ([]portainer.Stack, error) {
var stacks = make([]portainer.Stack, 0)
err := service.connection.GetAll(
BucketName,
&portainer.Stack{},
func(obj interface{}) (interface{}, error) {
stack, ok := obj.(portainer.Stack)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to Stack object")
return nil, fmt.Errorf("Failed to convert to Stack object: %s", obj)
}
if stack.Name == name {
stacks = append(stacks, stack)
}
return &portainer.Stack{}, nil
})
return stacks, err
}
// Stacks returns an array containing all the stacks.
func (service *Service) Stacks() ([]portainer.Stack, error) {
var stacks = make([]portainer.Stack, 0)
err := service.connection.GetAll(
BucketName,
&portainer.Stack{},
func(obj interface{}) (interface{}, error) {
stack, ok := obj.(*portainer.Stack)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to Stack object")
return nil, fmt.Errorf("Failed to convert to Stack object: %s", obj)
}
stacks = append(stacks, *stack)
return &portainer.Stack{}, nil
})
return stacks, err
}
// GetNextIdentifier returns the next identifier for a stack.
func (service *Service) GetNextIdentifier() int {
return service.connection.GetNextIdentifier(BucketName)
}
// CreateStack creates a new stack.
func (service *Service) Create(stack *portainer.Stack) error {
return service.connection.CreateObjectWithSetSequence(BucketName, int(stack.ID), stack)
}
// UpdateStack updates a stack.
func (service *Service) UpdateStack(ID portainer.StackID, stack *portainer.Stack) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.UpdateObject(BucketName, identifier, stack)
}
// DeleteStack deletes a stack.
func (service *Service) DeleteStack(ID portainer.StackID) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.DeleteObject(BucketName, identifier)
}
// StackByWebhookID returns a pointer to a stack object by webhook ID.
// It returns nil, errors.ErrObjectNotFound if there's no stack associated with the webhook ID.
func (service *Service) StackByWebhookID(id string) (*portainer.Stack, error) {
var s *portainer.Stack
stop := fmt.Errorf("ok")
err := service.connection.GetAll(
BucketName,
&portainer.Stack{},
func(obj interface{}) (interface{}, error) {
var ok bool
s, ok = obj.(*portainer.Stack)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to Stack object")
return &portainer.Stack{}, nil
}
if s.AutoUpdate != nil && strings.EqualFold(s.AutoUpdate.Webhook, id) {
return nil, stop
}
return &portainer.Stack{}, nil
})
if err == stop {
return s, nil
}
if err == nil {
return nil, errors.ErrObjectNotFound
}
return nil, err
}
// RefreshableStacks returns stacks that are configured for a periodic update
func (service *Service) RefreshableStacks() ([]portainer.Stack, error) {
stacks := make([]portainer.Stack, 0)
err := service.connection.GetAll(
BucketName,
&portainer.Stack{},
func(obj interface{}) (interface{}, error) {
stack, ok := obj.(*portainer.Stack)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to Stack object")
return nil, fmt.Errorf("Failed to convert to Stack object: %s", obj)
}
if stack.AutoUpdate != nil && stack.AutoUpdate.Interval != "" {
stacks = append(stacks, *stack)
}
return &portainer.Stack{}, nil
})
return stacks, err
}

View File

@ -4,10 +4,10 @@ import (
"testing"
"time"
"github.com/portainer/portainer/api/datastore"
"github.com/gofrs/uuid"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt"
bolterrors "github.com/portainer/portainer/api/bolt/errors"
"github.com/portainer/portainer/api/filesystem"
"github.com/stretchr/testify/assert"
)
@ -22,14 +22,14 @@ func newGuidString(t *testing.T) string {
type stackBuilder struct {
t *testing.T
count int
store *bolt.Store
store *datastore.Store
}
func TestService_StackByWebhookID(t *testing.T) {
if testing.Short() {
t.Skip("skipping test in short mode. Normally takes ~1s to run.")
}
store, teardown := bolt.MustNewTestStore(true)
_, store, teardown := datastore.MustNewTestStore(true)
defer teardown()
b := stackBuilder{t: t, store: store}
@ -48,7 +48,7 @@ func TestService_StackByWebhookID(t *testing.T) {
// returns nil and object not found error if there's no stack associated with the webhook
got, err = store.StackService.StackByWebhookID(newGuidString(t))
assert.Nil(t, got)
assert.ErrorIs(t, err, bolterrors.ErrObjectNotFound)
assert.True(t, store.IsErrObjectNotFound(err))
}
func (b *stackBuilder) createNewStack(webhookID string) portainer.Stack {
@ -77,7 +77,7 @@ func (b *stackBuilder) createNewStack(webhookID string) portainer.Stack {
stack.AutoUpdate = &portainer.StackAutoUpdate{Webhook: webhookID}
}
err := b.store.StackService.CreateStack(&stack)
err := b.store.StackService.Create(&stack)
assert.NoError(b.t, err)
return stack
@ -87,7 +87,7 @@ func Test_RefreshableStacks(t *testing.T) {
if testing.Short() {
t.Skip("skipping test in short mode. Normally takes ~1s to run.")
}
store, teardown := bolt.MustNewTestStore(true)
_, store, teardown := datastore.MustNewTestStore(true)
defer teardown()
staticStack := portainer.Stack{ID: 1}
@ -95,7 +95,7 @@ func Test_RefreshableStacks(t *testing.T) {
refreshableStack := portainer.Stack{ID: 3, AutoUpdate: &portainer.StackAutoUpdate{Interval: "1m"}}
for _, stack := range []*portainer.Stack{&staticStack, &stackWithWebhook, &refreshableStack} {
err := store.Stack().CreateStack(stack)
err := store.Stack().Create(stack)
assert.NoError(t, err)
}

View File

@ -0,0 +1,90 @@
package tag
import (
"fmt"
portainer "github.com/portainer/portainer/api"
"github.com/sirupsen/logrus"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "tags"
)
// Service represents a service for managing environment(endpoint) data.
type Service struct {
connection portainer.Connection
}
func (service *Service) BucketName() string {
return BucketName
}
// NewService creates a new instance of a service.
func NewService(connection portainer.Connection) (*Service, error) {
err := connection.SetServiceName(BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// Tags return an array containing all the tags.
func (service *Service) Tags() ([]portainer.Tag, error) {
var tags = make([]portainer.Tag, 0)
err := service.connection.GetAll(
BucketName,
&portainer.Tag{},
func(obj interface{}) (interface{}, error) {
tag, ok := obj.(*portainer.Tag)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to Tag object")
return nil, fmt.Errorf("Failed to convert to Tag object: %s", obj)
}
tags = append(tags, *tag)
return &portainer.Tag{}, nil
})
return tags, err
}
// Tag returns a tag by ID.
func (service *Service) Tag(ID portainer.TagID) (*portainer.Tag, error) {
var tag portainer.Tag
identifier := service.connection.ConvertToKey(int(ID))
err := service.connection.GetObject(BucketName, identifier, &tag)
if err != nil {
return nil, err
}
return &tag, nil
}
// CreateTag creates a new tag.
func (service *Service) Create(tag *portainer.Tag) error {
return service.connection.CreateObject(
BucketName,
func(id uint64) (int, interface{}) {
tag.ID = portainer.TagID(id)
return int(tag.ID), tag
},
)
}
// UpdateTag updates a tag.
func (service *Service) UpdateTag(ID portainer.TagID, tag *portainer.Tag) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.UpdateObject(BucketName, identifier, tag)
}
// DeleteTag deletes a tag.
func (service *Service) DeleteTag(ID portainer.TagID) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.DeleteObject(BucketName, identifier)
}

View File

@ -0,0 +1,123 @@
package team
import (
"fmt"
"strings"
"github.com/portainer/portainer/api/dataservices/errors"
"github.com/sirupsen/logrus"
portainer "github.com/portainer/portainer/api"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "teams"
)
// Service represents a service for managing environment(endpoint) data.
type Service struct {
connection portainer.Connection
}
func (service *Service) BucketName() string {
return BucketName
}
// NewService creates a new instance of a service.
func NewService(connection portainer.Connection) (*Service, error) {
err := connection.SetServiceName(BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// Team returns a Team by ID
func (service *Service) Team(ID portainer.TeamID) (*portainer.Team, error) {
var team portainer.Team
identifier := service.connection.ConvertToKey(int(ID))
err := service.connection.GetObject(BucketName, identifier, &team)
if err != nil {
return nil, err
}
return &team, nil
}
// TeamByName returns a team by name.
func (service *Service) TeamByName(name string) (*portainer.Team, error) {
var t *portainer.Team
stop := fmt.Errorf("ok")
err := service.connection.GetAll(
BucketName,
&portainer.Team{},
func(obj interface{}) (interface{}, error) {
team, ok := obj.(*portainer.Team)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to Team object")
return nil, fmt.Errorf("Failed to convert to Team object: %s", obj)
}
if strings.EqualFold(t.Name, name) {
t = team
return nil, stop
}
return &portainer.Team{}, nil
})
if err == stop {
return t, nil
}
if err == nil {
return nil, errors.ErrObjectNotFound
}
return nil, err
}
// Teams return an array containing all the teams.
func (service *Service) Teams() ([]portainer.Team, error) {
var teams = make([]portainer.Team, 0)
err := service.connection.GetAll(
BucketName,
&portainer.Team{},
func(obj interface{}) (interface{}, error) {
team, ok := obj.(*portainer.Team)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to Team object")
return nil, fmt.Errorf("Failed to convert to Team object: %s", obj)
}
teams = append(teams, *team)
return &portainer.Team{}, nil
})
return teams, err
}
// UpdateTeam saves a Team.
func (service *Service) UpdateTeam(ID portainer.TeamID, team *portainer.Team) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.UpdateObject(BucketName, identifier, team)
}
// CreateTeam creates a new Team.
func (service *Service) Create(team *portainer.Team) error {
return service.connection.CreateObject(
BucketName,
func(id uint64) (int, interface{}) {
team.ID = portainer.TeamID(id)
return int(team.ID), team
},
)
}
// DeleteTeam deletes a Team.
func (service *Service) DeleteTeam(ID portainer.TeamID) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.DeleteObject(BucketName, identifier)
}

View File

@ -0,0 +1,170 @@
package teammembership
import (
"fmt"
portainer "github.com/portainer/portainer/api"
"github.com/sirupsen/logrus"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "team_membership"
)
// Service represents a service for managing environment(endpoint) data.
type Service struct {
connection portainer.Connection
}
func (service *Service) BucketName() string {
return BucketName
}
// NewService creates a new instance of a service.
func NewService(connection portainer.Connection) (*Service, error) {
err := connection.SetServiceName(BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// TeamMembership returns a TeamMembership object by ID
func (service *Service) TeamMembership(ID portainer.TeamMembershipID) (*portainer.TeamMembership, error) {
var membership portainer.TeamMembership
identifier := service.connection.ConvertToKey(int(ID))
err := service.connection.GetObject(BucketName, identifier, &membership)
if err != nil {
return nil, err
}
return &membership, nil
}
// TeamMemberships return an array containing all the TeamMembership objects.
func (service *Service) TeamMemberships() ([]portainer.TeamMembership, error) {
var memberships = make([]portainer.TeamMembership, 0)
err := service.connection.GetAll(
BucketName,
&portainer.TeamMembership{},
func(obj interface{}) (interface{}, error) {
membership, ok := obj.(*portainer.TeamMembership)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to TeamMembership object")
return nil, fmt.Errorf("Failed to convert to TeamMembership object: %s", obj)
}
memberships = append(memberships, *membership)
return &portainer.TeamMembership{}, nil
})
return memberships, err
}
// TeamMembershipsByUserID return an array containing all the TeamMembership objects where the specified userID is present.
func (service *Service) TeamMembershipsByUserID(userID portainer.UserID) ([]portainer.TeamMembership, error) {
var memberships = make([]portainer.TeamMembership, 0)
err := service.connection.GetAll(
BucketName,
&portainer.TeamMembership{},
func(obj interface{}) (interface{}, error) {
membership, ok := obj.(*portainer.TeamMembership)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to TeamMembership object")
return nil, fmt.Errorf("Failed to convert to TeamMembership object: %s", obj)
}
if membership.UserID == userID {
memberships = append(memberships, *membership)
}
return &portainer.TeamMembership{}, nil
})
return memberships, err
}
// TeamMembershipsByTeamID return an array containing all the TeamMembership objects where the specified teamID is present.
func (service *Service) TeamMembershipsByTeamID(teamID portainer.TeamID) ([]portainer.TeamMembership, error) {
var memberships = make([]portainer.TeamMembership, 0)
err := service.connection.GetAll(
BucketName,
&portainer.TeamMembership{},
func(obj interface{}) (interface{}, error) {
membership, ok := obj.(*portainer.TeamMembership)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to TeamMembership object")
return nil, fmt.Errorf("Failed to convert to TeamMembership object: %s", obj)
}
if membership.TeamID == teamID {
memberships = append(memberships, *membership)
}
return &portainer.TeamMembership{}, nil
})
return memberships, err
}
// UpdateTeamMembership saves a TeamMembership object.
func (service *Service) UpdateTeamMembership(ID portainer.TeamMembershipID, membership *portainer.TeamMembership) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.UpdateObject(BucketName, identifier, membership)
}
// CreateTeamMembership creates a new TeamMembership object.
func (service *Service) Create(membership *portainer.TeamMembership) error {
return service.connection.CreateObject(
BucketName,
func(id uint64) (int, interface{}) {
membership.ID = portainer.TeamMembershipID(id)
return int(membership.ID), membership
},
)
}
// DeleteTeamMembership deletes a TeamMembership object.
func (service *Service) DeleteTeamMembership(ID portainer.TeamMembershipID) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.DeleteObject(BucketName, identifier)
}
// DeleteTeamMembershipByUserID deletes all the TeamMembership object associated to a UserID.
func (service *Service) DeleteTeamMembershipByUserID(userID portainer.UserID) error {
return service.connection.DeleteAllObjects(
BucketName,
func(obj interface{}) (id int, ok bool) {
membership, ok := obj.(portainer.TeamMembership)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to TeamMembership object")
//return fmt.Errorf("Failed to convert to TeamMembership object: %s", obj)
return -1, false
}
if membership.UserID == userID {
return int(membership.ID), true
}
return -1, false
})
}
// DeleteTeamMembershipByTeamID deletes all the TeamMembership object associated to a TeamID.
func (service *Service) DeleteTeamMembershipByTeamID(teamID portainer.TeamID) error {
return service.connection.DeleteAllObjects(
BucketName,
func(obj interface{}) (id int, ok bool) {
membership, ok := obj.(portainer.TeamMembership)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to TeamMembership object")
//return fmt.Errorf("Failed to convert to TeamMembership object: %s", obj)
return -1, false
}
if membership.TeamID == teamID {
return int(membership.ID), true
}
return -1, false
})
}

View File

@ -2,7 +2,6 @@ package tunnelserver
import (
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/internal"
)
const (
@ -13,12 +12,16 @@ const (
// Service represents a service for managing environment(endpoint) data.
type Service struct {
connection *internal.DbConnection
connection portainer.Connection
}
func (service *Service) BucketName() string {
return BucketName
}
// NewService creates a new instance of a service.
func NewService(connection *internal.DbConnection) (*Service, error) {
err := internal.CreateBucket(connection, BucketName)
func NewService(connection portainer.Connection) (*Service, error) {
err := connection.SetServiceName(BucketName)
if err != nil {
return nil, err
}
@ -32,7 +35,7 @@ func NewService(connection *internal.DbConnection) (*Service, error) {
func (service *Service) Info() (*portainer.TunnelServerInfo, error) {
var info portainer.TunnelServerInfo
err := internal.GetObject(service.connection, BucketName, []byte(infoKey), &info)
err := service.connection.GetObject(BucketName, []byte(infoKey), &info)
if err != nil {
return nil, err
}
@ -42,5 +45,5 @@ func (service *Service) Info() (*portainer.TunnelServerInfo, error) {
// UpdateInfo persists a TunnelServerInfo object.
func (service *Service) UpdateInfo(settings *portainer.TunnelServerInfo) error {
return internal.UpdateObject(service.connection, BucketName, []byte(infoKey), settings)
return service.connection.UpdateObject(BucketName, []byte(infoKey), settings)
}

View File

@ -0,0 +1,147 @@
package user
import (
"fmt"
"strings"
"github.com/portainer/portainer/api/dataservices/errors"
"github.com/sirupsen/logrus"
portainer "github.com/portainer/portainer/api"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "users"
)
// Service represents a service for managing environment(endpoint) data.
type Service struct {
connection portainer.Connection
}
func (service *Service) BucketName() string {
return BucketName
}
// NewService creates a new instance of a service.
func NewService(connection portainer.Connection) (*Service, error) {
err := connection.SetServiceName(BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// User returns a user by ID
func (service *Service) User(ID portainer.UserID) (*portainer.User, error) {
var user portainer.User
identifier := service.connection.ConvertToKey(int(ID))
err := service.connection.GetObject(BucketName, identifier, &user)
if err != nil {
return nil, err
}
return &user, nil
}
// UserByUsername returns a user by username.
func (service *Service) UserByUsername(username string) (*portainer.User, error) {
var u *portainer.User
stop := fmt.Errorf("ok")
err := service.connection.GetAll(
BucketName,
&portainer.User{},
func(obj interface{}) (interface{}, error) {
user, ok := obj.(*portainer.User)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to User object")
return nil, fmt.Errorf("Failed to convert to User object: %s", obj)
}
if strings.EqualFold(user.Username, username) {
u = user
return nil, stop
}
return &portainer.User{}, nil
})
if err == stop {
return u, nil
}
if err == nil {
return nil, errors.ErrObjectNotFound
}
return nil, err
}
// Users return an array containing all the users.
func (service *Service) Users() ([]portainer.User, error) {
var users = make([]portainer.User, 0)
err := service.connection.GetAll(
BucketName,
&portainer.User{},
func(obj interface{}) (interface{}, error) {
user, ok := obj.(*portainer.User)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to User object")
return nil, fmt.Errorf("Failed to convert to User object: %s", obj)
}
users = append(users, *user)
return &portainer.User{}, nil
})
return users, err
}
// UsersByRole return an array containing all the users with the specified role.
func (service *Service) UsersByRole(role portainer.UserRole) ([]portainer.User, error) {
var users = make([]portainer.User, 0)
err := service.connection.GetAll(
BucketName,
&portainer.User{},
func(obj interface{}) (interface{}, error) {
user, ok := obj.(*portainer.User)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to User object")
return nil, fmt.Errorf("Failed to convert to User object: %s", obj)
}
if user.Role == role {
users = append(users, *user)
}
return &portainer.User{}, nil
})
return users, err
}
// UpdateUser saves a user.
func (service *Service) UpdateUser(ID portainer.UserID, user *portainer.User) error {
identifier := service.connection.ConvertToKey(int(ID))
user.Username = strings.ToLower(user.Username)
return service.connection.UpdateObject(BucketName, identifier, user)
}
// CreateUser creates a new user.
func (service *Service) Create(user *portainer.User) error {
return service.connection.CreateObject(
BucketName,
func(id uint64) (int, interface{}) {
user.ID = portainer.UserID(id)
user.Username = strings.ToLower(user.Username)
return int(user.ID), user
},
)
}
// DeleteUser deletes a user.
func (service *Service) DeleteUser(ID portainer.UserID) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.DeleteObject(BucketName, identifier)
}

View File

@ -0,0 +1,91 @@
package version
import (
"strconv"
portainer "github.com/portainer/portainer/api"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "version"
versionKey = "DB_VERSION"
instanceKey = "INSTANCE_ID"
editionKey = "EDITION"
updatingKey = "DB_UPDATING"
)
// Service represents a service to manage stored versions.
type Service struct {
connection portainer.Connection
}
func (service *Service) BucketName() string {
return BucketName
}
// NewService creates a new instance of a service.
func NewService(connection portainer.Connection) (*Service, error) {
err := connection.SetServiceName(BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
// DBVersion retrieves the stored database version.
func (service *Service) DBVersion() (int, error) {
var version string
err := service.connection.GetObject(BucketName, []byte(versionKey), &version)
if err != nil {
return 0, err
}
return strconv.Atoi(version)
}
// Edition retrieves the stored portainer edition.
func (service *Service) Edition() (portainer.SoftwareEdition, error) {
var edition string
err := service.connection.GetObject(BucketName, []byte(editionKey), &edition)
if err != nil {
return 0, err
}
e, err := strconv.Atoi(edition)
if err != nil {
return 0, err
}
return portainer.SoftwareEdition(e), nil
}
// StoreDBVersion store the database version.
func (service *Service) StoreDBVersion(version int) error {
return service.connection.UpdateObject(BucketName, []byte(versionKey), strconv.Itoa(version))
}
// IsUpdating retrieves the database updating status.
func (service *Service) IsUpdating() (bool, error) {
var isUpdating bool
err := service.connection.GetObject(BucketName, []byte(updatingKey), &isUpdating)
return isUpdating, err
}
// StoreIsUpdating store the database updating status.
func (service *Service) StoreIsUpdating(isUpdating bool) error {
return service.connection.UpdateObject(BucketName, []byte(updatingKey), isUpdating)
}
// InstanceID retrieves the stored instance ID.
func (service *Service) InstanceID() (string, error) {
var id string
err := service.connection.GetObject(BucketName, []byte(instanceKey), &id)
return id, err
}
// StoreInstanceID store the instance ID.
func (service *Service) StoreInstanceID(ID string) error {
return service.connection.UpdateObject(BucketName, []byte(instanceKey), ID)
}

View File

@ -0,0 +1,149 @@
package webhook
import (
"fmt"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices/errors"
"github.com/sirupsen/logrus"
)
const (
// BucketName represents the name of the bucket where this service stores data.
BucketName = "webhooks"
)
// Service represents a service for managing webhook data.
type Service struct {
connection portainer.Connection
}
func (service *Service) BucketName() string {
return BucketName
}
// NewService creates a new instance of a service.
func NewService(connection portainer.Connection) (*Service, error) {
err := connection.SetServiceName(BucketName)
if err != nil {
return nil, err
}
return &Service{
connection: connection,
}, nil
}
//Webhooks returns an array of all webhooks
func (service *Service) Webhooks() ([]portainer.Webhook, error) {
var webhooks = make([]portainer.Webhook, 0)
err := service.connection.GetAll(
BucketName,
&portainer.Webhook{},
func(obj interface{}) (interface{}, error) {
webhook, ok := obj.(*portainer.Webhook)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to Webhook object")
return nil, fmt.Errorf("Failed to convert to Webhook object: %s", obj)
}
webhooks = append(webhooks, *webhook)
return &portainer.Webhook{}, nil
})
return webhooks, err
}
// Webhook returns a webhook by ID.
func (service *Service) Webhook(ID portainer.WebhookID) (*portainer.Webhook, error) {
var webhook portainer.Webhook
identifier := service.connection.ConvertToKey(int(ID))
err := service.connection.GetObject(BucketName, identifier, &webhook)
if err != nil {
return nil, err
}
return &webhook, nil
}
// WebhookByResourceID returns a webhook by the ResourceID it is associated with.
func (service *Service) WebhookByResourceID(ID string) (*portainer.Webhook, error) {
var w *portainer.Webhook
stop := fmt.Errorf("ok")
err := service.connection.GetAll(
BucketName,
&portainer.Webhook{},
func(obj interface{}) (interface{}, error) {
webhook, ok := obj.(*portainer.Webhook)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to Webhook object")
return nil, fmt.Errorf("Failed to convert to Webhook object: %s", obj)
}
if webhook.ResourceID == ID {
w = webhook
return nil, stop
}
return &portainer.Webhook{}, nil
})
if err == stop {
return w, nil
}
if err == nil {
return nil, errors.ErrObjectNotFound
}
return nil, err
}
// WebhookByToken returns a webhook by the random token it is associated with.
func (service *Service) WebhookByToken(token string) (*portainer.Webhook, error) {
var w *portainer.Webhook
stop := fmt.Errorf("ok")
err := service.connection.GetAll(
BucketName,
&portainer.Webhook{},
func(obj interface{}) (interface{}, error) {
webhook, ok := obj.(*portainer.Webhook)
if !ok {
logrus.WithField("obj", obj).Errorf("Failed to convert to Webhook object")
return nil, fmt.Errorf("Failed to convert to Webhook object: %s", obj)
}
if webhook.Token == token {
w = webhook
return nil, stop
}
return &portainer.Webhook{}, nil
})
if err == stop {
return w, nil
}
if err == nil {
return nil, errors.ErrObjectNotFound
}
return nil, err
}
// DeleteWebhook deletes a webhook.
func (service *Service) DeleteWebhook(ID portainer.WebhookID) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.DeleteObject(BucketName, identifier)
}
// CreateWebhook assign an ID to a new webhook and saves it.
func (service *Service) Create(webhook *portainer.Webhook) error {
return service.connection.CreateObject(
BucketName,
func(id uint64) (int, interface{}) {
webhook.ID = portainer.WebhookID(id)
return int(webhook.ID), webhook
},
)
}
// UpdateWebhook update a webhook.
func (service *Service) UpdateWebhook(ID portainer.WebhookID, webhook *portainer.Webhook) error {
identifier := service.connection.ConvertToKey(int(ID))
return service.connection.UpdateObject(BucketName, identifier, webhook)
}

View File

@ -1,4 +1,4 @@
package bolt
package datastore
import (
"fmt"
@ -6,20 +6,18 @@ import (
"path"
"time"
plog "github.com/portainer/portainer/api/bolt/log"
plog "github.com/portainer/portainer/api/datastore/log"
)
var backupDefaults = struct {
backupDir string
commonDir string
databaseFileName string
backupDir string
commonDir string
}{
"backups",
"common",
databaseFileName,
}
var backupLog = plog.NewScopedLog("bolt, backup")
var backupLog = plog.NewScopedLog("database, backup")
//
// Backup Helpers
@ -37,11 +35,11 @@ func (store *Store) createBackupFolders() {
}
func (store *Store) databasePath() string {
return path.Join(store.path, databaseFileName)
return path.Join(store.connection.GetStorePath(), store.connection.GetDatabaseFilename())
}
func (store *Store) commonBackupDir() string {
return path.Join(store.path, backupDefaults.backupDir, backupDefaults.commonDir)
return path.Join(store.connection.GetStorePath(), backupDefaults.backupDir, backupDefaults.commonDir)
}
func (store *Store) copyDBFile(from string, to string) error {
@ -55,24 +53,38 @@ func (store *Store) copyDBFile(from string, to string) error {
// BackupOptions provide a helper to inject backup options
type BackupOptions struct {
Version int
Version int // I can't find this used for anything other than a filename
BackupDir string
BackupFileName string
BackupPath string
}
// getBackupRestoreOptions returns options to store db at common backup dir location; used by:
// - db backup prior to version upgrade
// - db rollback
func getBackupRestoreOptions(backupDir string) *BackupOptions {
return &BackupOptions{
BackupDir: backupDir, //connection.commonBackupDir(),
BackupFileName: beforePortainerVersionUpgradeBackup,
}
}
func (store *Store) setupOptions(options *BackupOptions) *BackupOptions {
if options == nil {
options = &BackupOptions{}
}
if options.Version == 0 {
options.Version, _ = store.version()
version, err := store.version()
if err != nil {
version = 0
}
options.Version = version
}
if options.BackupDir == "" {
options.BackupDir = store.commonBackupDir()
}
if options.BackupFileName == "" {
options.BackupFileName = fmt.Sprintf("%s.%s.%s", backupDefaults.databaseFileName, fmt.Sprintf("%03d", options.Version), time.Now().Format("20060102150405"))
options.BackupFileName = fmt.Sprintf("%s.%s.%s", store.connection.GetDatabaseFilename(), fmt.Sprintf("%03d", options.Version), time.Now().Format("20060102150405"))
}
if options.BackupPath == "" {
options.BackupPath = path.Join(options.BackupDir, options.BackupFileName)
@ -81,7 +93,7 @@ func (store *Store) setupOptions(options *BackupOptions) *BackupOptions {
}
// BackupWithOptions backup current database with options
func (store *Store) BackupWithOptions(options *BackupOptions) (string, error) {
func (store *Store) backupWithOptions(options *BackupOptions) (string, error) {
backupLog.Info("creating db backup")
store.createBackupFolders()
@ -94,7 +106,7 @@ func (store *Store) BackupWithOptions(options *BackupOptions) (string, error) {
// Restore strategies:
// - default: restore latest from current edition
// - restore a specific
func (store *Store) RestoreWithOptions(options *BackupOptions) error {
func (store *Store) restoreWithOptions(options *BackupOptions) error {
options = store.setupOptions(options)
// Check if backup file exist before restoring
@ -116,11 +128,12 @@ func (store *Store) RestoreWithOptions(options *BackupOptions) error {
return err
}
return store.Open()
_, err = store.Open()
return err
}
// RemoveWithOptions removes backup database based on supplied options
func (store *Store) RemoveWithOptions(options *BackupOptions) error {
func (store *Store) removeWithOptions(options *BackupOptions) error {
backupLog.Info("Removing db backup")
options = store.setupOptions(options)

View File

@ -1,29 +1,20 @@
package bolt
package datastore
import (
"fmt"
"os"
"path"
"path/filepath"
"testing"
portainer "github.com/portainer/portainer/api"
)
// isFileExist is helper function to check for file existence
func isFileExist(path string) bool {
matches, err := filepath.Glob(path)
if err != nil {
return false
}
return len(matches) > 0
}
func TestCreateBackupFolders(t *testing.T) {
store, teardown := MustNewTestStore(false)
_, store, teardown := MustNewTestStore(false)
defer teardown()
backupPath := path.Join(store.path, backupDefaults.backupDir)
connection := store.GetConnection()
backupPath := path.Join(connection.GetStorePath(), backupDefaults.backupDir)
if isFileExist(backupPath) {
t.Error("Expect backups folder to not exist")
@ -36,38 +27,39 @@ func TestCreateBackupFolders(t *testing.T) {
}
func TestStoreCreation(t *testing.T) {
store, teardown := MustNewTestStore(true)
_, store, teardown := MustNewTestStore(true)
defer teardown()
if store == nil {
t.Error("Expect to create a store")
}
if store.edition() != portainer.PortainerCE {
if store.CheckCurrentEdition() != nil {
t.Error("Expect to get CE Edition")
}
}
func TestBackup(t *testing.T) {
store, teardown := MustNewTestStore(true)
_, store, teardown := MustNewTestStore(true)
connection := store.GetConnection()
defer teardown()
t.Run("Backup should create default db backup", func(t *testing.T) {
store.VersionService.StoreDBVersion(portainer.DBVersion)
store.BackupWithOptions(nil)
store.backupWithOptions(nil)
backupFileName := path.Join(store.path, "backups", "common", fmt.Sprintf("portainer.db.%03d.*", portainer.DBVersion))
backupFileName := path.Join(connection.GetStorePath(), "backups", "common", fmt.Sprintf("portainer.db.%03d.*", portainer.DBVersion))
if !isFileExist(backupFileName) {
t.Errorf("Expect backup file to be created %s", backupFileName)
}
})
t.Run("BackupWithOption should create a name specific backup at common path", func(t *testing.T) {
store.BackupWithOptions(&BackupOptions{
store.backupWithOptions(&BackupOptions{
BackupFileName: beforePortainerVersionUpgradeBackup,
BackupDir: store.commonBackupDir(),
})
backupFileName := path.Join(store.path, "backups", "common", beforePortainerVersionUpgradeBackup)
backupFileName := path.Join(connection.GetStorePath(), "backups", "common", beforePortainerVersionUpgradeBackup)
if !isFileExist(backupFileName) {
t.Errorf("Expect backup file to be created %s", backupFileName)
}
@ -75,7 +67,7 @@ func TestBackup(t *testing.T) {
}
func TestRemoveWithOptions(t *testing.T) {
store, teardown := MustNewTestStore(true)
_, store, teardown := MustNewTestStore(true)
defer teardown()
t.Run("successfully removes file if existent", func(t *testing.T) {
@ -92,7 +84,7 @@ func TestRemoveWithOptions(t *testing.T) {
}
f.Close()
err = store.RemoveWithOptions(options)
err = store.removeWithOptions(options)
if err != nil {
t.Errorf("RemoveWithOptions should successfully remove file; err=%w", err)
}
@ -108,7 +100,7 @@ func TestRemoveWithOptions(t *testing.T) {
BackupFileName: "test.txt",
}
err := store.RemoveWithOptions(options)
err := store.removeWithOptions(options)
if err == nil {
t.Error("RemoveWithOptions should fail for non-existent file")
}

View File

@ -0,0 +1,80 @@
package datastore
import (
"io"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/dataservices/errors"
)
func (store *Store) version() (int, error) {
version, err := store.VersionService.DBVersion()
if store.IsErrObjectNotFound(err) {
version = 0
}
return version, err
}
func (store *Store) edition() portainer.SoftwareEdition {
edition, err := store.VersionService.Edition()
if store.IsErrObjectNotFound(err) {
edition = portainer.PortainerCE
}
return edition
}
// NewStore initializes a new Store and the associated services
func NewStore(storePath string, fileService portainer.FileService, connection portainer.Connection) *Store {
return &Store{
fileService: fileService,
connection: connection,
}
}
// Open opens and initializes the BoltDB database.
func (store *Store) Open() (newStore bool, err error) {
newStore = true
err = store.connection.Open()
if err != nil {
return newStore, err
}
err = store.initServices()
if err != nil {
return newStore, err
}
// if we have DBVersion in the database then ensure we flag this as NOT a new store
if _, err := store.VersionService.DBVersion(); err == nil {
newStore = false
}
return newStore, nil
}
func (store *Store) Close() error {
return store.connection.Close()
}
// BackupTo backs up db to a provided writer.
// It does hot backup and doesn't block other database reads and writes
func (store *Store) BackupTo(w io.Writer) error {
return store.connection.BackupTo(w)
}
// CheckCurrentEdition checks if current edition is community edition
func (store *Store) CheckCurrentEdition() error {
if store.edition() != portainer.PortainerCE {
return errors.ErrWrongDBEdition
}
return nil
}
// TODO: move the use of this to dataservices.IsErrObjectNotFound()?
func (store *Store) IsErrObjectNotFound(e error) bool {
return e == errors.ErrObjectNotFound
}
func (store *Store) Rollback(force bool) error {
return store.connectionRollback(force)
}

View File

@ -0,0 +1,418 @@
package datastore
import (
"fmt"
"runtime"
"strings"
"testing"
"github.com/dchest/uniuri"
"github.com/pkg/errors"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/chisel"
"github.com/portainer/portainer/api/crypto"
"github.com/stretchr/testify/assert"
)
const (
adminUsername = "admin"
adminPassword = "password"
standardUsername = "standard"
standardPassword = "password"
agentOnDockerEnvironmentUrl = "tcp://192.168.167.207:30775"
edgeAgentOnKubernetesEnvironmentUrl = "tcp://192.168.167.207"
kubernetesLocalEnvironmentUrl = "https://kubernetes.default.svc"
)
// TestStoreFull an eventually comprehensive set of tests for the Store.
// The idea is what we write to the store, we should read back.
func TestStoreFull(t *testing.T) {
_, store, teardown := MustNewTestStore(true)
defer teardown()
testCases := map[string]func(t *testing.T){
"User Accounts": func(t *testing.T) {
store.testUserAccounts(t)
},
"Environments": func(t *testing.T) {
store.testEnvironments(t)
},
"Settings": func(t *testing.T) {
store.testSettings(t)
},
"SSL Settings": func(t *testing.T) {
store.testSSLSettings(t)
},
"Tunnel Server": func(t *testing.T) {
store.testTunnelServer(t)
},
"Custom Templates": func(t *testing.T) {
store.testCustomTemplates(t)
},
"Registries": func(t *testing.T) {
store.testRegistries(t)
},
"Resource Control": func(t *testing.T) {
store.testResourceControl(t)
},
"Schedules": func(t *testing.T) {
store.testSchedules(t)
},
"Tags": func(t *testing.T) {
store.testTags(t)
},
// "Test Title": func(t *testing.T) {
// },
}
for name, test := range testCases {
t.Run(name, test)
}
}
func (store *Store) testEnvironments(t *testing.T) {
id := store.CreateEndpoint(t, "local", portainer.KubernetesLocalEnvironment, "", true)
store.CreateEndpointRelation(id)
id = store.CreateEndpoint(t, "agent", portainer.AgentOnDockerEnvironment, agentOnDockerEnvironmentUrl, true)
store.CreateEndpointRelation(id)
id = store.CreateEndpoint(t, "edge", portainer.EdgeAgentOnKubernetesEnvironment, edgeAgentOnKubernetesEnvironmentUrl, true)
store.CreateEndpointRelation(id)
}
func newEndpoint(endpointType portainer.EndpointType, id portainer.EndpointID, name, URL string, TLS bool) *portainer.Endpoint {
endpoint := &portainer.Endpoint{
ID: id,
Name: name,
URL: URL,
Type: endpointType,
GroupID: portainer.EndpointGroupID(1),
PublicURL: "",
TLSConfig: portainer.TLSConfiguration{
TLS: false,
},
UserAccessPolicies: portainer.UserAccessPolicies{},
TeamAccessPolicies: portainer.TeamAccessPolicies{},
Extensions: []portainer.EndpointExtension{},
TagIDs: []portainer.TagID{},
Status: portainer.EndpointStatusUp,
Snapshots: []portainer.DockerSnapshot{},
Kubernetes: portainer.KubernetesDefault(),
}
if TLS {
endpoint.TLSConfig = portainer.TLSConfiguration{
TLS: true,
TLSSkipVerify: true,
}
}
return endpoint
}
func setEndpointAuthorizations(endpoint *portainer.Endpoint) {
endpoint.SecuritySettings = portainer.EndpointSecuritySettings{
AllowVolumeBrowserForRegularUsers: false,
EnableHostManagementFeatures: false,
AllowSysctlSettingForRegularUsers: true,
AllowBindMountsForRegularUsers: true,
AllowPrivilegedModeForRegularUsers: true,
AllowHostNamespaceForRegularUsers: true,
AllowContainerCapabilitiesForRegularUsers: true,
AllowDeviceMappingForRegularUsers: true,
AllowStackManagementForRegularUsers: true,
}
}
func (store *Store) CreateEndpoint(t *testing.T, name string, endpointType portainer.EndpointType, URL string, tls bool) portainer.EndpointID {
is := assert.New(t)
var expectedEndpoint *portainer.Endpoint
id := portainer.EndpointID(store.Endpoint().GetNextIdentifier())
switch endpointType {
case portainer.DockerEnvironment:
if URL == "" {
URL = "unix:///var/run/docker.sock"
if runtime.GOOS == "windows" {
URL = "npipe:////./pipe/docker_engine"
}
}
expectedEndpoint = newEndpoint(endpointType, id, name, URL, tls)
case portainer.AgentOnDockerEnvironment:
expectedEndpoint = newEndpoint(endpointType, id, name, URL, tls)
case portainer.AgentOnKubernetesEnvironment:
URL = strings.TrimPrefix(URL, "tcp://")
expectedEndpoint = newEndpoint(endpointType, id, name, URL, tls)
case portainer.EdgeAgentOnKubernetesEnvironment:
cs := chisel.NewService(store, nil)
expectedEndpoint = newEndpoint(endpointType, id, name, URL, tls)
edgeKey := cs.GenerateEdgeKey(URL, "", int(id))
expectedEndpoint.EdgeKey = edgeKey
store.testTunnelServer(t)
case portainer.KubernetesLocalEnvironment:
if URL == "" {
URL = kubernetesLocalEnvironmentUrl
}
expectedEndpoint = newEndpoint(endpointType, id, name, URL, tls)
}
setEndpointAuthorizations(expectedEndpoint)
store.Endpoint().Create(expectedEndpoint)
endpoint, err := store.Endpoint().Endpoint(id)
is.NoError(err, "Endpoint() should not return an error")
is.Equal(expectedEndpoint, endpoint, "endpoint should be the same")
return endpoint.ID
}
func (store *Store) CreateEndpointRelation(id portainer.EndpointID) {
relation := &portainer.EndpointRelation{
EndpointID: id,
EdgeStacks: map[portainer.EdgeStackID]bool{},
}
store.EndpointRelation().Create(relation)
}
func (store *Store) testSSLSettings(t *testing.T) {
is := assert.New(t)
ssl := &portainer.SSLSettings{
CertPath: "/data/certs/cert.pem",
HTTPEnabled: true,
KeyPath: "/data/certs/key.pem",
SelfSigned: true,
}
store.SSLSettings().UpdateSettings(ssl)
settings, err := store.SSLSettings().Settings()
is.NoError(err, "Get sslsettings should succeed")
is.Equal(ssl, settings, "Stored SSLSettings should be the same as what is read out")
}
func (store *Store) testTunnelServer(t *testing.T) {
is := assert.New(t)
expectPrivateKeySeed := uniuri.NewLen(16)
err := store.TunnelServer().UpdateInfo(&portainer.TunnelServerInfo{PrivateKeySeed: expectPrivateKeySeed})
is.NoError(err, "UpdateInfo should have succeeded")
serverInfo, err := store.TunnelServer().Info()
is.NoError(err, "Info should have succeeded")
is.Equal(expectPrivateKeySeed, serverInfo.PrivateKeySeed, "hashed passwords should not differ")
}
// add users, read them back and check the details are unchanged
func (store *Store) testUserAccounts(t *testing.T) {
is := assert.New(t)
err := store.createAccount(adminUsername, adminPassword, portainer.AdministratorRole)
is.NoError(err, "CreateAccount should succeed")
store.checkAccount(adminUsername, adminPassword, portainer.AdministratorRole)
is.NoError(err, "Account failure")
err = store.createAccount(standardUsername, standardPassword, portainer.StandardUserRole)
is.NoError(err, "CreateAccount should succeed")
store.checkAccount(standardUsername, standardPassword, portainer.StandardUserRole)
is.NoError(err, "Account failure")
}
// create an account with the provided details
func (store *Store) createAccount(username, password string, role portainer.UserRole) error {
var err error
user := &portainer.User{Username: username, Role: role}
// encrypt the password
cs := &crypto.Service{}
user.Password, err = cs.Hash(password)
if err != nil {
return err
}
err = store.User().Create(user)
if err != nil {
return err
}
return nil
}
func (store *Store) checkAccount(username, expectPassword string, expectRole portainer.UserRole) error {
// Read the account for username. Check password and role is what we expect
user, err := store.User().UserByUsername(username)
if err != nil {
return errors.Wrap(err, "failed to find user")
}
if user.Username != username || user.Role != expectRole {
return fmt.Errorf("%s user details do not match", user.Username)
}
// Check the password
cs := &crypto.Service{}
expectPasswordHash, err := cs.Hash(expectPassword)
if err != nil {
return errors.Wrap(err, "hash failed")
}
if user.Password != expectPasswordHash {
return fmt.Errorf("%s user password hash failure", user.Username)
}
return nil
}
func (store *Store) testSettings(t *testing.T) {
is := assert.New(t)
// since many settings are default and basically nil, I'm going to update some and read them back
expectedSettings, err := store.Settings().Settings()
is.NoError(err, "Settings() should not return an error")
expectedSettings.TemplatesURL = "http://portainer.io/application-templates"
expectedSettings.HelmRepositoryURL = "http://portainer.io/helm-repository"
expectedSettings.EdgeAgentCheckinInterval = 60
expectedSettings.AuthenticationMethod = portainer.AuthenticationLDAP
expectedSettings.LDAPSettings = portainer.LDAPSettings{
AnonymousMode: true,
StartTLS: true,
AutoCreateUsers: true,
Password: "random",
}
expectedSettings.SnapshotInterval = "10m"
err = store.Settings().UpdateSettings(expectedSettings)
is.NoError(err, "UpdateSettings() should succeed")
settings, err := store.Settings().Settings()
is.NoError(err, "Settings() should not return an error")
is.Equal(expectedSettings, settings, "stored settings should match")
}
func (store *Store) testCustomTemplates(t *testing.T) {
is := assert.New(t)
customTemplate := store.CustomTemplate()
is.NotNil(customTemplate, "customTemplate Service shouldn't be nil")
expectedTemplate := &portainer.CustomTemplate{
ID: portainer.CustomTemplateID(customTemplate.GetNextIdentifier()),
Title: "Custom Title",
Description: "Custom Template Description",
ProjectPath: "/data/custom_template/1",
Note: "A note about this custom template",
EntryPoint: "docker-compose.yaml",
CreatedByUserID: 10,
}
customTemplate.Create(expectedTemplate)
actualTemplate, err := customTemplate.CustomTemplate(expectedTemplate.ID)
is.NoError(err, "CustomTemplate should not return an error")
is.Equal(expectedTemplate, actualTemplate, "expected and actual template do not match")
}
func (store *Store) testRegistries(t *testing.T) {
is := assert.New(t)
regService := store.RegistryService
is.NotNil(regService, "RegistryService shouldn't be nil")
reg1 := &portainer.Registry{
ID: 1,
Type: portainer.DockerHubRegistry,
Name: "Dockerhub Registry Test",
}
reg2 := &portainer.Registry{
ID: 2,
Type: portainer.GitlabRegistry,
Name: "Gitlab Registry Test",
Gitlab: portainer.GitlabRegistryData{
ProjectID: 12345,
InstanceURL: "http://gitlab.com/12345",
ProjectPath: "mytestproject",
},
}
err := regService.Create(reg1)
is.NoError(err)
err = regService.Create(reg2)
is.NoError(err)
actualReg1, err := regService.Registry(reg1.ID)
is.NoError(err)
is.Equal(reg1, actualReg1, "registries differ")
actualReg2, err := regService.Registry(reg2.ID)
is.NoError(err)
is.Equal(reg2, actualReg2, "registries differ")
}
func (store *Store) testResourceControl(t *testing.T) {
// is := assert.New(t)
// resControl := store.ResourceControl()
// ctrl := &portainer.ResourceControl{
// }
// resControl().Create()
}
func (store *Store) testSchedules(t *testing.T) {
is := assert.New(t)
schedule := store.ScheduleService
s := &portainer.Schedule{
ID: portainer.ScheduleID(schedule.GetNextIdentifier()),
Name: "My Custom Schedule 1",
CronExpression: "*/5 * * * * portainer /bin/sh -c echo 'hello world'",
}
err := schedule.CreateSchedule(s)
is.NoError(err, "CreateSchedule should succeed")
actual, err := schedule.Schedule(s.ID)
is.NoError(err, "schedule should be found")
is.Equal(s, actual, "schedules differ")
}
func (store *Store) testTags(t *testing.T) {
is := assert.New(t)
tags := store.TagService
tag1 := &portainer.Tag{
ID: 1,
Name: "Tag 1",
}
tag2 := &portainer.Tag{
ID: 2,
Name: "Tag 1",
}
err := tags.Create(tag1)
is.NoError(err, "Tags.Create should succeed")
err = tags.Create(tag2)
is.NoError(err, "Tags.Create should succeed")
actual, err := tags.Tag(tag1.ID)
is.NoError(err, "tag1 should be found")
is.Equal(tag1, actual, "tags differ")
actual, err = tags.Tag(tag2.ID)
is.NoError(err, "tag2 should be found")
is.Equal(tag2, actual, "tags differ")
}

View File

@ -1,15 +1,14 @@
package bolt
package datastore
import (
"github.com/gofrs/uuid"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/errors"
)
// Init creates the default data set.
func (store *Store) Init() error {
instanceID, err := store.VersionService.InstanceID()
if err == errors.ErrObjectNotFound {
if store.IsErrObjectNotFound(err) {
uid, err := uuid.NewV4()
if err != nil {
return err
@ -24,8 +23,9 @@ func (store *Store) Init() error {
return err
}
_, err = store.SettingsService.Settings()
if err == errors.ErrObjectNotFound {
// TODO: these need to also be applied when importing
settings, err := store.SettingsService.Settings()
if store.IsErrObjectNotFound(err) {
defaultSettings := &portainer.Settings{
AuthenticationMethod: portainer.AuthenticationInternal,
BlackListedLabels: make([]portainer.Pair, 0),
@ -56,11 +56,16 @@ func (store *Store) Init() error {
}
} else if err != nil {
return err
} else if err == nil {
if settings.UserSessionTimeout == "" {
settings.UserSessionTimeout = portainer.DefaultUserSessionTimeout
store.Settings().UpdateSettings(settings)
}
}
_, err = store.SSLSettings().Settings()
if err != nil {
if err != errors.ErrObjectNotFound {
if !store.IsErrObjectNotFound(err) {
return err
}
@ -89,7 +94,7 @@ func (store *Store) Init() error {
TagIDs: []portainer.TagID{},
}
err = store.EndpointGroupService.CreateEndpointGroup(unassignedGroup)
err = store.EndpointGroupService.Create(unassignedGroup)
if err != nil {
return err
}

View File

@ -1,22 +1,52 @@
package bolt
package datastore
import (
"fmt"
"runtime/debug"
"github.com/portainer/portainer/api/cli"
"github.com/portainer/portainer/api/dataservices/errors"
plog "github.com/portainer/portainer/api/datastore/log"
"github.com/portainer/portainer/api/datastore/migrator"
"github.com/portainer/portainer/api/internal/authorization"
werrors "github.com/pkg/errors"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/errors"
plog "github.com/portainer/portainer/api/bolt/log"
"github.com/portainer/portainer/api/bolt/migrator"
"github.com/portainer/portainer/api/internal/authorization"
)
const beforePortainerVersionUpgradeBackup = "portainer.db.bak"
var migrateLog = plog.NewScopedLog("bolt, migrate")
var migrateLog = plog.NewScopedLog("database, migrate")
func (store *Store) MigrateData() error {
version, err := store.version()
if err != nil {
return err
}
migratorParams := &migrator.MigratorParameters{
DatabaseVersion: version,
EndpointGroupService: store.EndpointGroupService,
EndpointService: store.EndpointService,
EndpointRelationService: store.EndpointRelationService,
ExtensionService: store.ExtensionService,
RegistryService: store.RegistryService,
ResourceControlService: store.ResourceControlService,
RoleService: store.RoleService,
ScheduleService: store.ScheduleService,
SettingsService: store.SettingsService,
StackService: store.StackService,
TagService: store.TagService,
TeamMembershipService: store.TeamMembershipService,
UserService: store.UserService,
VersionService: store.VersionService,
FileService: store.fileService,
DockerhubService: store.DockerHubService,
AuthorizationService: authorization.NewService(store),
}
return store.connectionMigrateData(migratorParams)
}
// FailSafeMigrate backup and restore DB if migration fail
func (store *Store) FailSafeMigrate(migrator *migrator.Migrator) (err error) {
@ -36,18 +66,11 @@ func (store *Store) FailSafeMigrate(migrator *migrator.Migrator) (err error) {
// MigrateData automatically migrate the data based on the DBVersion.
// This process is only triggered on an existing database, not if the database was just created.
// if force is true, then migrate regardless.
func (store *Store) MigrateData(force bool) error {
if store.isNew && !force {
return store.VersionService.StoreDBVersion(portainer.DBVersion)
}
migrator, err := store.newMigrator()
if err != nil {
return err
}
func (store *Store) connectionMigrateData(migratorParams *migrator.MigratorParameters) error {
migrator := migrator.NewMigrator(migratorParams)
// backup db file before upgrading DB to support rollback
isUpdating, err := store.VersionService.IsUpdating()
isUpdating, err := migratorParams.VersionService.IsUpdating()
if err != nil && err != errors.ErrObjectNotFound {
return err
}
@ -71,56 +94,16 @@ func (store *Store) MigrateData(force bool) error {
return nil
}
func (store *Store) newMigrator() (*migrator.Migrator, error) {
version, err := store.version()
if err != nil {
return nil, err
}
migratorParams := &migrator.Parameters{
DB: store.connection.DB,
DatabaseVersion: version,
EndpointGroupService: store.EndpointGroupService,
EndpointService: store.EndpointService,
EndpointRelationService: store.EndpointRelationService,
ExtensionService: store.ExtensionService,
RegistryService: store.RegistryService,
ResourceControlService: store.ResourceControlService,
RoleService: store.RoleService,
ScheduleService: store.ScheduleService,
SettingsService: store.SettingsService,
StackService: store.StackService,
TagService: store.TagService,
TeamMembershipService: store.TeamMembershipService,
UserService: store.UserService,
VersionService: store.VersionService,
FileService: store.fileService,
DockerhubService: store.DockerHubService,
AuthorizationService: authorization.NewService(store),
}
return migrator.NewMigrator(migratorParams), nil
}
// getBackupRestoreOptions returns options to store db at common backup dir location; used by:
// - db backup prior to version upgrade
// - db rollback
func getBackupRestoreOptions(store *Store) *BackupOptions {
return &BackupOptions{
BackupDir: store.commonBackupDir(),
BackupFileName: beforePortainerVersionUpgradeBackup,
}
}
// backupVersion will backup the database or panic if any errors occur
func (store *Store) backupVersion(migrator *migrator.Migrator) error {
migrateLog.Info("Backing up database prior to version upgrade...")
options := getBackupRestoreOptions(store)
options := getBackupRestoreOptions(store.commonBackupDir())
_, err := store.BackupWithOptions(options)
_, err := store.backupWithOptions(options)
if err != nil {
migrateLog.Error("An error occurred during database backup", err)
removalErr := store.RemoveWithOptions(options)
removalErr := store.removeWithOptions(options)
if removalErr != nil {
migrateLog.Error("An error occurred during store removal prior to backup", err)
}
@ -131,7 +114,7 @@ func (store *Store) backupVersion(migrator *migrator.Migrator) error {
}
// Rollback to a pre-upgrade backup copy/snapshot of portainer.db
func (store *Store) Rollback(force bool) error {
func (store *Store) connectionRollback(force bool) error {
if !force {
confirmed, err := cli.Confirm("Are you sure you want to rollback your database to the previous backup?")
@ -140,12 +123,12 @@ func (store *Store) Rollback(force bool) error {
}
}
options := getBackupRestoreOptions(store)
options := getBackupRestoreOptions(store.commonBackupDir())
err := store.RestoreWithOptions(options)
err := store.restoreWithOptions(options)
if err != nil {
return err
}
return store.Close()
return store.connection.Close()
}

View File

@ -1,8 +1,9 @@
package bolt
package datastore
import (
"fmt"
"log"
"path/filepath"
"strings"
"testing"
@ -11,27 +12,32 @@ import (
// testVersion is a helper which tests current store version against wanted version
func testVersion(store *Store, versionWant int, t *testing.T) {
if v, _ := store.version(); v != versionWant {
v, err := store.VersionService.DBVersion()
if err != nil {
t.Errorf("Expect store version to be %d but was %d with error: %s", versionWant, v, err)
}
if v != versionWant {
t.Errorf("Expect store version to be %d but was %d", versionWant, v)
}
}
func TestMigrateData(t *testing.T) {
t.Run("MigrateData for New Store & Re-Open Check", func(t *testing.T) {
store, teardown := MustNewTestStore(false)
newStore, store, teardown := MustNewTestStore(false)
defer teardown()
if !store.IsNew() {
if !newStore {
t.Error("Expect a new DB")
}
store.MigrateData(false)
// not called for new stores
//store.MigrateData()
testVersion(store, portainer.DBVersion, t)
store.Close()
store.Open()
if store.IsNew() {
newStore, _ = store.Open()
if newStore {
t.Error("Expect store to NOT be new DB")
}
})
@ -40,24 +46,24 @@ func TestMigrateData(t *testing.T) {
version int
expectedVersion int
}{
{version: 2, expectedVersion: portainer.DBVersion},
{version: 17, expectedVersion: portainer.DBVersion},
{version: 21, expectedVersion: portainer.DBVersion},
}
for _, tc := range tests {
store, teardown := MustNewTestStore(true)
_, store, teardown := MustNewTestStore(true)
defer teardown()
// Setup data
store.VersionService.StoreDBVersion(tc.version)
// Required roles by migrations 22.2
store.RoleService.CreateRole(&portainer.Role{ID: 1})
store.RoleService.CreateRole(&portainer.Role{ID: 2})
store.RoleService.CreateRole(&portainer.Role{ID: 3})
store.RoleService.CreateRole(&portainer.Role{ID: 4})
store.RoleService.Create(&portainer.Role{ID: 1})
store.RoleService.Create(&portainer.Role{ID: 2})
store.RoleService.Create(&portainer.Role{ID: 3})
store.RoleService.Create(&portainer.Role{ID: 4})
t.Run(fmt.Sprintf("MigrateData for version %d", tc.version), func(t *testing.T) {
store.MigrateData(true)
store.MigrateData()
testVersion(store, tc.expectedVersion, t)
})
@ -69,25 +75,25 @@ func TestMigrateData(t *testing.T) {
}
t.Run("Error in MigrateData should restore backup before MigrateData", func(t *testing.T) {
store, teardown := MustNewTestStore(false)
_, store, teardown := MustNewTestStore(false)
defer teardown()
version := 2
version := 17
store.VersionService.StoreDBVersion(version)
store.MigrateData(true)
store.MigrateData()
testVersion(store, version, t)
})
t.Run("MigrateData should create backup file upon update", func(t *testing.T) {
store, teardown := MustNewTestStore(false)
_, store, teardown := MustNewTestStore(false)
defer teardown()
store.VersionService.StoreDBVersion(0)
store.MigrateData(true)
store.MigrateData()
options := store.setupOptions(getBackupRestoreOptions(store))
options := store.setupOptions(getBackupRestoreOptions(store.commonBackupDir()))
if !isFileExist(options.BackupPath) {
t.Errorf("Backup file should exist; file=%s", options.BackupPath)
@ -95,14 +101,14 @@ func TestMigrateData(t *testing.T) {
})
t.Run("MigrateData should fail to create backup if database file is set to updating", func(t *testing.T) {
store, teardown := MustNewTestStore(false)
_, store, teardown := MustNewTestStore(false)
defer teardown()
store.VersionService.StoreIsUpdating(true)
store.MigrateData(true)
store.MigrateData()
options := store.setupOptions(getBackupRestoreOptions(store))
options := store.setupOptions(getBackupRestoreOptions(store.commonBackupDir()))
if isFileExist(options.BackupPath) {
t.Errorf("Backup file should not exist for dirty database; file=%s", options.BackupPath)
@ -110,12 +116,12 @@ func TestMigrateData(t *testing.T) {
})
t.Run("MigrateData should not create backup on startup if portainer version matches db", func(t *testing.T) {
store, teardown := MustNewTestStore(false)
_, store, teardown := MustNewTestStore(false)
defer teardown()
store.MigrateData(true)
store.MigrateData()
options := store.setupOptions(getBackupRestoreOptions(store))
options := store.setupOptions(getBackupRestoreOptions(store.commonBackupDir()))
if isFileExist(options.BackupPath) {
t.Errorf("Backup file should not exist for dirty database; file=%s", options.BackupPath)
@ -125,10 +131,10 @@ func TestMigrateData(t *testing.T) {
}
func Test_getBackupRestoreOptions(t *testing.T) {
store, teardown := MustNewTestStore(false)
_, store, teardown := MustNewTestStore(false)
defer teardown()
options := getBackupRestoreOptions(store)
options := getBackupRestoreOptions(store.commonBackupDir())
wantDir := store.commonBackupDir()
if !strings.HasSuffix(options.BackupDir, wantDir) {
@ -144,11 +150,11 @@ func Test_getBackupRestoreOptions(t *testing.T) {
func TestRollback(t *testing.T) {
t.Run("Rollback should restore upgrade after backup", func(t *testing.T) {
version := 21
store, teardown := MustNewTestStore(false)
_, store, teardown := MustNewTestStore(false)
defer teardown()
store.VersionService.StoreDBVersion(version)
_, err := store.BackupWithOptions(getBackupRestoreOptions(store))
_, err := store.backupWithOptions(getBackupRestoreOptions(store.commonBackupDir()))
if err != nil {
log.Fatal(err)
}
@ -170,3 +176,12 @@ func TestRollback(t *testing.T) {
testVersion(store, version, t)
})
}
// isFileExist is helper function to check for file existence
func isFileExist(path string) bool {
matches, err := filepath.Glob(path)
if err != nil {
return false
}
return len(matches) > 0
}

View File

@ -0,0 +1,91 @@
package datastore
import (
"testing"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/datastore/migrator"
"github.com/portainer/portainer/api/internal/authorization"
)
const dummyLogoURL = "example.com"
// initTestingDBConn creates a settings service with raw database DB connection
// for unit testing usage only since using NewStore will cause cycle import inside migrator pkg
func initTestingSettingsService(dbConn portainer.Connection, preSetObj map[string]interface{}) error {
//insert a obj
if err := dbConn.UpdateObject("settings", []byte("SETTINGS"), preSetObj); err != nil {
return err
}
return nil
}
func setup(store *Store) error {
var err error
dummySettingsObj := map[string]interface{}{
"LogoURL": dummyLogoURL,
}
err = initTestingSettingsService(store.connection, dummySettingsObj)
if err != nil {
return err
}
return nil
}
func TestMigrateSettings(t *testing.T) {
_, store, teardown := MustNewTestStore(false)
defer teardown()
err := setup(store)
if err != nil {
t.Errorf("failed to complete testing setups, err: %v", err)
}
updatedSettings, err := store.SettingsService.Settings()
// SO -basically, this test _isn't_ testing migration, its testing golang type defaults.
if updatedSettings.LogoURL != dummyLogoURL { // ensure a pre-migrate setting isn't unset
t.Errorf("unexpected value changes in the updated settings, want LogoURL value: %s, got LogoURL value: %s", dummyLogoURL, updatedSettings.LogoURL)
}
if updatedSettings.OAuthSettings.SSO != false { // I recon golang defaulting will make this false
t.Errorf("unexpected default OAuth SSO setting, want: false, got: %t", updatedSettings.OAuthSettings.SSO)
}
if updatedSettings.OAuthSettings.LogoutURI != "" {
t.Errorf("unexpected default OAuth HideInternalAuth setting, want:, got: %s", updatedSettings.OAuthSettings.LogoutURI)
}
m := migrator.NewMigrator(&migrator.MigratorParameters{
DatabaseVersion: 29,
EndpointGroupService: store.EndpointGroupService,
EndpointService: store.EndpointService,
EndpointRelationService: store.EndpointRelationService,
ExtensionService: store.ExtensionService,
RegistryService: store.RegistryService,
ResourceControlService: store.ResourceControlService,
RoleService: store.RoleService,
ScheduleService: store.ScheduleService,
SettingsService: store.SettingsService,
StackService: store.StackService,
TagService: store.TagService,
TeamMembershipService: store.TeamMembershipService,
UserService: store.UserService,
VersionService: store.VersionService,
FileService: store.fileService,
DockerhubService: store.DockerHubService,
AuthorizationService: authorization.NewService(store),
})
if err := m.MigrateSettingsToDB30(); err != nil {
t.Errorf("failed to update settings: %v", err)
}
if err != nil {
t.Errorf("failed to retrieve the updated settings: %v", err)
}
if updatedSettings.LogoURL != dummyLogoURL {
t.Errorf("unexpected value changes in the updated settings, want LogoURL value: %s, got LogoURL value: %s", dummyLogoURL, updatedSettings.LogoURL)
}
if updatedSettings.OAuthSettings.SSO != false {
t.Errorf("unexpected default OAuth SSO setting, want: false, got: %t", updatedSettings.OAuthSettings.SSO)
}
if updatedSettings.OAuthSettings.LogoutURI != "" {
t.Errorf("unexpected default OAuth HideInternalAuth setting, want:, got: %s", updatedSettings.OAuthSettings.LogoutURI)
}
}

View File

@ -1,25 +1,19 @@
package migrator
package datastore
import (
"path"
"testing"
"time"
"github.com/boltdb/bolt"
portainer "github.com/portainer/portainer/api"
"github.com/portainer/portainer/api/bolt/internal"
"github.com/portainer/portainer/api/bolt/stack"
"github.com/portainer/portainer/api/datastore/migrator"
gittypes "github.com/portainer/portainer/api/git/types"
"github.com/stretchr/testify/assert"
)
func TestMigrateStackEntryPoint(t *testing.T) {
dbConn, err := bolt.Open(path.Join(t.TempDir(), "portainer-ee-mig-34.db"), 0600, &bolt.Options{Timeout: 1 * time.Second})
assert.NoError(t, err, "failed to init testing DB connection")
defer dbConn.Close()
_, store, teardown := MustNewTestStore(false)
defer teardown()
stackService, err := stack.NewService(&internal.DbConnection{DB: dbConn})
assert.NoError(t, err, "failed to init testing Stack service")
stackService := store.Stack()
stacks := []*portainer.Stack{
{
@ -34,17 +28,25 @@ func TestMigrateStackEntryPoint(t *testing.T) {
}
for _, s := range stacks {
err := stackService.CreateStack(s)
err := stackService.Create(s)
assert.NoError(t, err, "failed to create stack")
}
err = migrateStackEntryPoint(stackService)
assert.NoError(t, err, "failed to migrate entry point to Git ConfigFilePath")
s, err := stackService.Stack(1)
assert.NoError(t, err)
assert.Nil(t, s.GitConfig, "first stack should not have git config")
s, err = stackService.Stack(2)
assert.NoError(t, err)
assert.Equal(t, "", s.GitConfig.ConfigFilePath, "not migrated yet migrated")
err = migrator.MigrateStackEntryPoint(stackService)
assert.NoError(t, err, "failed to migrate entry point to Git ConfigFilePath")
s, err = stackService.Stack(1)
assert.NoError(t, err)
assert.Nil(t, s.GitConfig, "first stack should not have git config")
s, err = stackService.Stack(2)
assert.NoError(t, err)
assert.Equal(t, "dir/sub/compose.yml", s.GitConfig.ConfigFilePath, "second stack should have config file path migrated")

View File

@ -19,157 +19,8 @@ func (m *Migrator) Migrate() error {
return migrationError(err, "StoreIsUpdating")
}
// Portainer < 1.12
if m.currentDBVersion < 1 {
err := m.updateAdminUserToDBVersion1()
if err != nil {
return migrationError(err, "updateAdminUserToDBVersion1")
}
}
// Portainer 1.12.x
if m.currentDBVersion < 2 {
err := m.updateResourceControlsToDBVersion2()
if err != nil {
return migrationError(err, "updateResourceControlsToDBVersion2")
}
err = m.updateEndpointsToDBVersion2()
if err != nil {
return migrationError(err, "updateEndpointsToDBVersion2")
}
}
// Portainer 1.13.x
if m.currentDBVersion < 3 {
err := m.updateSettingsToDBVersion3()
if err != nil {
return migrationError(err, "updateSettingsToDBVersion3")
}
}
// Portainer 1.14.0
if m.currentDBVersion < 4 {
err := m.updateEndpointsToDBVersion4()
if err != nil {
return migrationError(err, "updateEndpointsToDBVersion4")
}
}
// https://github.com/portainer/portainer/issues/1235
if m.currentDBVersion < 5 {
err := m.updateSettingsToVersion5()
if err != nil {
return migrationError(err, "updateSettingsToVersion5")
}
}
// https://github.com/portainer/portainer/issues/1236
if m.currentDBVersion < 6 {
err := m.updateSettingsToVersion6()
if err != nil {
return migrationError(err, "updateSettingsToVersion6")
}
}
// https://github.com/portainer/portainer/issues/1449
if m.currentDBVersion < 7 {
err := m.updateSettingsToVersion7()
if err != nil {
return migrationError(err, "updateSettingsToVersion7")
}
}
if m.currentDBVersion < 8 {
err := m.updateEndpointsToVersion8()
if err != nil {
return migrationError(err, "updateEndpointsToVersion8")
}
}
// https: //github.com/portainer/portainer/issues/1396
if m.currentDBVersion < 9 {
err := m.updateEndpointsToVersion9()
if err != nil {
return migrationError(err, "updateEndpointsToVersion9")
}
}
// https://github.com/portainer/portainer/issues/461
if m.currentDBVersion < 10 {
err := m.updateEndpointsToVersion10()
if err != nil {
return migrationError(err, "updateEndpointsToVersion10")
}
}
// https://github.com/portainer/portainer/issues/1906
if m.currentDBVersion < 11 {
err := m.updateEndpointsToVersion11()
if err != nil {
return migrationError(err, "updateEndpointsToVersion11")
}
}
// Portainer 1.18.0
if m.currentDBVersion < 12 {
err := m.updateEndpointsToVersion12()
if err != nil {
return migrationError(err, "updateEndpointsToVersion12")
}
err = m.updateEndpointGroupsToVersion12()
if err != nil {
return migrationError(err, "updateEndpointGroupsToVersion12")
}
err = m.updateStacksToVersion12()
if err != nil {
return migrationError(err, "updateStacksToVersion12")
}
}
// Portainer 1.19.0
if m.currentDBVersion < 13 {
err := m.updateSettingsToVersion13()
if err != nil {
return migrationError(err, "updateSettingsToVersion13")
}
}
// Portainer 1.19.2
if m.currentDBVersion < 14 {
err := m.updateResourceControlsToDBVersion14()
if err != nil {
return migrationError(err, "updateResourceControlsToDBVersion14")
}
}
// Portainer 1.20.0
if m.currentDBVersion < 15 {
err := m.updateSettingsToDBVersion15()
if err != nil {
return migrationError(err, "updateSettingsToDBVersion15")
}
err = m.updateTemplatesToVersion15()
if err != nil {
return migrationError(err, "updateTemplatesToVersion15")
}
}
if m.currentDBVersion < 16 {
err := m.updateSettingsToDBVersion16()
if err != nil {
return migrationError(err, "updateSettingsToDBVersion16")
}
}
// Portainer 1.20.1
if m.currentDBVersion < 17 {
err := m.updateExtensionsToDBVersion17()
if err != nil {
return migrationError(err, "updateExtensionsToDBVersion17")
}
return migrationError(err, "migrating from less than Portainer 1.21.0 is not supported, please contact Portainer support.")
}
// Portainer 1.21.0

Some files were not shown because too many files have changed in this diff Show More