Commit Graph

197 Commits (3d21d54daba084821b410885adb8c6fe817ce0c6)

Author SHA1 Message Date
Nicholas Tindle 92bfbfad57
feat: generate simple auth tests (#8709) 2024-11-20 15:21:16 +00:00
Aarushi 520d0ca0e4
delete infra folder (#8555)
* delete infra folder

* remove ci
2024-11-19 14:21:57 +00:00
Nicholas Tindle ea214d9168
ci: fix classic ci (#8338)
* ci(frontend,backend,classic): update branch from develop to dev

* ci(frontend, infra): enable ci on other tools

* Update classic-autogpt-docker-ci.yml

* fix: don't error if the folder exists

* fix: drop bad test

* Revert "fix: drop bad test"

This reverts commit c478d3cf4c.

* fix: turn off the correct test 👀

* fix: remove more

* Discard changes to .github/workflows/classic-autogpt-ci.yml

* Update classic-autogpt-docker-ci.yml

* Update classic-autogpt-docker-release.yml

* Update classic-autogpts-ci.yml

* Discard changes to .github/workflows/classic-forge-ci.yml

* Discard changes to .github/workflows/classic-autogpts-ci.yml

* Discard changes to .github/workflows/classic-python-checks.yml

* Discard changes to .github/workflows/repo-pr-label.yml

* Discard changes to .github/workflows/platform-backend-ci.yml

* Update classic-benchmark-ci.yml

* Update classic-frontend-ci.yml
2024-11-15 01:48:00 -06:00
dependabot[bot] 9e4246602d
build(deps): bump peter-evans/repository-dispatch from 2 to 3 (#8609)
Bumps [peter-evans/repository-dispatch](https://github.com/peter-evans/repository-dispatch) from 2 to 3.
- [Release notes](https://github.com/peter-evans/repository-dispatch/releases)
- [Commits](https://github.com/peter-evans/repository-dispatch/compare/v2...v3)

---
updated-dependencies:
- dependency-name: peter-evans/repository-dispatch
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-11 19:59:12 +00:00
Reinier van der Leer c960bd870c
dx: Clean up PR template (#8541)
* dx: Clean up PR template
2024-11-08 17:44:55 +01:00
Aarushi c0f77c8e7a
ci(platform): Update pipelines to run from infra repo (#8595)
* deploy trigger

* update name

* test path change

* update prod deploy too

* remove envvars

* update prod deploy name
2024-11-07 22:39:59 -06:00
Nicholas Tindle b08ad973fa
Update codeql.yml (#8500)
Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
2024-11-07 02:51:01 +00:00
dependabot[bot] b223cc0233
build(deps): bump actions/checkout from 2 to 4 (#8553)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-04 16:21:54 -06:00
dependabot[bot] 7e63a8f94a
build(deps): bump google-github-actions/auth from 1 to 2 (#8468)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-28 20:28:05 +00:00
dependabot[bot] 456b1e55aa
build(deps): bump google-github-actions/setup-gcloud from 1 to 2 (#8469)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-28 15:24:01 -05:00
dependabot[bot] db0f1fa0ec
build(deps): bump peter-evans/create-pull-request from 6 to 7 (#8470)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-28 15:21:30 -05:00
dependabot[bot] 757fb075b1
build(deps): bump actions/cache from 2 to 4 (#8467)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-28 15:16:19 -05:00
Aarushi b2972cc9c6
fix(ci): Fix the prod workload identity pool defined in the prod yaml (#8456)
fix project id
2024-10-28 12:51:51 +00:00
dependabot[bot] 8f5bd40791
build(deps): bump google-github-actions/get-gke-credentials from 1 to 2 (#8437)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-25 17:53:14 -05:00
dependabot[bot] 19dc2093df
build(deps): bump docker/setup-buildx-action from 1 to 3 (#8436)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-25 22:47:52 +00:00
dependabot[bot] ca43f71fa8
build(deps): bump azure/setup-helm from 1 to 4 (#8439)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-25 22:45:44 +00:00
dependabot[bot] 2fa4d8042c
build(deps): bump helm/chart-testing-action from 2.6.0 to 2.6.1 in the production-dependencies group (#8435)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-25 17:43:36 -05:00
dependabot[bot] 75f3092ba2
build(deps): bump actions/setup-python from 4 to 5 (#8438)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-25 22:30:18 +00:00
Aarushi 08a8abeb58
docs(pr_template): Add reminder for config and infra changes (#8430)
add config change reminder
2024-10-24 19:31:53 +01:00
Aarushi d9b8e0d273
feat(platform/ci): Add prod migrations (#8396)
* ci with workload identity

* temp update

* update name

* wip

* update auth step

* update provider name

* remove audience

* temp set to false

* update registry naming

* update context

* update login

* revert temp updates

* add prod iam and pool

* add release deploy with approval

* use gha default approval behaviour

* add back in release trigger

* add new line

* add prod migrations

* prod migrations without check
2024-10-23 09:25:33 +01:00
Aarushi 404d0638de
feat(platform/ci) Add workflow for deploying to production (#8384)
* ci with workload identity

* temp update

* update name

* wip

* update auth step

* update provider name

* remove audience

* temp set to false

* update registry naming

* update context

* update login

* revert temp updates

* add prod iam and pool

* add release deploy with approval

* use gha default approval behaviour

* add back in release trigger

* add new line
2024-10-23 09:23:54 +01:00
Aarushi 2715b81ff5
feat(platform/ci): Run migrations in ci for dev (#8395)
* run migrations in ci

* update environment

* temp false

* add dev migrations

* remove code change step
2024-10-23 09:21:12 +01:00
Toran Bruce Richards 0c517216df
fix(frontend): Fix broken terms of use link (#8279)
Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-10-22 15:39:54 -05:00
Aarushi 30a62f898c
feat(platform/ci) Set up deploys from dev (#8355)
* ci with workload identity

* temp update

* update name

* wip

* update auth step

* update provider name

* remove audience

* temp set to false

* update registry naming

* update context

* update login

* revert temp updates
2024-10-22 13:45:30 +00:00
Nicholas Tindle 2437f58849
ci: create dependabot config (#8372)
* ci: create dependabot

* ci: target the dev branch for dependabot

* ci: group prs

* ci: group updates

---------

Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
2024-10-21 08:05:07 -05:00
Reinier van der Leer 5d4d2486da
ci: Enforce `dev` as base branch for development (#8369)
* Create repo-pr-enforce-base-branch.yml

* fix quotes

* test

* fix github token

* fix trigger and CLI config

* change back trigger because otherwise I can't test it

* fix the fix

* fix repo selection

* fix perms?

* fix quotes and newlines escaping in message

* Update repo-pr-enforce-base-branch.yml

* grrr escape sequences in bash

* test

* clean up

---------

Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
2024-10-17 17:15:15 +01:00
Nicholas Tindle d6d2820b92
fix(market): agent pagination and search errors (#8336)
* fix(market): agent pagination and search errors

* fix(frontend): search was not paginated

* fix: linting

* feat(market): linting ci

* fix(ci): branch limit name
2024-10-16 20:29:53 +00:00
Aarushi a2ef456525
fix(ci,platform): Add dev branch trigger to all ci (#8339)
* update ci for dev

* update classic

* remove duplicate dev
2024-10-15 10:57:24 +01:00
Reinier van der Leer 74e677baec
ci(frontend): Enforce consistent `yarn.lock` + minor DX improvements (#8316)
- ci(frontend): Ensure CI fails if `yarn.lock` is inconsistent with `package.json`
- dx(frontend): Add Prettier check to `lint` script in `package.json`
- dx(frontend): Add `packageManager` to `package.json` for Corepack support
- build(frontend): Use `yarn` consistently in the Dockerfile
2024-10-11 16:51:15 +02:00
Zamil Majdy daa054c79c
feat(backend): Make Redis connection Sync + Use Redis as Distributed Lock (#8197) 2024-10-07 15:39:32 +07:00
Reinier van der Leer c64a2d4101
dx(blocks): Auto-label block PRs (#8230)
Automatically apply the `platform/blocks` label to PRs that change files in `backend/blocks/`
2024-09-30 19:52:05 +00:00
Reinier van der Leer 5e2874c315
feat(platform): OAuth support + API key management + GitHub blocks (#8044)
## Config
- For Supabase, the back end needs `SUPABASE_URL`, `SUPABASE_SERVICE_ROLE_KEY`, and `SUPABASE_JWT_SECRET`
- For the GitHub integration to work, the back end needs `GITHUB_CLIENT_ID` and `GITHUB_CLIENT_SECRET`
- For integrations OAuth flows to work in local development, the back end needs `FRONTEND_BASE_URL` to generate login URLs with accurate redirect URLs

## REST API
- Tweak output of OAuth `/login` endpoint: add `state_token` separately in response
- Add `POST /integrations/{provider}/credentials` (for API keys)
- Add `DELETE /integrations/{provider}/credentials/{cred_id}`

## Back end
- Add Supabase support to `AppService`
- Add `FRONTEND_BASE_URL` config option, mainly for local development use

### `autogpt_libs.supabase_integration_credentials_store`
- Add `CredentialsType` alias
- Add `.bearer()` helper methods to `APIKeyCredentials` and `OAuth2Credentials`

### Blocks
- Add `CredentialsField(..) -> CredentialsMetaInput`

## Front end
### UI components
- `CredentialsInput` for use on `CustomNode`: allows user to add/select credentials for a service.
  - `APIKeyCredentialsModal`: a dialog for creating API keys
  - `OAuth2FlowWaitingModal`: a dialog to indicate that the application is waiting for the user to log in to the 3rd party service in the provided pop-up window
- `NodeCredentialsInput`: wrapper for `CredentialsInput` with the "usual" interface of node input components
- New icons: `IconKey`, `IconKeyPlus`, `IconUser`, `IconUserPlus`

### Data model
- `CredentialsProvider`: introduces the app-level `CredentialsProvidersContext`, which acts as an application-wide store and cache for credentials metadata.
- `useCredentials` for use on `CustomNode`: uses `CredentialsProvidersContext` and provides node-specific credential data and provider-specific data/functions
- `/auth/integrations/oauth_callback` route to close the loop to the `CredentialsInput` after a user completes sign-in to the external service
- Add `BlockIOCredentialsSubSchema`

### API client
- Add `isAuthenticated` method
- Add methods for integration OAuth flow: `oAuthLogin`, `oAuthCallback`
- Add CRD methods for credentials: `createAPIKeyCredentials`, `listCredentials`, `getCredentials`, `deleteCredentials`
- Add mirrored types `CredentialsMetaResponse`, `CredentialsMetaInput`, `OAuth2Credentials`, `APIKeyCredentials`
- Add GitHub blocks + "DEVELOPER_TOOLS" category
- Add `**kwargs` to `Block.run(..)` signature to support additional kwargs
- Add support for loading blocks from nested modules (e.g. `blocks/github/issues.py`)

#### Executor
- Add strict support for `credentials` fields on blocks
- Fetch credentials for graph execution and pass them down through to the node execution
2024-09-26 00:36:29 +02:00
Nicholas Tindle 46b8f9af0a
feat(builder): scaffold playwright (#8109) 2024-09-24 23:31:38 +00:00
Reinier van der Leer f109c3e019
ci: Set up customized CodeQL scanning (#8106)
This replaces the dynamically configured CodeQL scanning that we had set up through the GitHub settings.
2024-09-24 18:26:15 +02:00
Reinier van der Leer 679245416f
ci: Reduce PR Status Checker interval (#8137)
Reduce interval from 3 minutes to 30 seconds to reduce CI lag
2024-09-23 13:10:23 +00:00
Reinier van der Leer c533044cdc
ci(backend): Add Supabase (#8123)
- Set up Supabase in the Backend CI
  - Remove separate Postgres DB setup
- Remove unused MinIO service
- Remove multi-platform stuff
2024-09-23 09:45:20 +00:00
Swifty ef7cfbb860
refactor: AutoGPT Platform Stealth Launch Repo Re-Org (#8113)
Restructuring the Repo to make it clear the difference between classic autogpt and the autogpt platform:
* Move the "classic" projects `autogpt`, `forge`, `frontend`, and `benchmark` into a `classic` folder
  * Also rename `autogpt` to `original_autogpt` for absolute clarity
* Rename `rnd/` to `autogpt_platform/`
  * `rnd/autogpt_builder` -> `autogpt_platform/frontend`
  * `rnd/autogpt_server` -> `autogpt_platform/backend`
* Adjust any paths accordingly
2024-09-20 16:50:43 +02:00
Aarushi e3f35d79c7
tweak(.github): Update pr template wording (#8103)
* update pr template wording

* add what and how

* Update .github/PULL_REQUEST_TEMPLATE.md

---------

Co-authored-by: Toran Bruce Richards <toran.richards@gmail.com>
2024-09-19 12:44:50 +00:00
Aarushi 0040495143
tweak(.github): Update PR template (#8100)
* update PR template

* Update .github/PULL_REQUEST_TEMPLATE.md

Co-authored-by: Krzysztof Czerwinski <34861343+kcze@users.noreply.github.com>

* add note

* typo

---------

Co-authored-by: Krzysztof Czerwinski <34861343+kcze@users.noreply.github.com>
2024-09-19 13:00:16 +01:00
Reinier van der Leer 8fd22bcfd7
feat(server, builder): Implement "STOP" button for graph runs (#7892)
- feat(builder): Add "Stop Run" buttons to monitor and builder
  - Implement additional state management in `useAgentGraph` hook
    - Add "stop" request mechanism
    - Implement execution status tracking using WebSockets
    - Add `isSaving`, `isRunning`, `isStopping` outputs
    - Add `requestStopRun` method
      - Rename `requestSaveRun` to `requestSaveAndRun` for clarity
  - Add needed functionality for the above to `AutoGPTServerAPI` client
    - Add `stopGraphExecution` method
    - Add support for multiple handlers per WebSocket method
    - Fix parsing of timestamps in `execution_event` WebSocket messages
  - Add `IconSquare` from Lucide to `@/components/ui/icons`

- feat(server): Add `POST /graphs/{graph_id}/executions/{graph_exec_id}/stop` route
  - Add `stop_graph_run` method to `AgentServer`

- feat(server): Add `cancel_execution` method to `ExecutionManager`
  - Replace node executor `ProcessPoolExecutor` by `multiprocessing.Pool` (which has a `terminate()` method)
    - Remove now unnecessary `Executor.wait_future(..)` method
  - Add `get_graph_execution(..)` in `.data.execution`

- fix(server): Reduce number of node executors to 5 per graph executor
  This is necessary because `multiprocessing.Pool` spawns its workers on init, instead of based on demand like `ProcessPoolExecutor` does

- dx(server): Improve debug logging in `ExecutionManager`
- ci(server): Add debug logging mode to CI Pytest step

### Other improvements
Server:
- Improve output type of `ExecutionManager.add_execution(..)`
- Renamed a few things in `.server.rest_api` for consistency

Front end:
- Improved typing in `AutoGPTServerAPI` client
2024-09-05 14:42:28 +02:00
Aarushi 5000aa7ee0
tweak(rnd,docker) Remove SQLite (#7966)
* move migrations, update networking and dockignore

* update docs

* remove sqlite from ci

* remove schema linting checks

* fix formatting

* remove schema linting

* add test script

* formatting and linting

* stop pg not down

* seperate test db

* diff port

* remove duplicate
2024-09-04 10:18:57 +01:00
Reinier van der Leer 96ef35536c
ci(server): Run tests even if linting fails 2024-08-29 15:42:49 +02:00
Nicholas Tindle 1df7d527dd
Remove debug (#7890) 2024-08-25 17:59:41 +00:00
Nicholas Tindle efcd0f93ed
ci(all): take two on the global status checker (#7849)
* ci(all): fundamentally change how we do this

* Update workflow-checker.yml

* ci: dupe and move file in attempt to figure out why its not accessible

* Update workflow-checker.yml

* fix: sleep before checking, move files, improve code

* Update workflow-checker.yml

* Update check_actions_status.py

* Update check_actions_status.py

* Update workflow-checker.yml

* Update workflow-checker.yml

* ci: remove debug, add more

* Update check_actions_status.py

* Update check_actions_status.py

* Update check_actions_status.py

* Revert "Update check_actions_status.py"

This reverts commit 36134527af.

* Revert "Update check_actions_status.py"

This reverts commit 828aabf532.

* Update check_actions_status.py

* Update check_actions_status.py

* Update check_actions_status.py

* Update check_actions_status.py
2024-08-22 07:06:15 -05:00
lelcruz 60d25135e6
fix(github): Added in fallback variables for postgres testing (#7715)
Co-authored-by: Leslie Cruz <lelcruz@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-08-06 15:09:02 -05:00
Andy Hooker 6fa7d22c91
feat(builder): Addition of prettier for aligned of development styles (#7629)
* fix(builder): Adding prettier configuration files and to package.

* fix(builder): Running script "format" added to the package.json

* feat(builder): Adding a job to the yaml file. This job will run "format" which leverages prettier.

* feat(builder): Running script "format" and merging master

* feat(builder): Setting configuration to prettier defaults in .prettierrc.json, and adding a requested newline in the .prettierignore

* feat(builder): Updating the CI to not add a job for prettier but instead add a check to verify prettier was run before commiting.

* feat(builder): Confirming CI update fails when user does not run prettier first. Updating with file changes after prettier

* feat(builder): Running prettier write to fix warnings

* fix(builder): Removing .prett
per PR change request

* fix(builder): Running prettier formatter

* fix(builder): Running prettier formatter check found additional files

* fix(builder): Running prettier format

* fix(builder): Removing running "format" command from PR due to a change request.

* fix(builder): Removing running "format" command from PR due to a change request.

---------

Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-08-05 09:14:02 +02:00
Andy Hooker a21fd30fce
fix(github): Update CODEOWNERS for rnd directories (#7684)
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-08-02 17:23:28 -05:00
Aarushi ac45b7cae9
feat(rnd,infra): Add Helm linting in CI (#7633)
* Set up helm and tf for backend

* update helm charts and settings

* remove example files

* use latest tag

* delay and timeouts for probes

* env based pyro host

* default backend

* linting

* add helm linting in CI

* read from settings

* fix formatting

* update to use config
2024-08-01 09:22:04 +01:00
Aarushi 22b6dbbf6a
feat(rnd,infra) Set up terraform (#7565)
* add terraform

* gitignore update

* linting

* formatting and linting in ci

* store state in backend bucket
2024-07-25 09:45:36 +01:00
Nicholas Tindle 3e384c9771
ci(server): add sqlite processing (#7586)
* ci(server): add sqlite processing

* ci(server): try setting DATABASE_URL based on db platform

* fix(server): swap default back to sqlite

* ci(server): go back to database url

---------

Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
2024-07-24 14:46:39 +01:00