## Config
- For Supabase, the back end needs `SUPABASE_URL`, `SUPABASE_SERVICE_ROLE_KEY`, and `SUPABASE_JWT_SECRET`
- For the GitHub integration to work, the back end needs `GITHUB_CLIENT_ID` and `GITHUB_CLIENT_SECRET`
- For integrations OAuth flows to work in local development, the back end needs `FRONTEND_BASE_URL` to generate login URLs with accurate redirect URLs
## REST API
- Tweak output of OAuth `/login` endpoint: add `state_token` separately in response
- Add `POST /integrations/{provider}/credentials` (for API keys)
- Add `DELETE /integrations/{provider}/credentials/{cred_id}`
## Back end
- Add Supabase support to `AppService`
- Add `FRONTEND_BASE_URL` config option, mainly for local development use
### `autogpt_libs.supabase_integration_credentials_store`
- Add `CredentialsType` alias
- Add `.bearer()` helper methods to `APIKeyCredentials` and `OAuth2Credentials`
### Blocks
- Add `CredentialsField(..) -> CredentialsMetaInput`
## Front end
### UI components
- `CredentialsInput` for use on `CustomNode`: allows user to add/select credentials for a service.
- `APIKeyCredentialsModal`: a dialog for creating API keys
- `OAuth2FlowWaitingModal`: a dialog to indicate that the application is waiting for the user to log in to the 3rd party service in the provided pop-up window
- `NodeCredentialsInput`: wrapper for `CredentialsInput` with the "usual" interface of node input components
- New icons: `IconKey`, `IconKeyPlus`, `IconUser`, `IconUserPlus`
### Data model
- `CredentialsProvider`: introduces the app-level `CredentialsProvidersContext`, which acts as an application-wide store and cache for credentials metadata.
- `useCredentials` for use on `CustomNode`: uses `CredentialsProvidersContext` and provides node-specific credential data and provider-specific data/functions
- `/auth/integrations/oauth_callback` route to close the loop to the `CredentialsInput` after a user completes sign-in to the external service
- Add `BlockIOCredentialsSubSchema`
### API client
- Add `isAuthenticated` method
- Add methods for integration OAuth flow: `oAuthLogin`, `oAuthCallback`
- Add CRD methods for credentials: `createAPIKeyCredentials`, `listCredentials`, `getCredentials`, `deleteCredentials`
- Add mirrored types `CredentialsMetaResponse`, `CredentialsMetaInput`, `OAuth2Credentials`, `APIKeyCredentials`
- Add GitHub blocks + "DEVELOPER_TOOLS" category
- Add `**kwargs` to `Block.run(..)` signature to support additional kwargs
- Add support for loading blocks from nested modules (e.g. `blocks/github/issues.py`)
#### Executor
- Add strict support for `credentials` fields on blocks
- Fetch credentials for graph execution and pass them down through to the node execution
Restructuring the Repo to make it clear the difference between classic autogpt and the autogpt platform:
* Move the "classic" projects `autogpt`, `forge`, `frontend`, and `benchmark` into a `classic` folder
* Also rename `autogpt` to `original_autogpt` for absolute clarity
* Rename `rnd/` to `autogpt_platform/`
* `rnd/autogpt_builder` -> `autogpt_platform/frontend`
* `rnd/autogpt_server` -> `autogpt_platform/backend`
* Adjust any paths accordingly
* update pr template wording
* add what and how
* Update .github/PULL_REQUEST_TEMPLATE.md
---------
Co-authored-by: Toran Bruce Richards <toran.richards@gmail.com>
- feat(builder): Add "Stop Run" buttons to monitor and builder
- Implement additional state management in `useAgentGraph` hook
- Add "stop" request mechanism
- Implement execution status tracking using WebSockets
- Add `isSaving`, `isRunning`, `isStopping` outputs
- Add `requestStopRun` method
- Rename `requestSaveRun` to `requestSaveAndRun` for clarity
- Add needed functionality for the above to `AutoGPTServerAPI` client
- Add `stopGraphExecution` method
- Add support for multiple handlers per WebSocket method
- Fix parsing of timestamps in `execution_event` WebSocket messages
- Add `IconSquare` from Lucide to `@/components/ui/icons`
- feat(server): Add `POST /graphs/{graph_id}/executions/{graph_exec_id}/stop` route
- Add `stop_graph_run` method to `AgentServer`
- feat(server): Add `cancel_execution` method to `ExecutionManager`
- Replace node executor `ProcessPoolExecutor` by `multiprocessing.Pool` (which has a `terminate()` method)
- Remove now unnecessary `Executor.wait_future(..)` method
- Add `get_graph_execution(..)` in `.data.execution`
- fix(server): Reduce number of node executors to 5 per graph executor
This is necessary because `multiprocessing.Pool` spawns its workers on init, instead of based on demand like `ProcessPoolExecutor` does
- dx(server): Improve debug logging in `ExecutionManager`
- ci(server): Add debug logging mode to CI Pytest step
### Other improvements
Server:
- Improve output type of `ExecutionManager.add_execution(..)`
- Renamed a few things in `.server.rest_api` for consistency
Front end:
- Improved typing in `AutoGPTServerAPI` client
* move migrations, update networking and dockignore
* update docs
* remove sqlite from ci
* remove schema linting checks
* fix formatting
* remove schema linting
* add test script
* formatting and linting
* stop pg not down
* seperate test db
* diff port
* remove duplicate
* fix(builder): Adding prettier configuration files and to package.
* fix(builder): Running script "format" added to the package.json
* feat(builder): Adding a job to the yaml file. This job will run "format" which leverages prettier.
* feat(builder): Running script "format" and merging master
* feat(builder): Setting configuration to prettier defaults in .prettierrc.json, and adding a requested newline in the .prettierignore
* feat(builder): Updating the CI to not add a job for prettier but instead add a check to verify prettier was run before commiting.
* feat(builder): Confirming CI update fails when user does not run prettier first. Updating with file changes after prettier
* feat(builder): Running prettier write to fix warnings
* fix(builder): Removing .prett
per PR change request
* fix(builder): Running prettier formatter
* fix(builder): Running prettier formatter check found additional files
* fix(builder): Running prettier format
* fix(builder): Removing running "format" command from PR due to a change request.
* fix(builder): Removing running "format" command from PR due to a change request.
---------
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
* Set up helm and tf for backend
* update helm charts and settings
* remove example files
* use latest tag
* delay and timeouts for probes
* env based pyro host
* default backend
* linting
* add helm linting in CI
* read from settings
* fix formatting
* update to use config
* ci(server): add sqlite processing
* ci(server): try setting DATABASE_URL based on db platform
* fix(server): swap default back to sqlite
* ci(server): go back to database url
---------
Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
### Background
Add formatter & linter command.
Tools: ruff --> isort --> black --> pyright.
### Changes 🏗️
Introduced:
* `poetry run format`
* `poetry run lint`
`poetry run lint` will be executed on CI.
- Add `autogpt` and `forge` dependency to the `autogpt_server`
- Add `AutoGPTAgentBlock` that initializes and runs a single agent loop on execution
- Add `BlockAgent` that inherits from `autogpt` `Agent` and is a thin extension on the agent that allows to disable components
- Add `OutputComponent` that adds `output` command for the agent
* Update instructions to set up OpenAI / GPT-4 access
* Add instructions to set up Anthropic access
* Add instructions to set up Groq access
* Remove GPT-specific `--gpt3only`, `--gpt4only` CLI flags and related logic
* Remove duplicate config instructions from docker setup page, replace it by a link to the standard setup instructions
- **FIX ALL LINT/TYPE ERRORS IN AUTOGPT, FORGE, AND BENCHMARK**
### Linting
- Clean up linter configs for `autogpt`, `forge`, and `benchmark`
- Add type checking with Pyright
- Create unified pre-commit config
- Create unified linting and type checking CI workflow
### Testing
- Synchronize CI test setups for `autogpt`, `forge`, and `benchmark`
- Add missing pytest-cov to benchmark dependencies
- Mark GCS tests as slow to speed up pre-commit test runs
- Repair `forge` test suite
- Add `AgentDB.close()` method for test DB teardown in db_test.py
- Use actual temporary dir instead of forge/test_workspace/
- Move left-behind dependencies for moved `forge`-code to from autogpt to forge
### Notable type changes
- Replace uses of `ChatModelProvider` by `MultiProvider`
- Removed unnecessary exports from various __init__.py
- Simplify `FileStorage.open_file` signature by removing `IOBase` from return type union
- Implement `S3BinaryIOWrapper(BinaryIO)` type interposer for `S3FileStorage`
- Expand overloads of `GCSFileStorage.open_file` for improved typing of read and write modes
Had to silence type checking for the extra overloads, because (I think) Pyright is reporting a false-positive:
https://github.com/microsoft/pyright/issues/8007
- Change `count_tokens`, `get_tokenizer`, `count_message_tokens` methods on `ModelProvider`s from class methods to instance methods
- Move `CompletionModelFunction.schema` method -> helper function `format_function_def_for_openai` in `forge.llm.providers.openai`
- Rename `ModelProvider` -> `BaseModelProvider`
- Rename `ChatModelProvider` -> `BaseChatModelProvider`
- Add type `ChatModelProvider` which is a union of all subclasses of `BaseChatModelProvider`
### Removed rather than fixed
- Remove deprecated and broken autogpt/agbenchmark_config/benchmarks.py
- Various base classes and properties on base classes in `forge.llm.providers.schema` and `forge.models.providers`
### Fixes for other issues that came to light
- Clean up `forge.agent_protocol.api_router`, `forge.agent_protocol.database`, and `forge.agent.agent`
- Add fallback behavior to `ImageGeneratorComponent`
- Remove test for deprecated failure behavior
- Fix `agbenchmark.challenges.builtin` challenge exclusion mechanism on Windows
- Fix `_tool_calls_compat_extract_calls` in `forge.llm.providers.openai`
- Add support for `any` (= no type specified) in `JSONSchema.typescript_type`
- Move `autogpt/Dockerfile` to `Dockerfile.autogpt`
- Write new selective `.dockerignore` (in repo root) to keep build context clean
- Amend `autogpt/docker-compose.yml` and all `autogpt-docker-*.yml` workflows accordingly
- Include `forge/` in docker build context so it can be used as a path dependency
- Include `frontend/` in docker builds
- Moved `autogpt` and `forge` to project root
- Removed `autogpts` directory
- Moved and renamed submodule `autogpts/autogpt/tests/vcr_cassettes` to `autogpt/tests/vcr_cassettes`
- When using CLI agents will be created in `agents` directory (instead of `autogpts`)
- Renamed relevant docs, code and config references from `autogpts/[forge|autogpt]` to `[forge|autogpt]` and from `*../../*` to `*../*`
- Updated `CODEOWNERS`, GitHub Actions and Docker `*.yml` configs
- Updated symbolic links in `docs`
* Matrix the AutoGPT Python CI's `test` job across Ubuntu, macOS and Windows
- Set up MinIO in a step rather than specifying it under `jobs[test].services`, because services are only supported on Linux runners
- Add Windows version of step to install Poetry
- Add macOS compatibility patches to 'Install Poetry (Unix)' and `setup_git_auth` steps
**Caveats:**
- **No Docker on macOS or Windows**
* Windows comes with Docker but only supports running Windows containers, while we're mainly interested in using Linux containers for code execution and/or running auxiliary services.
* [The macOS runner doesn't come with Docker](https://github.com/actions/runner-images/issues/17). Setting it up is possible but takes ~3-4 minutes, and the performance of the Colima engine is poor: a `docker pull` that takes 2 seconds on Linux takes 45 seconds on macOS.
- **No S3 service available on Windows**
It seems that running a background process [isn't possible on Windows](https://github.com/actions/runner/issues/598#issuecomment-2011890429), and neither is running Linux-based Docker containers.
* Add `autogpt-agent` and OS-specific flags to Codecov upload step
* Improve caching of Python dependencies in CI by changing the cache key
- Include hash of `poetry.lock` instead of `pyproject.toml` in key
- Remove date component from key; it was included to avoid getting stuck to old cached versions of packages when we were still using `requirements.txt`. With `poetry.lock` that is no longer a concern.
* Fix skip check in test_s3_file_storage.py
Zamil Majdy
Reinier van der Leer
Nicholas Tindle
Swifty
Aarushi
lelcruz
Andy Hooker
Krzysztof Czerwinski