Commit Graph

77 Commits (master)

Author SHA1 Message Date
Zamil Majdy 4bfeddc03d
feat(platform/docker): add frontend service to docker-compose with env config improvements (#10615)
## Summary
This PR adds the frontend service to the Docker Compose configuration,
enabling `docker compose up` to run the complete stack, including the
frontend. It also implements comprehensive environment variable
improvements, unified .env file support, and fixes Docker networking
issues.

## Key Changes

### 🐳 Docker Compose Improvements
- **Added frontend service** to `docker-compose.yml` and
`docker-compose.platform.yml`
- **Production build**: Uses `pnpm build + serve` instead of dev server
for better stability and lower memory usage
- **Service dependencies**: Frontend now waits for backend services
(`rest_server`, `websocket_server`) to be ready
- **YAML anchors**: Implemented DRY configuration to avoid duplicating
environment values

### 📁 Unified .env File Support
- **Frontend .env loading**: Automatically loads `.env` file during
Docker build and runtime
- **Backend .env loading**: Optional `.env` file support with fallback
to sensible defaults in `settings.py`
- **Single source of truth**: All `NEXT_PUBLIC_*` and API keys can be
defined in respective `.env` files
- **Docker integration**: Updated `.dockerignore` to include `.env`
files in build context
- **Git tracking**: Frontend and backend `.env` files are now trackable
(removed from gitignore)

### 🔧 Environment Variable Architecture
- **Dual environment strategy**: 
- Server-side code uses Docker service names
(`http://rest_server:8006/api`)
  - Client-side code uses localhost URLs (`http://localhost:8006/api`)
- **Comprehensive config**: Added build args and runtime environment
variables
- **Network compatibility**: Fixes connection issues between frontend
and backend containers
- **Shared backend variables**: Common environment variables (service
hosts, auth settings) centralized using YAML anchors

### 🛠️ Code Improvements
- **Centralized env-config helper** (`/frontend/src/lib/env-config.ts`)
with server-side priority
- **Updated all frontend code** to use shared environment helpers
instead of direct `process.env` access
- **Consistent API**: All environment variable access now goes through
helper functions
- **Settings.py improvements**: Better defaults for CORS origins and
optional .env file loading

### 🔗 Files Changed
- `docker-compose.yml` & `docker-compose.platform.yml` - Added frontend
service and shared backend env vars
- `frontend/Dockerfile` - Simplified build process to use .env files
directly
- `backend/settings.py` - Optional .env loading and better defaults
- `frontend/src/lib/env-config.ts` - New centralized environment
configuration
- `.dockerignore` - Allow .env files in build context
- `.gitignore` - Updated to allow frontend/backend .env files
- Multiple frontend files - Updated to use env helpers
- Updates to both auto installer scripts to work with the latest setup!

## Benefits
-  **Single command deployment**: `docker compose up` now runs
everything
-  **Better reliability**: Production build reduces memory usage and
crashes
-  **Network compatibility**: Proper container-to-container
communication
-  **Maintainable config**: Centralized environment variable management
with .env files
-  **Development friendly**: Works in both Docker and local development
-  **API key management**: Easy configuration through .env files for
all services
-  **No more manual env vars**: Frontend and backend automatically load
their respective .env files

## Testing
-  Verified Docker service communication works correctly
-  Frontend responds and serves content properly  
-  Environment variables are correctly resolved in both server and
client contexts
-  No connection errors after implementing service dependencies
-  .env file loading works correctly in both build and runtime phases
-  Backend services work with and without .env files present

### Checklist 📋

#### For configuration changes:
- [x] `.env.default` is updated or already compatible with my changes
- [x] `docker-compose.yml` is updated or already compatible with my
changes
- [x] I have included a list of my configuration changes in the PR
description (under **Changes**)

🤖 Generated with [Claude Code](https://claude.ai/code)

---------

Co-authored-by: Lluis Agusti <hi@llu.lu>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
Co-authored-by: Claude <claude@users.noreply.github.com>
Co-authored-by: Bentlybro <Github@bentlybro.com>
2025-08-14 03:28:18 +00:00
Ubbe a33d58dd33
chore(frontend): add generated files/queries to Git (#10281)
## Changes 🏗️

We want to make running the AutoGPT Front-end as easy as possible. For
that, you should be able to run it with the least amount of commands.

We recently added generated queries and types on the Front-end from the
Back-end OpenAPI schema, to make development easier and catch bugs
earlier. However, with the current setup, developers are forced to run
`pnpm generate:api-all` with the Back-end running, which is annoying.

After this PR, the Front-end can be rerun with just `pnpm i & pnpm dev`.

## Checklist 📋

### For code changes

- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] Run the Front-end with just `pnpm dev` and it works

---------

Co-authored-by: Abhimanyu Yadav <122007096+Abhi1992002@users.noreply.github.com>
2025-07-01 06:01:05 +00:00
Abhimanyu Yadav b5c7f381c1
feat(platform): centralize api calls in nextjs for token handling (#10222)
This PR helps to send all the React query requests through a Next.js
server proxy. It works something like this: when a user sends a request,
our custom mutator sends a request to the proxy server, where we add the
auth token to the header and send it to the backend again. 🌐

Users can send a client-side request directly to the backend server
because their browser does have access to auth tokens, so they need to
go via the Next.js server. 🚀

### Changes 🏗️

- Change the position of the generated client, mutator, and transfer
inside `/src/app/api`
- Update the mutator to send the request to the proxy server 
- Add a proxy server at `/api/proxy`, which handles the request using
`makeAuthenticatedRequest` and `makeAuthenticatedFileUpload` helpers and
sends the request to the backend
- Remove `getSupabaseClient`, because we do not have access to the auth
token on client side, hence no need 🔑
- Update Orval configs to generate the client at the new position 
- Added new backend updates to the auto-generated client.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] The setting page is using React Query and is working fine.
  - [x] The mutator is sending requests to the proxy server correctly.
  - [x] The proxy server is handling requests correctly.
- [x] The response handling is correct in both the proxy server and the
custom mutator.
2025-06-30 08:31:08 +00:00
Abhimanyu Yadav 94aed94113
feat(platform): setup and configure orval (#10209)
This pull request sets up and configures Orval for API client
generation. It automates the process of creating TypeScript clients from
the backend's OpenAPI specification, improving development efficiency
and reducing manual code maintenance.

### Changes 🏗️

- Configures Orval with a new configuration file (`orval.config.ts`).
- Adds scripts to `package.json` for fetching the OpenAPI spec and
generating the API client.
- Implements a custom mutator for handling authentication.
- Adds API client generation as a step in the CI workflow.
- Adds `.gitignore` entry for generated API client files.
- Adds a security middleware to prevent caching of sensitive data.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] Verified that the API client is generated correctly.
- [x] Confirmed that the custom mutator is functioning as expected for
authentication.
- [x] Ensured that the new CI workflow step for API client generation is
successful.
  - [x] Tested generated API calls

#### For configuration changes:
- [x] `.env.example` is updated or already compatible with my changes
- [ ] `docker-compose.yml` is updated or already compatible with my
changes
- [x] I have included a list of my configuration changes in the PR
description (under **Changes**)
2025-06-24 14:00:19 +00:00
Nicholas Tindle 81d3eb7c34
feat(backend, frontend): make changes to use our security modules more effectively (#10123)
<!-- Clearly explain the need for these changes: -->
Doing the CASA Audit and this is something to check
### Changes 🏗️
- limits APIs to use their specific endpoints
- use expected trusted sources for each block and requests call
- Use cryptographically valid string comparisons
- Don't log secrets

<!-- Concisely describe all of the changes made in this pull request:
-->

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
  - [x] Testing in dev branch once merged

---------

Co-authored-by: Swifty <craigswift13@gmail.com>
2025-06-16 15:22:08 +00:00
Swifty 2de5e3dd83
feat(platform): Agent Store V2 (#8874)
# 🌎 Overview

AutoGPT Store Version 2 expands on the Pre-Store by enhancing agent
discovery, providing richer content presentation, and introducing new
user engagement features. The focus is on creating a visually appealing
and interactive marketplace that allows users to explore and evaluate
agents through images, videos, and detailed descriptions.

### Vision

To create a visually compelling and interactive open-source marketplace
for autonomous AI agents, where users can easily discover, evaluate, and
interact with agents through media-rich listings, ratings, and version
history.

### Objectives

📊 Incorporate visuals (icons, images, videos) into agent listings.
 Introduce a rating system and agent run count.
🔄 Provide version history and update logs from creators.
🔍 Improve user experience with advanced search and filtering features.

### Changes 🏗️

<!-- Concisely describe all of the changes made in this pull request:
-->

### Checklist 📋

#### For code changes:
- [ ] I have clearly listed my changes in the PR description
- [ ] I have made a test plan
- [ ] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
  - [ ] ...

<details>
  <summary>Example test plan</summary>
  
  - [ ] Create from scratch and execute an agent with at least 3 blocks
- [ ] Import an agent from file upload, and confirm it executes
correctly
  - [ ] Upload agent to marketplace
- [ ] Import an agent from marketplace and confirm it executes correctly
  - [ ] Edit an agent from monitor, and confirm it executes correctly
</details>

#### For configuration changes:
- [ ] `.env.example` is updated or already compatible with my changes
- [ ] `docker-compose.yml` is updated or already compatible with my
changes
- [ ] I have included a list of my configuration changes in the PR
description (under **Changes**)

<details>
  <summary>Examples of configuration changes</summary>

  - Changing ports
  - Adding new services that need to communicate with each other
  - Secrets or environment variable changes
  - New or infrastructure changes such as databases
</details>

---------

Co-authored-by: Bently <tomnoon9@gmail.com>
Co-authored-by: Aarushi <aarushik93@gmail.com>
2024-12-13 16:35:02 +00:00
Nicholas Tindle b62f411518
feat(frontend): monitor tests (#8880)
<!-- Clearly explain the need for these changes: -->
We want to be able to test the monitor page with importing and exporting
agents
### Changes 🏗️

- Adds more test ids
- Builds out monitor.page.ts
- adds import export tests
- Fixes #8791, fixes #8795, fixes #8792
<!-- Concisely describe all of the changes made in this pull request:
-->

### Checklist 📋

#### For code changes:
- [ ] I have clearly listed my changes in the PR description
- [ ] I have made a test plan
- [ ] I have tested my changes according to the test plan:
Writing/Running the automated tests
2024-12-06 21:14:33 +00:00
Nicholas Tindle 5dd151b41e
feat(tests): add baseline utility for integration testing from frontend ui (#8765) 2024-11-27 09:44:19 +00:00
Swifty ef7cfbb860
refactor: AutoGPT Platform Stealth Launch Repo Re-Org (#8113)
Restructuring the Repo to make it clear the difference between classic autogpt and the autogpt platform:
* Move the "classic" projects `autogpt`, `forge`, `frontend`, and `benchmark` into a `classic` folder
  * Also rename `autogpt` to `original_autogpt` for absolute clarity
* Rename `rnd/` to `autogpt_platform/`
  * `rnd/autogpt_builder` -> `autogpt_platform/frontend`
  * `rnd/autogpt_server` -> `autogpt_platform/backend`
* Adjust any paths accordingly
2024-09-20 16:50:43 +02:00
Reinier van der Leer b7a90ce768 fix(autogpt_builder): Unignore `lib/` and add `lib/utils.ts` 2024-07-04 17:40:59 -06:00
Nicholas Tindle 785a40ff9d
feat(server, autogpt): Add Example files and update build option (#7271) 2024-06-27 09:56:21 -05:00
Aarushi dd960f9306
Add support for nextjs based app (#7266)
* Getting started with nextjs

* fix linting

* remove gitignore for package.json
2024-06-27 10:02:54 +01:00
Krzysztof Czerwinski bcc5282aba
refactor(forge): Clean `forge` (#7117)
Remove unused `forge` code and improve structure of `forge`.

* Put all Agent Protocol stuff together in `forge.agent_protocol`
  * ... including `forge.agent_protocol.database` (was `forge.db`)
* Remove duplicate/unused parts from `forge`
  * `forge.actions`, containing old commands; replaced by `forge.components` from `autogpt`
  * `forge/agent.py` (the old one, `ForgeAgent`)
  * `forge/app.py`, which was used to serve and run the `ForgeAgent`
  * `forge/db.py` (`ForgeDatabase`), which was used for `ForgeAgent`
  * `forge/llm.py`, which has been replaced by new `forge.llm` module which was ported from `autogpt.core.resource.model_providers`
  * `forge.memory`, which is not in use and not being maintained
  * `forge.sdk`, much of which was moved into other modules and the rest is deprecated
  * `AccessDeniedError`: unused
  * `forge_log.py`: replaced with `logging`
  * `validate_yaml_file`: not needed
  * `ai_settings_file` and associated loading logic and env var `AI_SETTINGS_FILE`: unused
  * `prompt_settings_file` and associated loading logic and env var `PROMPT_SETTINGS_FILE`: default directives are now provided by the `SystemComponent`
    * `request_user_double_check`, which was only used in `AIDirectives.load`
  * `TypingConsoleHandler`: not used
2024-05-21 19:18:47 +01:00
Swifty 34fdbaa26b
Remove arena (#7134)
* remove arena

* refactor: Remove Arena intake workflow

* Remove all mention of the arena

* remove evo.ninja
2024-05-09 11:36:40 +02:00
Jordan Ellis 0e332c0c12
feat: add evo.ninja (setup only) (#6379) 2023-11-27 17:29:32 +01:00
Merwane Hamadi 5fd82c7f15 Add CI for the hackathon and fix TestAgent.json 2023-10-26 08:27:47 -07:00
Jonathan Balloch 416093ab69
Granger entering the arena (#5401)
* .github_access_token added to git ignore

* Granger entering the arena
2023-09-29 18:44:10 +02:00
SwiftyOS 50842af1e5 Made the action only trigger if the frontend is modified 2023-09-19 12:10:39 +02:00
SwiftyOS cd589e9891 added git commands workflow to cli 2023-09-15 19:44:37 +02:00
SwiftyOS cc4706ece0 Added ignore private files 2023-09-12 11:00:30 +02:00
Reinier van der Leer 1cfc3f9d43
Auto-GPT: unbreak Docker release build 2023-09-09 03:52:04 +02:00
Luke f26ccda097
Bulleting update & version bump (#5112)
* Bulleting update & version bump

* Bulletin.md updates

* Format bulletin

* Added info about new documentation theme.

---------

Co-authored-by: lc0rp <2609411+lc0rp@users.noreply.github.com>
2023-08-11 14:44:27 +02:00
Reinier van der Leer 2c53530e99
Fix path processing (#5032)
* Fix and clean up path processing in logs module

* Fix path processing throughout the project

* Fix plugins test

* Fix borky pytest vs mkdir(exist_ok=True)

* Update docs and gitignore for new workspace location

* Fix borky pytest vol.2

* ok james
2023-07-21 11:36:15 -07:00
Reinier van der Leer e8b6676b22
Restructure logs.py into a module; include log_cycle (#4921)
* Consolidate all logging stuff into one module

* Merge import statement for `logs` and `logs.log_cycle`

---------

Co-authored-by: James Collins <collijk@uw.edu>
2023-07-09 20:14:25 +02:00
Peter Petermann 9e99db1d3e
Only take subclasses of AutoGPTPluginTemplate as plugins
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: Luke <2609441+lc0rp@users.noreply.github.com>
2023-06-26 15:25:21 -04:00
merwanehamadi 512d7ba208
Remove analyze_code (#4705)
Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>
Co-authored-by: Erik Peterson <e@eriklp.com>
2023-06-15 12:55:50 -07:00
Nicholas Tindle dae58f8167
Update .gitignore (#4610) 2023-06-07 01:46:09 -05:00
Robin Richtsfeld 624af3945c
Update .gitignore (#882)
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
2023-06-07 00:12:12 -05:00
John Cole a6d4deaf20
Update setup.md (#3690)
Co-authored-by: Richard Beales <rich@richbeales.net>
2023-05-19 18:39:12 +01:00
James Collins dfcbf6eee6
Refactor/move singleton out of config module (#3161) 2023-04-24 17:24:57 -05:00
Richard Beales ad5d8b2341
Re-work Docs and split out README (using MkDocs) (#2894)
* Initial Documentation re-org

* remove testing link from readme

* rewrite quickstart

* get code blocks working across mkdocs and github

* add link to plugins repo

* add link to plugins repo and move readme to plugin template repo

* Add emoji to "Extensibility with Plugins" in readme

Co-authored-by: Reinier van der Leer <github@pwuts.nl>

* Make docs deploy workflow path-selective

* Also run workflow when the workflow is updated

* fix readme links under configuration subfolder

* shrink subheadings in readme

---------

Co-authored-by: Toran Bruce Richards <toran.richards@gmail.com>
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-04-22 12:56:22 +01:00
BillSchumacher 23c650ca10
Merge branch 'master' of https://github.com/BillSchumacher/Auto-GPT into plugin-support 2023-04-19 17:28:17 -05:00
Walter Nasich f582d9ca49
Delete unused folder /outputs/ (#1130)
Delete unused folder /outputs/ as it is no being used to store output files
2023-04-19 02:36:32 +02:00
BillSchumacher 085842d43c
Merge branch 'master' of https://github.com/BillSchumacher/Auto-GPT into plugin-support 2023-04-18 18:44:40 -05:00
Richard Beales 88ebebf74f Implement suggestions from pi - save current news to file 2023-04-18 21:45:09 +01:00
Evgeny Vakhteev c62c8c6e71 merge BillSchumacher/plugin-support, conflicts 2023-04-18 13:13:38 -07:00
Evgeny Vakhteev 7f4e38844f adding openai plugin loader 2023-04-17 14:57:55 -07:00
superherointj d47466ddf9 Add Nix flakes support through direnv
* Nix (https://nixos.org) is a reproducible build system.
* Enables Nix users to use/develop Auto-GPT, without installing PIP or any other future Auto-GPT dependency.
2023-04-17 16:22:46 +01:00
EH 9589334a30
Add File Downloading Capabilities (#1680)
* Added 'download_file' command

* Added util and fixed spinner

* Fixed comma and added autogpt/auto_gpt_workspace to .gitignore

* Fix linter issues

* Fix more linter issues

* Fix Lint Issues

* Added 'download_file' command

* Added util and fixed spinner

* Fixed comma and added autogpt/auto_gpt_workspace to .gitignore

* Fix linter issues

* Fix more linter issues

* Conditionally add the 'download_file' prompt

* Update args.py

* Removed Duplicate Prompt

* Switched to using path_in_workspace function
2023-04-17 03:34:02 +01:00
Merwane Hamadi d934d226ce Update .gitignore to properly handle virtual environments 2023-04-16 11:32:27 -07:00
Richard Beales de575eba60
Merge pull request #1586 from nicostubi/feature/gitignore-additions
Some more file extensions to ignore
2023-04-15 16:39:36 +01:00
Nicolas Stübi 84aed05ebb Merge branch 'master' into feature/gitignore-additions
# Conflicts:
#	.gitignore
2023-04-15 16:25:31 +02:00
Nicolas Stübi f9265e9b01 Some more file extensions to ignore 2023-04-15 16:20:26 +02:00
Merwane Hamadi dc4094b264 added smoke test 2023-04-14 17:20:02 -07:00
0xArty 6ca6a8aa60 added more tools 2023-04-14 20:17:37 +01:00
Dino Hensen d64f866bfa Convert to python module named autogpt.
Also fixed the Dockerfile.
Converting to module makes development easier.
Fixes coverage script in CI and test imports.
2023-04-14 10:27:41 -07:00
Maiko Bossuyt 8093ac7949
Merge branch 'master' into add_ingest_documents_script 2023-04-14 18:12:23 +02:00
Mike Kelly 9e27e0165d gitignore the logs file 2023-04-14 13:19:30 +01:00
Maiko Bossuyt 334400edd1
Merge branch 'Torantulino:master' into add_ingest_documents_script 2023-04-13 13:50:41 +02:00
Richard Beales c8b8673286
Merge pull request #802 from sweetlilmre/more_azure
More fixes for Azure hosting
2023-04-13 07:58:41 +01:00