Commit Graph

5750 Commits (c533044cdc582b6b3b738b9f7cbf7489744b1773)

Author SHA1 Message Date
Amelia Bruno 52bd033a02
tweak(setup): Improvements to setup script (#7469)
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
Co-authored-by: Toran Bruce Richards <toran.richards@gmail.com>
2024-08-05 00:13:25 -05:00
Andy Hooker bb5baadeb2
feat(builder): Implementation of UI / UX improvement to editor (#7690)
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-08-05 00:09:31 -05:00
Andy Hooker db97b24518
feat(builder): Addition of blocks control component (#7688)
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-08-04 23:58:12 -05:00
Andy Hooker 533d7b7da8
feat(builder): Addition of SaveControl component for ControlPanel (#7687)
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-08-04 23:39:42 -05:00
Andy Hooker 183c2a4845
feat(builder): Addition ControlPanel and Separator components (#7680) 2024-08-04 23:38:14 -05:00
alex 6440a8e217
fix punctuation in setup.md (#7689) 2024-08-04 06:53:32 -07:00
Toran Bruce Richards e0930ba39d
feat(Block): Add AdvancedLlmCallBlock (#7677)
* feat(Block): Add AdvancedLlmCallBlock

Adds a block for handling advanced LLM calls, enabling messages to be handled within the AutoGPT builder.

* fix linting
2024-08-02 18:17:03 -05:00
Andy Hooker a21fd30fce
fix(github): Update CODEOWNERS for rnd directories (#7684)
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-08-02 17:23:28 -05:00
Nicholas Tindle e2df2cd90d
Revert "feat(builder): Rewrite node input UI" (#7685) 2024-08-02 17:07:03 -05:00
Toran Bruce Richards 6bdb849150
feat(Block): Add blocks for Building Lists, Dictionaries & Strings (#7678)
* feat(Block): Add block for building lists

* feat(Block): Add block for building dictionaries

* feat(Block): Add TextCombinerBlock for combining multiple input texts

* Fix linting issues where default values are None

---------

Co-authored-by: Bently <tomnoon9@gmail.com>
2024-08-02 16:49:51 -05:00
Bently 8469fafc6f
Feat(Builder): Discord bots blocks (#7670)
* Super early version of the discord bots blocks

* updated to add secrets for token + other fixes

* lint & format

* rename DiscordBot to DiscordReader

* add discord-py

* fix poetry lock

* rm duplicated file

* updated name to add Block to end

* update .env.template to add DISCORD_BOT_TOKEN

* updates to add description Field

* swap channel name and message content + add field description

---------

Co-authored-by: Bently <bently@bentlybro.com>
2024-08-02 22:21:58 +01:00
Reinier van der Leer 3c2c3e57a0
feat(builder): Rewrite node input UI (#7626)
- feat(builder): Rewrite & split up `NodeInputField`
  - Create `NodeObjectInputTree`
  - Create `NodeGenericInputField`
  - Create `NodeKeyValueInput`
  - Create `NodeArrayInput`
  - Create `NodeStringInput`
  - Create `NodeNumberInput`
  - Create `NodeBooleanInput`
  - Create `NodeFallbackInput`
  - Create `ClickableInput` from `renderClickableInput(..)`
  - Amend usage in `CustomNode`
  - Remove deprecated/unused styling from `flow.css` and `customnode.css`
  - Fix alignment between `NodeHandle` and `NodeInputField`
  - Split up `BlockIOSchema` & rename to `BlockIOSubSchema`
    - Create `BlockIOObjectSubSchema`
    - Create `BlockIOKVSubSchema`
    - Create `BlockIOArraySubSchema`
    - Create `BlockIOStringSubSchema`
    - Create `BlockIONumberSubSchema`
    - Create `BlockIOBooleanSubSchema`
    - Create `BlockIONullSubSchema`
  - Install `Select` component from shad/cn

- refactor(builder): Move `NodeInputField.tsx` to `node-input.tsx`
2024-08-02 19:28:46 +01:00
lelcruz ec6bae0467
fix(documentation): Corrected Markdown formatting for server start command (#7649)
Co-authored-by: Leslie Cruz <lelcruz@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-08-02 12:33:12 -05:00
Bently f5fe96260e
Fix(Builder): beautifyString duplicated import (#7679) 2024-08-02 15:39:42 +01:00
SwiftyOS 49a18437ac fix(builder) fix merge conflict errors 2024-08-02 16:29:55 +02:00
Reinier van der Leer 3cee893314
fix(builder): Rewrite `additionalProperties` input field (#7608)
Co-authored-by: Swifty <craigswift13@gmail.com>
2024-08-02 16:11:38 +02:00
Reinier van der Leer 5d1035aeb0
fix(builder): Fix `BlockSchema` handling & type propagation (#7604)
- Merge `Flow.tsx:CustomNodeData` type definition into `CustomNode.tsx:CustomNodeData`
- Move `lib/types:BlockSchema` type definition into `lib/autogpt-server-api/types` in place of `ObjectSchema`
- Expand and rename `BlockSchema` -> `BlockIOSchema` + `BlockIORootSchema`
- Fix all `BlockIOSchema` related type narrowing checks
- Add warning messages to fallback cases in `NodeInputField` logic

Co-authored-by: Swifty <craigswift13@gmail.com>
2024-08-02 16:09:46 +02:00
Bently 2e2c6fed52
feat(Builder): Implement undo/redo functionality (#7559)
* feat(Builder): Implement undo/redo functionality

* updates to work with latest UI

* add CTRL + Z & CTRL + Y support

* fixed undo/redo for inputs in nodes

* fix for deleting node

* fixes to make the undo/redo work better

* small fix

* Updates based on feedback

* add margin right to space out buttons

* added CTRL + SHIFT + Z for redo
2024-08-02 16:06:40 +02:00
Krzysztof Czerwinski ca9c52f76a
feat(builder): Add Supabase Auth, session and sign in UI (#7655)
This PR adds Supabase Auth (cloud) integration, login and profile UI, configures password login and three OAuth providers: Google, GitHub and Discord.

For `Account` button to show up and ability to login two env vars need to be set in `.env.local`: `NEXT_PUBLIC_SUPABASE_URL` and `NEXT_PUBLIC_SUPABASE_ANON_KEY`. OAuth providers are by the Supabase and don't require env vars.

Email confirmation (for email/password signup) is disabled because there's limit of 3 emails per hour without custom SMTP server configuration. [Link](https://supabase.com/dashboard/project/adfjtextkuilwuhzdjpf/auth/templates) to configure custom SMTP server and email template.

### Added dependencies:
- "@supabase/ssr": "^0.4.0"
- "@supabase/supabase-js": "^2.45.0"
- "react-icons": "^5.2.1"

### Added pages/routes:
- `app/auth/auth-code-error/page.tsx`: displayed when login using OAuth provider fails
- `app/auth/callback/route.ts`: route accessed when logging in using OAuth provider; it passes session code to Supabase
- `app/auth/confirm/route.ts`: accessed when confirming email, users will be directed here from email they get after signing in.
- `app/error/page.tsx`: Generic error page without explanation (any errors should be visible in the console)
- `app/login/page.tsx` and `app/login/actions.ts`: Login page and related login/signup server actions
- `app/profile/page.tsx`: Profile page, displays email address of the user and button to logout

### Changes
- Update `layout.tsx`: add `Log In` button and make icons consistent. The log in button shows up if user is logged out, avatar is shown when logged in, and if supabase is unavailable nothing shows up.
- Login form is verified using `zod` on the frontend (recommended by shadcn) and in case login fails feedback is displayed. On successful login users are redirected to `/profile`
- Add `PasswordInput` component, [source](https://gist.github.com/mjbalcueva/b21f39a8787e558d4c536bf68e267398)
- Add `SupabaseProvider` with context for Supabase accessed via hook `useSupabase(): { supabase: SupabaseClient | null, isLoading: boolean }`
- Add `useUser` hook to get `{ user, session, isLoading, error }` on the client 
- Add `getServerUser`: async function to get `{ user: User | null, error: string | null }` on the server side
- Add `src/middleware.ts` and `client.ts`, `server.ts`, `middleware.ts` in `src/lib/supabase` which are utility functions and middleware to refresh auth token
2024-08-02 12:58:28 +01:00
Zamil Majdy 973822d973
feat(rnd): Add InputBlock & OutputBlock (#7654)
### Background

We need an explicit block for providing input & output for the graph.
This will later allow us to build a subgraph with pre-declared input & output schema.
This will also allow us to set input for the node in the middle of the graph, and enable a graph to have output values.

### Changes 🏗️

* Add InputBlock & OutputBlock
* Add graph structure validation  on the graph execution step that asserts the following property:
    - All mandatory input pin, has to be connected or have a default value, except the `InputBlock` node.
    - All links have to connect valid nodes, and the sink & source name using the valid block field.
2024-08-02 13:33:34 +07:00
Bently e773329391
Feat(Builder): Clear Status and Output upon graph edit (#7656)
* Feat(Builder): Clear Status and Output upon graph edit

* close output dropdown on graph edit

* make deleting edge clear block outputs

* make it so deleting nodes clears block output
2024-08-01 17:21:01 +01:00
Zamil Majdy c9d41e69bd
feat(rnd): Add postgres schema check on linting step + auto gen migration and schema sync command (poetry run schema) (#7667)
### Background

Migration not being synced has caused issues and CI is not catching this yet.

### Changes 🏗️

* Added schema check in both sqlite & postgres prisma.schema on linting step.
* Introduced `poetry run schema` for syncing prisma.schema + generating migration in both files.
2024-08-01 20:11:07 +07:00
Nicholas Tindle 08905d71f9
docs(server): add docs link to server readme (#7638)
Co-authored-by: Swifty <craigswift13@gmail.com>
2024-08-01 14:32:08 +02:00
Aarushi 8becde370c
docs(ollama) Add ollama to mkdocs (#7666)
add ollama to mkdocs
2024-08-01 13:14:40 +01:00
Aarushi dccc33152b
feat(docs) Add how to run ollama with AutoGPT (#7665)
add ollama howto
2024-08-01 12:12:01 +01:00
Aarushi b23bd9c479
feat(server): Add JWT validation (#7642)
* add auth middleware

* update readnme

* Update rnd/autogpt_server/pyproject.toml

Co-authored-by: Krzysztof Czerwinski <34861343+kcze@users.noreply.github.com>

* address newline feedback

* update poetry lock

---------

Co-authored-by: Krzysztof Czerwinski <34861343+kcze@users.noreply.github.com>
2024-08-01 11:11:40 +01:00
Aarushi ac45b7cae9
feat(rnd,infra): Add Helm linting in CI (#7633)
* Set up helm and tf for backend

* update helm charts and settings

* remove example files

* use latest tag

* delay and timeouts for probes

* env based pyro host

* default backend

* linting

* add helm linting in CI

* read from settings

* fix formatting

* update to use config
2024-08-01 09:22:04 +01:00
Aarushi 3d54a9103c
feat(rnd, infra): Add deployment for backend (#7612)
* Set up helm and tf for backend

* update helm charts and settings

* remove example files

* use latest tag

* delay and timeouts for probes

* env based pyro host

* default backend

* linting

* read from settings

* fix formatting

* update to use config
2024-08-01 09:21:12 +01:00
Nicholas Tindle ca7182403b
fix(server): enable other methods of setting configs outside of the config.json file (#7645) 2024-07-31 12:33:36 -05:00
Zamil Majdy 53826ab360
fix(rnd): Fix bug caused by different AJV version (#7653) 2024-07-31 16:47:31 +07:00
Andy Hooker eac5548023
feat(builder): UI / UX upgrade and simplification of tailwind styles (#7647)
* fix(builder): Implementing a basic shadCn theme until color palette is decided upon

* feat(builder): Separating NavBar into its own component and providing a standard UI/UX Approach

* feat(builder): Removing old implementation of logo, removing excessive css implementation, updating styles to better support standard desktop views.

* feature(builder): Addition of UI component Sheet from ShadCn for update
2024-07-31 10:43:13 +02:00
Zamil Majdy 122f544966
fix(rnd): Disable unused prisma connection on pyro API Server process (#7641)
### Background

Pyro for API Server is not using Prisma, but still holding a Prisma connection.
The fast-API thread is also holding a Prisma connection, making Prisma connected in two different loop within a single process.

### Changes 🏗️

Disable a Prisma connection on Pyro thread for Server API process.
Fix test flakiness issue due to concurrency issue.
2024-07-30 22:33:28 +07:00
Zamil Majdy 29ba4c2c73
feat(rnd): Add freezed/static input link feature for retaining input data to be re-used in multiple node executions (#7631)
### Background

Input from the input pin is consumed only once. While this is required in most of the use cases, there are some cases where the input can only be produced once, and that input needs to be re-used just like an input default value, that is passively providing input data, without triggering any execution. The scope of this change is providing that functionality in the link level, this property will be called **`static link`** in this system.

### Changes 🏗️

Provides a static link feature with the following behaviours:
* A link can be marked `static` to become a static link.
* Once a node produces an output it will persist the output data and propagate the output to the other nodes through the link, for a static link, instead of making the data queued in the input pin, it will override the default value.
* Any input executions still waiting for the input will be backfilled using this output produced by the static link.
* And any upcoming executions that will use the input will always reuse the output produced by the static link.

See the added test to see the expected usage.
2024-07-30 10:37:38 +07:00
Nicholas Tindle 76feead3b1
docs(server): written setup guide for server (#7595)
Co-authored-by: Swifty <craigswift13@gmail.com>
2024-07-29 22:24:03 +01:00
Bently 081df805df
Fix(Builder): Copy and Paste when editing textbox causes issue (#7636)
Fix(Builder): Copy and Paste when editing textbox causes graph elements to be erroneously pasted
2024-07-29 21:55:19 +01:00
Nicholas Tindle acc1d79146
feat(builder): add delete handles to the edges and nodes (#7614)
Co-authored-by: Andy Hooker <58448663+andrewhooker2@users.noreply.github.com>
Co-authored-by: Toran Bruce Richards <toran.richards@gmail.com>
2024-07-29 21:30:56 +01:00
Nicholas Tindle 07811b2133
fix(builder): show colors for all statuses (#7625) 2024-07-29 20:04:48 +01:00
Amr Aly 01b6c2d4bf
Update README.advanced.md (#7607)
* Update README.advanced.md

The copy .env command is wrong. it should be .env.example instead of env.example

* fix(server): extra space removal

---------

Co-authored-by: Nicholas Tindle <nicktindle@outlook.com>
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
2024-07-29 12:00:58 +01:00
Krzysztof Czerwinski 905b1df218
feat(builder): Add input validation on the frontend (#7606)
- Add `ajv` dependency to check values against json schema
- Add `errors` and `setErrors` to `CustomNodeData`
- Add `validateNodes` run before executing agent
- Add `*` on labels for required fields
- Add `setNestedProperty` and `removeEmptyStringsAndNulls` utility function
2024-07-29 10:17:27 +01:00
Andy Hooker edf84fb9f8
feat(builder): Apply standard theming for shadCn and centralize providers (#7622) 2024-07-26 17:59:44 -05:00
Swifty b62c24dc77
Add Marketplace template (#7605)
* init

* Added sentry.io integration

* updated readme
2024-07-26 17:11:26 +02:00
Zamil Majdy dfa855f533
fix(rnd): Fix broken server execution update due to split prisma connection to different loops (#7620) 2024-07-26 15:47:41 +01:00
Aarushi da2111bafb
fix(server): Add missing PG migrations (#7619)
missing migrations
2024-07-26 14:38:05 +01:00
Reinier van der Leer b2dba39810
fix(builder/api-client): Fix websocket client types (#7603)
- Fix type signatures of `sendWebSocketMessage(..)`, `onWebSocketMessage(..)`, `runGraph(..)` in `autogpt-server-api/client`
  - Add `WebsocketMessageTypeMap`

- Fix type signature of `updateNodesWithExecutionData` in `FlowEditor`
2024-07-26 13:26:11 +02:00
Zamil Majdy d2a5bb286f
feat(rnd): Fix concurrency issue on Agent Server & Apply max-1-execution-per-node constraint in graph execution (#7551)
### Background

When multiple executors are executing the same node within the same graph execution, two node executions can read the same queue of input and read the same value—making the data that is supposed to be consumed once, consumed by two executions. The lack of lock & concurrency support for parallel execution within a single graph causes this issue.

Node concurrency also introduces poor UX in the current frontend implementation, when two nodes are executed in parallel, the current UI will not display its parallel execution update, but instead, it shows the updates that override each other. Until the execution observability is improved on the builder UI, this capability will be limited.

### Changes 🏗️

The scope of this change is to solve this issue by:
* Decouple Graph execution & Node execution, each has its own configured process pool.
* Make sure there is only 1 execution per node (we still allow parallel executions on different nodes) in a graph.
* Fixed concurrency issue by adding distributed lock API on agent_server.
* Few cleanups:
    - Add more logging with geid & neid prefix on graph/node executions
    - Moved execution status update to agent-server for a single source of status update (required by conn-manager/web-socket)
    - Configured node parallelism to 10 & graph parallelism to 10 by default, so in the very rare worst-case, there can be 100 node executions.
    - Re-use server resource for each integration test run
2024-07-26 17:08:03 +07:00
Toran Bruce Richards 36b9a0a930
feat(Blocks): Add conditional logic block (#7536)
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-07-25 14:25:07 -05:00
Aarushi f40db85b43
feat(rnd, infra) Add static ips module (#7588)
* add terraform

* gitignore update

* linting

* formatting and linting in ci

* store state in backend bucket

* Update .github/workflows/autogpt-infra-ci.yml

Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>

* add static ips module

* formatting

---------

Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
Co-authored-by: Swifty <craigswift13@gmail.com>
2024-07-25 17:50:19 +01:00
Reinier van der Leer 0767b17779
dx: Set Python type checking mode to "basic" for all subprojects (#7599) 2024-07-25 12:25:56 +01:00
Nicholas Tindle 0f0c13bae8
fix(server): gitignore was missing new file name (#7596)
Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
2024-07-25 10:07:16 +01:00
A.Daee 3b0cd9518d
feat(froge): Improvement in web components (#7068)
- Add `duckduckgo_backend` field to `WebSearchComponent` configuration
- Add `selenium_proxy` to `WebSeleniumComponent` configuration
- Update docs
2024-07-25 10:02:34 +01:00