Commit Graph

5864 Commits (405dd1659ed0ba92bbf5753a561a084344b40245)

Author SHA1 Message Date
Krzysztof Czerwinski 0454a9a7be fix(builder): Prevent zooming on input field modal
- Add `nowheel` class to Textarea parent div
2024-09-03 14:30:35 +02:00
Krzysztof Czerwinski 09951fed4b
feat(builder, server): Add advanced block inputs (#7934)
- Add `advanced` to `SchemaField` and pass it to `json_extra`
- Add `advanced` to `BlockIOSubSchemaMeta` type
- Update `CustomNode`, so that:
  - non-required advanced inputs are hidden
  - non-advanced and required inputs are always shown
2024-09-03 12:01:42 +01:00
Aarushi 6204d82d84
feat(server): Integrate `forge.logging` (#7915)
* feat(server): Integrate `forge.logging`

- Add `configure_logging()` in `.util.logging` - a wrapper for `forge.logging.configure_logging()` with project-specific extras
- Call `configure_logging()` in `.app.main()`, and in child process initializers (e.g. `AppProcess.execute_run_command(..)`, `ExecutorManager.on_graph_executor_start()`)
- Change some `logger.warning` statements to `logger.info` where appropriate

* fix warnings to info

* fix(rnd): Fix broken test and Input/Output block field renaming

* Rename

* fix(rnd): Fix flaky CI

* feat(server): Add OAuth handlers for GitHub, Notion, Google & amend store data structure (#7868)

- Add `BaseOAuthHandler` + 3 initial implementations
  - Add `GitHubOAuthHandler`
  - Add `NotionOAuthHandler`
  - Add `GoogleOAuthHandler`
- Amend `OAuth2Credentials` type
  - Add `metadata` attribute
  - Make `access_token_expires_at`, `refresh_token`, `refresh_token_expires_at` optional

* extend GCP Logger

* update manager & add flag

* linting

* use default logger behaviour

* update messages

* update another message

---------

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co>
2024-09-03 11:56:21 +01:00
Krzysztof Czerwinski 8c9fe5c167
feat(builder): Resizable input modal (#7955)
- Add minimize/maximize button in the corner of modal to make it significantly larger and centered
- Add copy button to copy all text
- Add optional `title` to display as a modal header
2024-09-03 11:42:25 +01:00
Reinier van der Leer 71de1a6a5e
fix(server): Fix type checking and propagation issues (#7941)
- fix type propagation by `AppService.run_and_wait(..)`
- fix type propagation by `@expose` and add note
- fix type propagation by `wait(..)` in `.executor.manager.execute_node(..)`
- fix type propagation by `wait(..)` in `.executor.manager._enqueue_next_nodes(..)`
- remove unnecessary null checks for `.data.graph.get_node(..)`
- fix type issue in `ExecutionScheduler`
- reduce use of `# type: ignore` in `.data.execution`
- reduce usage of `# type: ignore` in `.executor.manager`
- reduce usage of `# type: ignore` in `.server`
- reduce usage of `# type: ignore` in cli.py
- update `pyright` to v1.1.378
2024-09-02 14:13:56 +00:00
Bently 956165adf3
Feat(Builder): Fix block menu width (#7946) 2024-09-02 09:54:52 +00:00
Aarushi e4dc16a867
docs(server): Update docs with new docker compose steps (#7944)
* update setup

* add docker links
2024-09-02 10:55:54 +02:00
Aarushi cfa0b6610c
feat(rnd,infra): Pull out websockets away from server api (#7899)
* standalone websocket server

* add websocket url

* wip: talk to ws directly

* rename to api server

* dockerfile and queue

* fix paths

* update poetry lock

* helm charts for websockets

* create seperate deployments for websockets and rest server with redis queue for async comms

* delete duplicate queue

* add depends in ws_api

* singleton for conn manager

* update from review

* fix CI

* address feedback

* update readme

* update docker file and add migration step in readm

* ad watch

* add step to copy example env file

* put connect back in
2024-09-02 10:32:51 +02:00
Krzysztof Czerwinski 933baa0e8d
feat(builder): Store and display output data history in nodes (#7930)
- Update styling and use tailwind more
- Add `react-toast` dependency
- Fix output button not changing checked state on execution
- Make status a badge in node's corner
- Rename `output_data` to `executionResults` and store multiple results with execution UUIDs
- Add `DataTable` component that displays execution outputs
- Outputs can be copied and there's a toast displayed as a feedback
2024-09-01 11:18:57 +01:00
Bently 370b2dabe8
Feat(Builder): Add block categories to block menu (#7918) 2024-08-31 19:22:37 +00:00
Nicholas Tindle baa00a5b03
feat(builder,server): add review of agent submissions to the admin portal (#7914)
Co-authored-by: Aarushi <aarushik93@gmail.com>
2024-08-30 18:20:59 +00:00
Zamil Majdy 60a8e00578
fix(rnd): Fix overflowing card due to long block name (#7931) 2024-08-30 18:02:55 +00:00
Bently 85e7d678ce
Feat(Builder): Fix Multiple Connections Between Pins (#7924) 2024-08-30 18:15:18 +01:00
Krzysztof Czerwinski 476b307d69
feat(builder): Draw selected edges above nodes (#7893) 2024-08-30 16:39:22 +01:00
Bently 5dbfb4e3f1
feat(builder): Turn block border red on error (#7910)
* Turn block border red on error

* prettier
2024-08-30 15:09:19 +00:00
Zamil Majdy f6d09c74f5
feat(rnd): Rename Blocks (#7925)
### Background

Standardize block names with a focus of making them intuitive to a non-technical person.

### Changes 🏗️

Replace these names:
* TextLlmCallBlock -> AITextGeneratorBlock
* ObjectLlmCallBlock -> AIStructuredResponseBlock
* AdvancedLlmCallBlock -> AIConversationBlock
* CreateTalkingAvatarClipBlock -> CreateTalkingAvatarVideoBlock
* DiscordReaderBlock -> ReadDiscordMessagesBlock
* DiscordMessageSenderBlock -> SendDiscordMessageBlock
* YouTubeTranscriberBlock -> TranscribeYouTubeVideoBlock
* CreateMediumPostBlock -> PublishToMediumBlock
* ForEachBlock -> ListIteratorBlock
* MathsBlock -> CalculatorBlock
* CounterBlock -> CountItemsBlock
* ValueBlock -> StoreValueBlock
* PrintingBlock -> PrintToConsoleBlock
* DictionaryAddEntryBlock -> AddToDictionaryBlock
* ListAddEntryBlock -> AddToListBlock
* ObjectLookupBlock -> FindInDictionaryBlock
* RedditGetPostsBlock -> GetRedditPostsBlock
* RedditPostCommentBlock -> PostRedditCommentBlock
* WikipediaSummaryBlock -> GetWikipediaSummaryBlock
* WebSearchBlock -> SearchTheWebBlock
* WebScraperBlock -> ExtractWebsiteContentBlock
* GetOpenWeatherMapBlock -> GetWeatherInformationBlock
* HttpRequestBlock -> SendWebRequestBlock
* CurrentTimeBlock -> GetCurrentTimeBlock
* CurrentDateBlock -> GetCurrentDateBlock
* CurrentDateAndTimeBlock -> GetCurrentDateAndTimeBlock
* TimerBlock -> CountdownTimerBlock
* RSSReaderBlock -> ReadRSSFeedBlock
* TextMatcherBlock -> MatchTextPatternBlock
* TextParserBlock -> ExtractTextInformationBlock
* TextFormatterBlock -> FillTextTemplateBlock
* TextCombinerBlock -> CombineTextsBlock
2024-08-30 21:36:42 +07:00
Zamil Majdy 6d17e627e8
fix(rnd): Add null checking & remove console logs on Input UI component (#7927)
fix(rnd): And null checking & remove logs
2024-08-30 12:21:09 +00:00
Bently 5cfa807f00
Feat(Builder): Make the copy block button work (#7920)
* Feat(Builder): Make the copy block button work

* prettier

* Fixes

* fix block id

* prettier

* fix type
2024-08-30 11:48:33 +00:00
Bently 6fff06f0f6
Feat(Builder): Fix delete block is non-undoable (#7922)
* Feat(Builder): Fix delete block is non-undoable

* prettier
2024-08-30 08:05:37 +00:00
Aarushi cbe553a547
feat(infra): Add deployment for Market (#7907)
* deployment for marketplace

* set up deployment for marketplace
update nodes

update helm

* update health check & allow builder origin
2024-08-29 18:23:40 +00:00
Reinier van der Leer 96ef35536c
ci(server): Run tests even if linting fails 2024-08-29 15:42:49 +02:00
Reinier van der Leer 087d3a3760
test(server): Add type to `server` fixture usages 2024-08-29 15:42:15 +02:00
Zamil Majdy 5da58aa284
fix(rnd): Fix jumping caret problem on builder input text field (#7917)
Issue 1:
Input text field cursor keeps moving to the end of the text.
Try to type "Hello World!" into the input text. Then try to type "some string" in the middle of the "Hello" and "World".

Issue 2:
History should only tracks on the input box onBlur/onLeave
Try to type a "longcharacters" and try to undo it, the undo is removing 1 character at a time, polluting the history, and make the undo pretty much unusable.

Issue 3:
KeyValue & ArrayInput is non-undoable.
Try to add key-value or add an entry to the list, it doesn't undo the value, but you need to click as many number of entries being added to make the undo work again
2024-08-29 13:12:10 +00:00
Bently 7de12a2200
Feat(Builder): Add first guide tutorial (#7862)
* Feat(Builder): Add first guide tutorial

* added more steps + some fixes

* added local storage to fix starting every time going to build

* update copy & paste to support mac

* small fix

* Prettier fixes

* Added "Skip Tutorial" button to first step

* some fixes based on requests

* revert camelCase change

* add ability to use url to reset tutorial

* prettier

* Added Tutorial button next to tally

* prettier

* change pinBlocksPopover to setPinBlocksPopover

* fixes + update + prettier

* made the resetTutorial url dynamic

* force to /build on reset tutorial

* fix renaming

* prettier
2024-08-29 07:53:45 -05:00
Reinier van der Leer 8f1c63a7ea
feat(server): Add OAuth handlers for GitHub, Notion, Google & amend store data structure (#7868)
- Add `BaseOAuthHandler` + 3 initial implementations
  - Add `GitHubOAuthHandler`
  - Add `NotionOAuthHandler`
  - Add `GoogleOAuthHandler`
- Amend `OAuth2Credentials` type
  - Add `metadata` attribute
  - Make `access_token_expires_at`, `refresh_token`, `refresh_token_expires_at` optional
2024-08-29 09:11:37 +00:00
Zamil Majdy 6ec200f912 fix(rnd): Fix flaky CI 2024-08-28 15:59:51 -05:00
Zamil Majdy b5db7f575e Rename 2024-08-28 15:56:02 -05:00
Zamil Majdy 98c909f99f fix(rnd): Fix broken test and Input/Output block field renaming 2024-08-28 15:54:48 -05:00
Krzysztof Czerwinski c5615aa862
fix(builder): Prevent overflow in node output (#7912) 2024-08-28 16:27:42 +01:00
Bently e725305e15
fix(builder): Set default `value` in `NodeStringInput` to silence uncontrolled input warning (#7909)
add default value to fix bug
2024-08-28 12:44:06 +01:00
Aarushi 9551f54c35
feat(infra): Add websockets IP (#7905)
tf changes from original branch to unblock other work
2024-08-27 20:35:31 +01:00
Bently 777f7d25bf
Fix copy-paste of text creating unintended blocks (#7903) 2024-08-27 11:49:35 +01:00
Zamil Majdy ea6f37bf98
feat(rnd): Integrate Jinja2 into TextFormatter (#7891)
Add Jinja2 to TextFormatter while maintaining backward compatibility.
2024-08-27 07:17:57 +00:00
Krzysztof Czerwinski 299530cf95
refactor(builder): Update ReactFlow to version 12 & split up `Flow.tsx` (#7808)
Update ReactFlow to version 12 and split `Flow.tsx` into `useAgentGraph` hook that takes care of agent state and API calls to the server.

- Update ReactFlow to v12 ([migration guide](https://reactflow.dev/learn/troubleshooting/migrate-to-v12))
- Move `setIsAnyModalOpen` to `FlowContext`
- Make `setHardcodedValues` and `setErrors` functions of `CustomNode` and utilize new `updateNodeData` ReactFlow API
- Fix type errors
- `useAgentGraph` hook
  - Take care of all API calls, websocket, agent state and logic
  - Make saving and execution async and thus more consistent and reliable
	- Save&run requests are state
	- Wait for node ids to sync with backend reactively
	- Queue execution updates
  - Memoize functions using `useCallback`
2024-08-26 11:45:05 +01:00
Nicholas Tindle 1df7d527dd
Remove debug (#7890) 2024-08-25 17:59:41 +00:00
Aarushi 407cf858e7
fix(server): Ensure nameserver is started before other processes (#7788)
ensure nameserver is started before other processes
2024-08-24 20:54:11 +00:00
Aarushi a670b384f6
tweak(infra): Add 1 more node to GKE (#7889)
add one more node
2024-08-24 18:53:24 +01:00
Zamil Majdy f9b8b0a41a
feat(rnd): Add dynamic input pin for input object construction (#7871)
### Background

Currently, there is no way to construct the output of nodes into a composite data structure (list/dict/object) using the builder UI.

The backend already supports this feature by connecting the output pin to the input pin using these format:
* <pin_name>_$_<list_index> for constructing list
* <pin_name>_#_<dict_key> for constructing dict
* <pin_name>_@_<field_name> for constructing object

The scope of this PR is implementing the UX for this in the builder UI.

### Changes 🏗️

<img width="765" alt="image" src="https://github.com/user-attachments/assets/8fc319a4-1350-410f-98cf-24f2aa2bc34b">

This allows you to add more pins in a key value & list input: `_$_` list constructor & `_#_` dict constructor.
2024-08-23 18:21:38 +00:00
Zamil Majdy e59e138352
fix(rnd): Prevent boolean with no default value on AGPT-builder (#7884)
### Background

Boolean without default value is a UX problem. It's currently displayed as a toggle and it has no way to describe the `null` value.
So we need to prevent blocks from introducing a nullable boolean.

### Changes 🏗️

Add explicit check to prevent nullable boolean. Fix existing block field that has nullable boolean.
2024-08-23 18:03:21 +00:00
Aarushi a95ee693dd
fix(rnd): Add missing libs folder (#7887)
add missing libs folder
2024-08-23 16:42:24 +00:00
Aarushi 26f56114d1
feat(infra): Add ws server infra changes (#7886)
add ws server infra changes
2024-08-23 11:25:57 -05:00
Aarushi 45ace8ccab
Add number of workers to environment variables in in charts (#7869)
add workers to env vars
2024-08-23 20:21:51 +07:00
Aarushi 95af63b5ad
feat(libs): Add integration credentials store (#7826)
- Add `SupabaseIntegrationCredentialsStore` in `.supabase_integration_credentials_store`
- Add `supabase` dependency
- Add `pydantic` dependency

---------

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
2024-08-23 12:39:26 +02:00
Aarushi 012bad72e8
feat(rnd,blocks): Add D-ID Block (#7798)
* talking head

* linting

* remove clip id, not needed

* add more descriptive name

* add min requirement to polling attempts and intervals

* add docs and link to docs

* remove extra space

* force new tab

* fix linting

* add did key to .env.template
2024-08-23 09:39:55 +01:00
Nicholas Tindle efcd0f93ed
ci(all): take two on the global status checker (#7849)
* ci(all): fundamentally change how we do this

* Update workflow-checker.yml

* ci: dupe and move file in attempt to figure out why its not accessible

* Update workflow-checker.yml

* fix: sleep before checking, move files, improve code

* Update workflow-checker.yml

* Update check_actions_status.py

* Update check_actions_status.py

* Update workflow-checker.yml

* Update workflow-checker.yml

* ci: remove debug, add more

* Update check_actions_status.py

* Update check_actions_status.py

* Update check_actions_status.py

* Revert "Update check_actions_status.py"

This reverts commit 36134527af.

* Revert "Update check_actions_status.py"

This reverts commit 828aabf532.

* Update check_actions_status.py

* Update check_actions_status.py

* Update check_actions_status.py

* Update check_actions_status.py
2024-08-22 07:06:15 -05:00
Swifty 4c32b46d40
feat(builder): Add google analytics to the project (#7860) 2024-08-22 10:24:12 +01:00
Konstantinos Voulgaropoulos 41fbfe35fb
tweak(rnd): Trim Whitespace from BlockSecret to Prevent Authentication Issues (#7789)
* ensure secret value of BlockSecret is always trimmed

* avoid logging secret values when trimming
2024-08-21 14:44:26 -05:00
Zamil Majdy c719e4f177
fix(rnd): Fix JS event-loop freeze caused by websocket connection retry (#7861)
### Background

Websocket connection retry has no backoff period which causes event-loop freeze.

### Changes 🏗️

Add a back-off period on retry.
2024-08-21 21:41:15 +07:00
Zamil Majdy 3d62cec553
fix(rnd): Guarantee execution ordering per node by waiting the node completion (#7855)
### Background

We don't have an ordering guarantee on the node execution.
Let's say we have a node that has to execute different data A, B, and C.
The current implementation limits the execution to 1 execution at a time, but there is no guarantee that A, B, and C will be executed in order.

The initial implementation did not have any restrictions, so it used to be A, B, and C executed in parallel
In the current implementation with the per-node constraint, it's A, B, C are executed serially but with no guarantee of ordering.

The scope of this PR is to guarantee that order.

### Changes 🏗️

Guaranteeing the execution per node ordering by avoiding any re-enqueue mechanism. If there are two executions run in the same node, the first one will be executed and the other will block. The blocking mechanism is indeed sub-optimal, the performance improvement can be done later (a follow-up issue will be added).
2024-08-21 19:08:18 +07:00
Aarushi fa12564954
bug(infra): Add frontend config to ingres (#7854)
add frontend config to ingres
2024-08-21 11:31:16 +01:00