Commit Graph

139 Commits (f0a5250da5d8893ba8ec7318384d4e2996f98df4)

Author SHA1 Message Date
Reinier van der Leer f0a5250da5
Implement loading `MemoryItem`s from file in `JSONFileMemory` (#4703)
Further changes:
* remove `init` param from `get_memory()`, replace usages by `memory.clear()`
* make token length calculation optional in `MemoryItem.dump()`
2023-06-15 17:45:14 +02:00
Erik Peterson 07d9b584f7
Correct and clean up JSON handling (#4655)
* Correct and clean up JSON handling

* Use ast for message history too

* Lint

* Add comments explaining why we use literal_eval

* Add descriptions to llm_response_format schema

* Parse responses in code blocks

* Be more careful when parsing in code blocks

* Lint
2023-06-13 09:54:50 -07:00
Reinier van der Leer a9d177eeeb
Remove unused function `split_file` from file_operations.py (#4658) 2023-06-12 02:20:39 +02:00
Erik Peterson 0594ba33a2
Pass agent to commands instead of config (#4645)
* Add config as attribute to Agent, rename old config to ai_config

* Code review: Pass ai_config

* Pass agent to commands instead of config

* Lint

* Fix merge error

* Fix memory challenge a

---------

Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
2023-06-10 15:48:50 -07:00
Erik Peterson 5fe600af9d
Clean up and fix issues with env configuration and .env.template (#4630)
Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
2023-06-09 15:28:30 -07:00
Erik Peterson ff4e53d0e6
Streamline / clarify shell command control configuration (#4628)
* Streamline / clarify shell command control configuration

* Fix lint
2023-06-09 11:48:20 -07:00
Erik Peterson 94280b2d14
Add command for directly executing python code (#4581)
* Add command for directly executing python code

* Fix docstring

* Clarify / update filename references

---------

Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
2023-06-09 10:40:32 -07:00
Erik Peterson fdc6e12945
Improve logic and error messages for file reading and writing with Python code (#4567)
* Fix issues with file reading and writing with Python code

* Change error message, use Workspace.get_path

---------

Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-06-07 02:54:02 +02:00
merwanehamadi 1b04e5cafc
Fix duckduckgo rate limiting (#4592)
* Fix duckduckgo rate limiting

* use list instead of loop

---------

Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-06-07 01:20:52 +02:00
Erik Peterson 055806e124
Fix inverted logic for deny_command (#4563) 2023-06-06 22:56:17 +02:00
Benny van der Lans 74e8a886e6
Add `replace_in_file` command (#4565)
Resubmission of #3643

---------

Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-06-04 19:37:35 +02:00
merwanehamadi ee9f10a8d8
remove unused imports automatically (#4449)
* remove unused imports automatically

* add linters to pr template

* remove useless try statement
2023-05-28 05:50:50 -07:00
k-boikov 064e95b46b
Allow spinning to be disabled (#4329) 2023-05-27 00:01:34 +03:00
Nicholas Tindle acfd966aa4
Pass Configs to Commands and remove CFG = Config() in the commands/ folder (#4328)
* feat: pass config to call_ai_functions in coimmands

* feat: config for read_audio_from_file

* feat: file operations cfg

NOTE: we replaced the CFG in the command enable with TRUE b/c not sure how to handle this yet

* feat: git command conversion

* feat: google search

* feat: image generation

* feat: extract cfg from browser commands

* feat: remove cfg from execute code commands

* fix: file operation related tests

* fix: linting

* fix: tests for read_audio

* fix: test error

* feat: update cassettes

* fix: linting

* fix: test typechecking

* fix: google_search errors if unexpected kw arg is passed

* fix: pass config param to google search test

* fix: agent commands were broken + cassettes

* fix: agent test

* feat: cassettes

* feat: enable/disable logic for commands

* fix: some commands threw errors

* feat: fix tests

* Add new cassettes

* Add new cassettes

* ci: trigger ci

* Update autogpt/commands/execute_code.py

Co-authored-by: Reinier van der Leer <github@pwuts.nl>

* fix prompt

* fix prompt + rebase

* add config remove useless imports

* put back CFG just for download file

* lint

* The signature should be mandatory in the decorator

* black isort

* fix: remove the CFG

* fix: non typed arg

* lint: type some args

* lint: add types for libraries

* Add new cassettes

* fix: windows compatibility

* fix: add config access to decorator

* fix: remove twitter mention

* DDGS search works at 3.0.2 version

* ci: linting

---------

Co-authored-by: Auto-GPT-Bot <github-bot@agpt.co>
Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
Co-authored-by: kinance <kinance@gmail.com>
2023-05-26 08:39:25 -07:00
Reinier van der Leer bfbe613960
Vector memory revamp (part 1: refactoring) (#4208)
Additional changes:

* Improve typing

* Modularize message history memory & fix/refactor lots of things

* Fix summarization

* Move memory relevance calculation to MemoryItem & improve test

* Fix import warnings in web_selenium.py

* Remove `memory_add` ghost command

* Implement overlap in `split_text`

* Move memory tests into subdirectory

* Remove deprecated `get_ada_embedding()` and helpers

* Fix used token calculation in `chat_with_ai`

* Replace Message TypedDict by dataclass

* Fix AgentManager singleton issues in tests

---------

Co-authored-by: Auto-GPT-Bot <github-bot@agpt.co>
2023-05-25 20:31:11 +02:00
Nicholas Tindle 43565b280f
Remove Twitter Command (#4381) 2023-05-24 22:40:30 -05:00
Kinance 5168cb5219
Migrate google search to use DDGS.text function (#4383) 2023-05-24 19:36:23 +03:00
Robin Richtsfeld 77ee9f8119
Pass command line args as list (#1486)
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
Co-authored-by: Richard Beales <rich@richbeales.net>
2023-05-22 22:53:31 +03:00
sherif-med 31525dfef7
Text file loaders (#3031)
* adding requiered packages for loading pdf, docx, md, tex files (preferably pure python packages)

* adding text file utils providing function to load file based on extension && adding read_text_file command

* adding test cases for text file loading (pdf file creation is hardcoded due to external package requierment for creation (a sample file can be added))

* formatting

* changing command name from 'read_text_file' to 'parse_text_document'

* fallback to txtParser if file extension is not known to read script and code files

* adding extension respective parsers

* adding binary file check function

* adding file existance check && raising valueError for unsupported binary file formats

* adding check file type (binary) in test_parsers for specific extensions && fixing mock pdf generation to include null bytes

* adding .yml extension parser

* removal of .doc parser

* updating file loading commands names

* updating test (removing .doc mock function)

* fix: import sort

* new cassette for mem A

* feat: update Cassettes

* feat: consolidate commands

* feat: linting

* feat: updates to cassettes

---------

Co-authored-by: Reinier van der Leer <github@pwuts.nl>
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
2023-05-21 14:48:40 -05:00
k-boikov bcc32cc441
Fix split_file when overlap = 0, add test (#3599)
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
2023-05-20 20:20:03 -05:00
WladBlank bf33f4a7b0
add command shell blacklist and whitelist (#3950)
Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: Richard Beales <rich@richbeales.net>
2023-05-19 18:50:43 +01:00
k-boikov 2cd835e5a4
Added custom_search_engine_id as mandatory for official google search (#4228)
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
2023-05-16 22:20:02 -05:00
Kory Becker f424fac1d8
Huggingface retry generate_image with delay (#2745)
Co-authored-by: Media <12145726+rihp@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: Nicholas Tindle <nicktindle@outlook.com>
Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
Co-authored-by; lc0rp
2023-05-16 10:02:55 -05:00
Boostrix 3c80e05e00
Improve error message by providing the name of the env file (#3964)
Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
2023-05-16 02:26:13 +03:00
k-boikov cf35112291
Fixed error in safe_google_results, added tests (#3606) 2023-05-15 20:54:40 +01:00
k-boikov d06d8a609f
Fix commands with same name overwriting (#4226) 2023-05-15 19:41:11 +01:00
Abdelkarim Habouch b958386689
Add Edge browser support using EdgeChromiumDriverManager (#3058)
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
2023-05-14 00:28:30 +03:00
bszollosinagy ea08050049
BUGFIX: Selenium Driver object reference was included in the browsing results for some reason (#3642)
* * there is really no need to return the  reference to the Selenium driver along with the text summary and list of links.

* * removing unused second return value from browse_website()

* * updated cassette

* * updated YAML cassette for test_browse_website

* * after requirements reinstall, another update YAML cassette for test_browse_website

* * another update YAML cassette for test_browse_website, only as a placholder commit to trigger re-testing due to some docker TCP timeout issue

* * another update YAML cassette for test_browse_website

---------

Co-authored-by: batyu <batyu@localhost>
2023-05-04 23:11:21 +01:00
Nicholas Tindle d74428057e
Move task_complete command out of prompt (#3663)
* feat: move task_complete command out of prompt

* fix: formatting fixes

* Add the shutdown command to the test agents

* tests: update test vcrs

---------

Co-authored-by: James Collins <collijk@uw.edu>
2023-05-03 16:40:49 -07:00
sidewaysthought a5f856328d
Fix multi-byte character handling in `read_file` (#3173)
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-05-01 19:50:50 +02:00
non-adjective 7fc6f2abfc
update web_selenium.py to use try-with for headers (#2988)
Co-authored-by: Richard Beales <rich@richbeales.net>
2023-05-01 16:45:52 +01:00
Bob 94ec4a4ea5
Fix file operations logger (#3489)
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-05-01 17:37:30 +02:00
Richard Beales c1329c92fd
rename search_files to list_files (#3595) 2023-04-30 16:14:53 -05:00
Richard Beales 06ae4684c8
replace 50+ occurrences of print() with logger (#3056)
Co-authored-by: James Collins <collijk@uw.edu>
Co-authored-by: Luke Kyohere <lkyohere@mfsafrica.com>
Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
Co-authored-by: Media <12145726+rihp@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
2023-04-29 23:40:57 -05:00
k-boikov 2d058feaf8
Extend & improve file operations tests (#3404)
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-04-29 16:55:47 +02:00
Steven Baumann 9c6494aca7
Fix `clone_repository` to conform to URL validation (#3150)
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-04-29 14:57:48 +02:00
James Collins 92009ceb32
More graceful browsing error handling (#3494) 2023-04-28 22:12:47 +01:00
James Collins 3b74d2150e
Organize all the llm stuff into a subpackage (#3436) 2023-04-28 12:00:54 -07:00
rickythefox 20ef130341
Add tests for code/shell execution & improve config fixture (#1268)
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-04-28 14:51:29 +02:00
Irmius 6b4ad1f933
Fix `browse_website` headless mode for Firefox (#2816)
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-04-27 19:32:31 +02:00
Montana Flynn 7cd76b8d8e
Add makedirs to file operations (#3289)
* Add makedirs to file operations

* Add new directory tests for file operations

* Fix wrong setUp test error

* Simplify makedirs and use correct nested path

* Fix linter error

---------

Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: James Collins <collijk@uw.edu>
2023-04-27 09:12:24 -07:00
BillSchumacher d8c16de123
The unlooping and fixing of file execution. (#3368)
* The unlooping and fixing of file execution.

* lint

* Use static random seed during testing. remove unused import.

* Fix bug

* Actually fix bug.

* lint

* Unloop a bit more an fix json.

* Fix another bug.

* lint.

---------

Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
2023-04-26 21:07:28 -07:00
James Collins eec86a7b82
Load .env in package init (#3251) 2023-04-25 14:53:13 -07:00
Peter Svensson fac8f7da21
adding probably erroneously removed return value from execut_shell, giving 'None' in return always otherise - not ideal (#3212)
Co-authored-by: James Collins <collijk@uw.edu>
2023-04-25 13:32:39 -07:00
James Collins f962939737
Use explicit API keys when querying openai rather than import time manipulation of the package attributes (#3241) 2023-04-25 11:38:06 -07:00
Reinier van der Leer 9c60eecce6
Improve docker setup & config (#1843)
* Improve docker setup & config

* fix(browsing): Selenium needs access to home directory

* fix(docker): allow overriding memory backend settings

* simplify Dockerfile and docker-compose config

* add .dockerignore

* adjust Docker CI with release build type arg

* replace Chrome by Chromium in devcontainer

* update docs

* update bulletin

* use preinstalled chromedriver in web_selenium.py

* update installation.md

* fix code blocks for mkdocs

* fix links to docs
2023-04-24 14:27:53 +01:00
Eddie Cohen 40a75c804c
Validate URLs in web commands before execution (#2616)
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-04-24 12:33:44 +02:00
James Collins 43fa67ca81 Remove unnecessary memory call 2023-04-23 16:54:32 -07:00
James Collins dcd6aa912b
Add workspace abstraction (#2982)
* Add workspace abstraction

* Remove old workspace implementation

* Extract path resolution to a helper function

* Add api key requirements to new tests
2023-04-23 14:36:04 -05:00
Richard Beales 0efbe23d89
Merge pull request #2756 from gklab/master
adjust file_operations.py code format
2023-04-22 10:24:54 +01:00