A new property named _isWaitingForAgentResponse has been introduced to the ChatViewModel class to track the loading state when waiting for the agent's response. This boolean property is set to true when a chat message is being sent and reverts to false upon completion, whether successful or not. The notifyListeners() method is invoked to update the UI accordingly. This enhancement facilitates the display of a loading indicator, offering users visual feedback during the wait time for agent responses.
Implemented a new LoadingIndicator widget that displays a pulsating gradient line, giving users visual feedback during asynchronous operations such as API calls. The indicator animates a gradient from white to AppColors.primaryLight and back to the grey background color, signifying the loading state.
- Enhanced the `Chat` model to include an `artifacts` field to hold associated artifacts.
- Updated the `AgentMessageTile` widget to display the number of artifacts and added functionality to trigger artifact downloads upon button press.
- Introduced a method in `ChatViewModel` to leverage the `ChatService` for artifact downloads.
This commit updates the Leaderboard Submission Dialog to pass the team name, repository URL, and commit SHA as parameters to the submitToLeaderboard function in the SkillTreeViewModel. These changes ensure that the dialog and the ViewModel are aligned in terms of parameter requirements, facilitating efficient and accurate leaderboard submissions.
This commit incorporates significant enhancements to the SkillTreeViewModel, introducing the ability to track current benchmark runs and submit results to the leaderboard. A new list, `currentBenchmarkRuns`, is introduced to store each benchmark run object during a specific benchmark session. This list is reset to an empty state when initiating a new benchmark.
Changes made:
- Introduced `currentBenchmarkRuns` to track ongoing benchmark runs, ensuring real-time data availability.
- Enhanced `runBenchmark` method to populate `currentBenchmarkRuns` with benchmark run objects as the benchmark progresses.
- Implemented `submitToLeaderboard` method, accepting parameters `teamName`, `repoUrl`, and `agentGitCommitSha`, and updating each run object with this information. All runs share a common UUID generated at the beginning of the submission process.
These enhancements ensure that benchmark run data is readily available and organized, facilitating a streamlined process for submitting well-structured data to the leaderboard. It fosters a more interactive and informative user experience, offering insights into each benchmark run's progress and outcomes.
This commit integrates the newly created UriUtility class into the LeaderboardSubmissionDialog. The isURL method from UriUtility is used to add an additional layer of validation for the GitHub repository URL input field. It ensures that users enter a valid URL format before submitting their leaderboard entries.
Changes made:
- Integrated UriUtility’s isURL method in the _validateAndSubmit function of the LeaderboardSubmissionDialog.
- Added a specific error message "Invalid URL format" to inform users when the entered URL does not meet the validation criteria.
- Updated the state management to reflect the URL validation status and re-render the dialog with appropriate error messages when necessary.
With this integration, the application now provides real-time feedback on the validity of the entered URL, enhancing user experience and data integrity by ensuring that only valid URLs are submitted to the leaderboard.
This commit introduces the UriUtility class to the codebase. This utility class contains a static method, isURL, used for validating URLs. The method checks for common URL validation criteria, ensuring that the input string adheres to the standard URL format. It verifies that the URL is non-empty, does not contain invalid characters, is not a mailto link, can be parsed as a URI, and has both a scheme and a host. Additional validations include checks on user info and port numbers.
Changes made:
- Created UriUtility class with isURL static method for URL validation.
- Included validations for empty strings, invalid characters, mailto links, scheme, host, user info, and port numbers.
- The utility will be utilized in forms and other areas where URL validation is necessary.
This enhancement improves the robustness of URL validation across the application, ensuring that only valid URLs are processed and stored.
Enhanced the RestApiUtility class to support HTTP PUT requests. The new method, `put`, allows for updating data at a specified endpoint. This addition provides more comprehensive CRUD operations for the API interactions, ensuring the client can perform updates when necessary.
Refactored the TaskService and TaskViewModel classes to fully support paginated fetching of tasks. The solution fetches tasks page by page until all tasks have been retrieved, accommodating the API's pagination mechanism. Renamed `listAllTasks` to `fetchTasksPage` to better reflect its purpose and introduced a new method, `fetchAllTasks`, to handle the paginated fetching logic.
Integrated both the TestSuiteButton and LeaderboardSubmissionButton in the TaskQueueView. Added conditions to disable the LeaderboardSubmissionButton based on benchmark running status and if the benchmarkStatusMap is empty. The onPressed logic for both buttons has been integrated as per requirements.
Added a new LeaderboardSubmissionButton widget to facilitate user submissions to the leaderboard. The button is styled consistently with other application buttons and features an emoji trophy icon to indicate leaderboard submissions. It provides a mechanism for disabling submissions based on certain conditions.
Introduced a new reusable TestSuiteButton widget. It provides flexibility for positioning and is intended for initiating the test suite in the TaskQueueView.
Introduce the LeaderboardSubmissionDialog widget which allows users to submit their team name, GitHub repo URL, and commit SHA for leaderboard consideration.
Features:
- Field validation: Ensure that all required fields are filled before submission.
- SharedPreferences integration: Save the user's last submission values and auto-populate the fields with these values when the dialog is shown next time.
This enhancement provides a smoother user experience by reducing repetitive data entry for subsequent leaderboard submissions.
- Introduced a new dialog, ContinuousModeDialog, to inform users about the repercussions of the continuous mode.
- Integrated ContinuousModeDialog with ChatInputField. The dialog is shown when the fast-forward icon is clicked, based on the user's shared preferences.
- Leveraged shared preferences to remember if the user has chosen not to see the dialog again.
- Enhanced the ChatInputField to handle different states and user interactions related to continuous mode.
- Introduced a new `SkillTreeType` enum to represent different skill tree categories: General, Coding, Data, and Scrape/Synthesize.
- Extended the `SkillTreeType` enum to provide associated string values and JSON file names for each category.
- Refactored the `SkillTreeViewModel` to reload the skill tree data based on the selected category.
- Enhanced `SkillTreeView` by adding a positioned dropdown in the top-left corner to allow users to select and load different skill tree categories dynamically.
Introduced a `AppColors` class to house all color constants provided by the design team. This class segregates colors into light and dark modes, ensuring easy and consistent theming throughout the application. The centralized approach simplifies updates and ensures color consistency across the app.
Added a FeatureFlags utility class to manage feature toggling within the application. This initial commit introduces a flag for the userExperienceIterationTwo feature, allowing for controlled testing and phased rollout as development progresses. The flag can be manually adjusted as needed.
- Deprecated `ApiSettingsViewModel` in favor of enhancing `SettingsViewModel`.
- Moved the API Base URL field from `TaskView` to `SettingsView` to centralize configuration.
- Integrated `RestApiUtility` dependency into `SettingsViewModel` to ensure consistent URL management across the app.
Refactored the ChatViewModel's sendChatMessage method to utilize the continuousModeSteps setting from the SettingsViewModel. This allows the continuous chat mode to honor the user's preference from the settings. Also introduced an optional currentStep parameter with a default value, enabling better control over the progression of continuous mode.
Changed the architecture of the SettingsView to align with the existing pattern in the application. Instead of creating its own instance of SettingsViewModel, the view now accepts an external viewModel as a parameter. This ensures consistency across the application and allows for better modularity and reusability of components.
This commit enriches the `SettingsViewModel` by integrating state persistence functionality and a sign-out method, thus ensuring a more robust and user-friendly settings feature.
Key Enhancements:
1. **State Persistence**:
- Leveraging the `shared_preferences` package, the app now stores and retrieves user preferences related to app settings, ensuring consistency across app restarts.
- Preferences like Dark Mode, Developer Mode, Base URL, and Continuous Mode Steps are persistently stored and loaded when the ViewModel is initialized.
2. **Sign Out Method**:
- A `signOut` method has been introduced in the ViewModel, utilizing the `AuthService` to facilitate user sign-out processes.
- This addition allows for seamless integration of sign-out functionality within the settings interface, granting users the ability to easily terminate sessions.
3. **SettingsView Enhancement**:
- The `SettingsView` has been adapted to incorporate a UI element invoking the `signOut` method, enhancing user interaction within the settings environment.
This enhancement not only bolsters the resilience and usability of the app’s settings but also lays down a structured approach for potential future additions to user preferences and settings-related functionalities.
This commit integrates the newly created `SettingsView` into the `MainLayout`, allowing users to access and interact with the settings through the main user interface of the app. When the user selects the settings tab from the `SideBarView`, the `SettingsView` is displayed alongside the `ChatView`, maintaining consistent layout aesthetics with other views like `TaskView`.
Key Changes:
- A new condition has been added in the `ValueListenableBuilder` within the `MainLayout` to check if the selected view is `'SettingsView'`.
- When `'SettingsView'` is selected, `SettingsView` is rendered with the same width as `TaskView`, and `ChatView` is displayed next to it.
- The state of the skill tree is reset when navigating to `SettingsView` to ensure a clean state.
This integration ensures a seamless user experience, allowing users to navigate and configure app settings easily and efficiently from the main layout of the app.
This commit augments the `SideBarView` by introducing a new `IconButton` that represents a tab for the newly created `SettingsView`. This enhancement allows users to navigate to the settings page directly from the sidebar, improving accessibility to app configuration options.
Key Changes:
- A new `IconButton` with a settings icon has been added to the sidebar’s `Column` widget.
- Selecting the settings tab updates the `selectedViewNotifier.value` to `'SettingsView'`, indicating the user's intent to navigate to the settings page.
- Visual feedback is provided by altering the color of the settings icon based on whether the settings tab is currently selected.
This addition ensures seamless integration of the `SettingsView` into the existing navigation structure, enabling users to easily configure app settings.
This commit refines the `populateSelectedNodeHierarchy` method in the `SkillTreeViewModel` to accurately represent nodes that possess multiple parent nodes, ensuring a comprehensive and non-redundant representation of the entire hierarchy leading back to the root nodes.
Modifications and Features:
- The method now employs recursion to traverse all possible paths back to the root nodes from a selected node, capturing every unique node in the hierarchies.
- A `Set` is utilized to monitor and ensure that each node is only added once to the `_selectedNodeHierarchy` list, eliminating the possibility of duplicates.
- The finalized `_selectedNodeHierarchy` list is constructed such that the root of the tree is the last item in the list, providing a more logical representation of the hierarchy.
These enhancements ensure a more accurate and efficient representation of the skill tree structure, particularly in scenarios where nodes have multiple parents, facilitating better navigation and interaction within the skill tree.
This commit introduces the `SettingsView` class, which is responsible for rendering the user interface for the settings feature of the app. This View is associated with the `SettingsViewModel` for state management and logic.
Features Represented in this View:
- Dark Mode: A switch to enable or disable Dark Mode.
- Developer Mode: A switch to enable or disable Developer Mode.
- Base URL: A text field to input and configure the Base URL.
- Continuous Mode Steps: User-friendly '+' and '-' buttons allowing users to configure the number of steps for continuous mode.
The `SettingsView` utilizes the Flutter `ListView` to display various settings options, providing an intuitive and user-friendly experience. It employs reactive state management using the `Provider` package, ensuring that any state changes in the associated ViewModel are reflected immediately in the View. Proper documentation and comments have been added for better code readability and maintenance.
This commit introduces the `SettingsViewModel` class, part of the MVVM architecture, responsible for managing the state and business logic for the settings feature of the app. This ViewModel is associated with the `SettingsView`.
Features Managed by this ViewModel:
- Dark Mode: Toggle the state of Dark Mode and notify listeners.
- Developer Mode: Toggle the state of Developer Mode and notify listeners.
- Base URL: Update the state of Base URL and notify listeners.
- Continuous Mode Steps: Increment and decrement the number of Continuous Mode Steps and notify listeners.
Each change in the state is followed by a notification to the listeners to rebuild the UI components that depend on this state, ensuring a reactive UI. To-do comments have been added where the state needs to be persisted or synchronized with a server.
This commit integrates the `LeaderboardService` into `SkillTreeViewModel` to enable benchmark report submissions to the leaderboard. A `BenchmarkRun` object is created from the evaluation response and submitted using the `submitReport` method from `LeaderboardService`.
This commit extends the `RestApiUtility` class to include support for a new leaderboard base URL. A new `ApiType` enum value `ApiType.leaderboard` has been added, and the `_getEffectiveBaseUrl` method has been updated to handle this new type. The leaderboard base URL is "https://leaderboard.vercel.app/".
This commit adds a new `LeaderboardService` class featuring a `submitReport` method. This method allows for the submission of `BenchmarkRun` objects to the leaderboard via a POST request to the `/api/reports` endpoint. The new service uses the `ApiType.leaderboard` enum value.
This commit introduces the `BenchmarkRun` class, designed to model a complete benchmark run. The class encapsulates all data and sub-models related to a benchmark, providing a centralized object to handle various aspects of a benchmark run.
The `BenchmarkRun` class includes the following sub-models:
- `RepositoryInfo`: Information about the repository and team.
- `RunDetails`: Specific details like the run identifier, command, and timings.
- `TaskInfo`: Information about the task being benchmarked.
- `Metrics`: Performance metrics for the benchmark run.
- `Config`: Configuration settings for the benchmark run.
A `reachedCutoff` field is also included to indicate whether a certain cutoff was reached during the benchmark run.
Methods for serializing and deserializing the object to and from JSON are also provided.
This commit introduces the RepositoryInfo class, designed to encapsulate details about the repository and team associated with a benchmark run.
The class includes the following fields:
- repoUrl: The URL of the repository where the benchmark code resides.
- teamName: The name of the team responsible for the benchmark.
- benchmarkGitCommitSha: The Git commit SHA for the benchmark code.
- agentGitCommitSha: The Git commit SHA for the agent code.
The class supports JSON serialization and deserialization, making it easy to use with Flutter's JSON handling mechanisms.
Added a new Dart class called `RunDetails` to represent specific details related to a benchmark run.
The class includes fields for:
- The unique run identifier (`runId`)
- The command used to initiate the benchmark (`command`)
- The time the benchmark was completed (`completionTime`)
- The time the benchmark started (`benchmarkStartTime`)
- The name of the test being run (`testName`)
Serialization and deserialization methods are also provided for JSON compatibility.
Added a new TaskInfo class to encapsulate information related to a specific benchmark task.
- The TaskInfo class holds attributes like the data file path, regression status, task categories, task details, expected answer, and description.
- Included methods for JSON serialization and deserialization.
- Added comprehensive documentation to describe the purpose, properties, and methods of the TaskInfo class.
Added a new Metrics class to represent key performance metrics of a benchmark test run.
- The Metrics class encapsulates various data points like difficulty, success rate, attempted status, success percentage, cost, and runtime.
- Included serialization and deserialization methods for converting between Metrics objects and JSON.
- Added comprehensive documentation to describe the purpose, properties, and methods of the Metrics class.
This commit introduces a new `Config` class, designed to manage and store configuration settings related to the benchmark run. The class contains two key fields:
1. `agentBenchmarkConfigPath`: The path to the agent's benchmark configuration file.
2. `host`: The address of the host where the benchmark is running.
The class includes methods for serialization and deserialization, allowing easy conversion between `Config` objects and JSON maps.
Documentation comments have also been added for better code readability and understanding.
* New benchmark data models
* Update _benchmarkBaseUrl
* Remove ReportRequestBody
* Update benchmark service methods for proxy approach
* Add eval id to SkillNodeData
* Refactor runBenchmark Method for proxy approach
Swifty
hunteraraujo
merwanehamadi