Commit Graph

1 Commits (master)

Author SHA1 Message Date
Swifty 243400e128
feat(platform): Add Block Development SDK with auto-registration system (#10074)
## Block Development SDK - Simplifying Block Creation

### Problem
Currently, creating a new block requires manual updates to **5+ files**
scattered across the codebase:
- `backend/data/block_cost_config.py` - Manually add block costs
- `backend/integrations/credentials_store.py` - Add default credentials
- `backend/integrations/providers.py` - Register new providers
- `backend/integrations/oauth/__init__.py` - Register OAuth handlers
- `backend/integrations/webhooks/__init__.py` - Register webhook
managers

This creates significant friction for developers, increases the chance
of configuration errors, and makes the platform difficult to scale.

### Solution
This PR introduces a **Block Development SDK** that provides:
- Single import for all block development needs: `from backend.sdk
import *`
- Automatic registration of all block configurations
- Zero external file modifications required
- Provider-based configuration with inheritance

### Changes 🏗️

#### 1. **New SDK Module** (`backend/sdk/`)
- **`__init__.py`**: Unified exports of 68+ block development components
- **`registry.py`**: Central auto-registration system for all block
configurations
- **`builder.py`**: `ProviderBuilder` class for fluent provider
configuration
- **`provider.py`**: Provider configuration management
- **`cost_integration.py`**: Automatic cost application system

#### 2. **Provider Builder Pattern**
```python
# Configure once, use everywhere
my_provider = (
    ProviderBuilder("my-service")
    .with_api_key("MY_SERVICE_API_KEY", "My Service API Key")
    .with_base_cost(5, BlockCostType.RUN)
    .build()
)
```

#### 3. **Automatic Cost System**
- Provider base costs automatically applied to all blocks using that
provider
- Override with `@cost` decorator for block-specific pricing
- Tiered pricing support with cost filters

#### 4. **Dynamic Provider Support**
- Modified `ProviderName` enum to accept any string via `_missing_`
method
- No more manual enum updates for new providers

#### 5. **Application Integration**
- Added `sync_all_provider_costs()` to `initialize_blocks()` for
automatic cost registration
- Maintains full backward compatibility with existing blocks

#### 6. **Comprehensive Examples** (`backend/blocks/examples/`)
- `simple_example_block.py` - Basic block structure
- `example_sdk_block.py` - Provider with credentials
- `cost_example_block.py` - Various cost patterns
- `advanced_provider_example.py` - Custom API clients
- `example_webhook_sdk_block.py` - Webhook configuration

#### 7. **Extensive Testing**
- 6 new test modules with 30+ test cases
- Integration tests for all SDK features
- Cost calculation verification
- Provider registration tests

### Before vs After

**Before SDK:**
```python
# 1. Multiple complex imports
from backend.data.block import Block, BlockCategory, BlockOutput
from backend.data.model import SchemaField, CredentialsField
# ... many more imports

# 2. Update block_cost_config.py
BLOCK_COSTS[MyBlock] = [BlockCost(...)]

# 3. Update credentials_store.py
DEFAULT_CREDENTIALS.append(...)

# 4. Update providers.py enum
# 5. Update oauth/__init__.py
# 6. Update webhooks/__init__.py
```

**After SDK:**
```python
from backend.sdk import *

# Everything configured in one place
my_provider = (
    ProviderBuilder("my-service")
    .with_api_key("MY_API_KEY", "My API Key")
    .with_base_cost(10, BlockCostType.RUN)
    .build()
)

class MyBlock(Block):
    class Input(BlockSchema):
        credentials: CredentialsMetaInput = my_provider.credentials_field()
        data: String = SchemaField(description="Input data")
    
    class Output(BlockSchema):
        result: String = SchemaField(description="Result")
    
    # That's it\! No external files to modify
```

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] Created new blocks using SDK pattern with provider configuration
  - [x] Verified automatic cost registration for provider-based blocks
  - [x] Tested cost override with @cost decorator
  - [x] Confirmed custom providers work without enum modifications
  - [x] Verified all example blocks execute correctly
  - [x] Tested backward compatibility with existing blocks
  - [x] Ran all SDK tests (30+ tests, all passing)
  - [x] Created blocks with credentials and verified authentication
  - [x] Tested webhook block configuration
  - [x] Verified application startup with auto-registration

#### For configuration changes:
- [x] `.env.example` is updated or already compatible with my changes
(no changes needed)
- [x] `docker-compose.yml` is updated or already compatible with my
changes (no changes needed)
- [x] I have included a list of my configuration changes in the PR
description (under **Changes**)

### Impact

- **Developer Experience**: Block creation time reduced from hours to
minutes
- **Maintainability**: All block configuration in one place
- **Scalability**: Support hundreds of blocks without enum updates
- **Type Safety**: Full IDE support with proper type hints
- **Testing**: Easier to test blocks in isolation

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Abhimanyu Yadav <122007096+Abhi1992002@users.noreply.github.com>
2025-07-10 16:17:55 +02:00