Skip to content

Understanding Agent Executor Interceptors

Jump to bottom
Pavel Buchnev edited this page Sep 19, 2024 · 1 revision

Feature Explanation

The Agent Executor Interceptors feature is a crucial component of the LLM (Language Model) Agents system. It implements a flexible pipeline architecture that allows for modifying the execution flow of an agent's task. This feature enables developers to add pre-processing, post-processing, or alter the behavior of the executor without changing its core implementation.

The interceptors work on the principle of a chain of responsibility pattern. Each interceptor in the pipeline can perform operations on the execution input, decide whether to pass the execution to the next interceptor, or short-circuit the chain and return a result immediately.

This system is particularly useful for tasks such as:

  • Logging
  • Modifying context or options
  • Implementing complex execution strategies
  • Error handling
  • Performance monitoring

Use Cases

1. Automated Logging System

  • Implement a logging interceptor to automatically log all agent interactions.
final readonly class LoggingInterceptor implements ExecutorInterceptorInterface
{
    public function __construct(
        private LoggerInterface $logger
    ) {}

    public function execute(ExecutionInput $input, InterceptorHandler $next): Execution
    {
        $this->logger->info('Executing agent: ' . $input->agent);
        
        $startTime = microtime(true);
        $execution = $next($input);
        $duration = microtime(true) - $startTime;
        
        $this->logger->info('Execution completed', [
            'agent' => $input->agent,
            'duration' => $duration,
            'resultType' => $execution->result::class,
        ]);
        
        return $execution;
    }
}

2. Dynamic Model Selection

  • Create an interceptor that selects the most appropriate language model based on the input complexity.
final class DynamicModelSelector implements ExecutorInterceptorInterface
{
    public function execute(ExecutionInput $input, InterceptorHandler $next): Execution
    {
        $complexity = $this->analyzeComplexity($input->prompt);
        $model = match (true) {
            $complexity > 0.8 => OpenAIModel::Gpt4o,
            $complexity > 0.5 => OpenAIModel::Gpt4oMini,
            default => OpenAIModel::Gpt3Turbo,
        };
        
        $newInput = $input->withOptions($input->options->withModel($model));
        return $next($newInput);
    }
    
    private function analyzeComplexity(string|\Stringable $prompt): float
    {
        // Implement complexity analysis logic here
    }
}

3. Rate Limiting

  • Implement a rate-limiting interceptor to prevent overuse of the LLM API.
final class RateLimitInterceptor implements ExecutorInterceptorInterface
{
    public function __construct(
        private RateLimiterInterface $limiter
    ) {}

    public function execute(ExecutionInput $input, InterceptorHandler $next): Execution
    {
        if (!$this->limiter->consume(1)->isAccepted()) {
            throw new RateLimitException('API rate limit exceeded');
        }
        
        return $next($input);
    }
}

4. Context Enrichment

  • Create an interceptor that enriches the execution context with additional information.
final class ContextEnrichmentInterceptor implements ExecutorInterceptorInterface
{
    public function __construct(
        private UserRepositoryInterface $userRepository,
        private PreferencesServiceInterface $preferencesService
    ) {}

    public function execute(ExecutionInput $input, InterceptorHandler $next): Execution
    {
        $user = $this->userRepository->findByUuid($input->context->getUserUuid());
        $preferences = $this->preferencesService->getForUser($user);
        
        $enrichedContext = $input->context->withUserPreferences($preferences);
        $newInput = $input->withContext($enrichedContext);
        
        return $next($newInput);
    }
}

5. Error Recovery

  • Implement an interceptor that attempts to recover from certain types of errors.
final class ErrorRecoveryInterceptor implements ExecutorInterceptorInterface
{
    public function execute(ExecutionInput $input, InterceptorHandler $next): Execution
    {
        try {
            return $next($input);
        } catch (TokenLimitExceededException $e) {
            $newInput = $input->withOptions(
                $input->options->with(Option::MaxTokens, $e->currentLimit + 500)
            );
            return $next($newInput);
        } catch (TimeoutException) {
            sleep(5);
            return $next($input);
        }
    }
}

Configuration Options

  • Interceptor Order: The order in which interceptors are added to the pipeline is significant.

    • Option Name: withInterceptor

      • Default Value: [] (empty array)
      • Description: Adds one or more interceptors to the executor's pipeline.
      $executor = $executor->withInterceptor(
    new LoggingInterceptor($logger),
    new RateLimitInterceptor($limiter),
    new ErrorRecoveryInterceptor()
);

  • Execution Options: Various options can be set for each execution.

    • Option Name: withOptions

      • Default Value: Depends on the specific option
      • Description: Sets options for the execution, such as model, temperature, max tokens, etc.
      $input = $input->withOptions(
    $input->options
        ->withModel(OpenAIModel::Gpt4oMini)
        ->withTemperature(0.7)
        ->withMaxTokens(150)
);

Related Classes

ExecutorInterface

  • Purpose and Functionality: Defines the contract for agent executors.
  • Interaction with Main Feature: Provides the main method execute that the interceptors wrap around.

ExecutionInput

  • Purpose and Functionality: Represents the immutable input for an agent execution.
  • Interaction with Main Feature: Passed through the interceptor chain, allowing modifications at each step.

Execution

  • Purpose and Functionality: Represents the result of an agent's execution.
  • Interaction with Main Feature: Returned by the execute method and can be modified by interceptors.

InterceptorHandler

  • Purpose and Functionality: Handles the invocation of the next interceptor in the chain.
  • Interaction with Main Feature: Passed to each interceptor to allow continuation of the chain.

ExecutorPipeline

  • Purpose and Functionality: Implements the execution pipeline, managing the flow through interceptors.
  • Interaction with Main Feature: Core class that orchestrates the interceptor chain.

Mermaid Class Diagram

classDiagram
    class ExecutorInterface {
        +execute(agent: string, prompt: string|Stringable|Prompt, context: ContextInterface, options: OptionsInterface, promptContext: PromptContextInterface) Execution
        +withInterceptor(interceptor: ExecutorInterceptorInterface) ExecutorInterface
    }
    class ExecutorInterceptorInterface {
        +execute(input: ExecutionInput, next: InterceptorHandler) Execution
    }
    class ExecutionInput {
        +agent: string
        +prompt: string|Stringable|PromptInterface
        +context: ContextInterface
        +options: OptionsInterface
        +promptContext: PromptContextInterface
    }
    class Execution {
        +result: Response
        +prompt: PromptInterface
    }
    class InterceptorHandler {
        -executor: ExecutorInterface
        +__invoke(input: ExecutionInput) Execution
    }
    class ExecutorPipeline {
        -interceptors: ExecutorInterceptorInterface[]
        -offset: int
        +execute(agent: string, prompt: string|Stringable|Prompt, context: ContextInterface, options: OptionsInterface, promptContext: PromptContextInterface) Execution
        +withInterceptor(interceptor: ExecutorInterceptorInterface) ExecutorInterface
    }
    ExecutorInterface <|-- ExecutorPipeline
    ExecutorPipeline o-- ExecutorInterceptorInterface
    ExecutorInterceptorInterface ..> ExecutionInput
    ExecutorInterceptorInterface ..> Execution
    ExecutorInterceptorInterface ..> InterceptorHandler
    InterceptorHandler o-- ExecutorInterface
Loading

This class diagram illustrates the relationships between the main components of the Agent Executor Interceptors feature. The ExecutorPipeline implements the ExecutorInterface and manages a collection of ExecutorInterceptorInterface objects. Each interceptor works with ExecutionInput and Execution objects, using the InterceptorHandler to manage the flow of execution through the pipeline.

Clone this wiki locally