Feat: New plugin routing feature

[terylt]

Plugin Routing with Declarative Rules

This document describes the declarative rule-based plugin routing system. Instead of nesting plugins under gateway→servers→tools hierarchy, we define flat routing rules that specify what entities plugins apply to using exact matches (fast path) and complex expressions (flexible path).

Key Features

• Fast exact matching via name and tags (hash lookups, no expression evaluation)
• Flexible filtering via when expressions for complex logic (regex, metadata, compound conditions)
• Implicit priority using list position (no explicit priorities needed for simple cases)
• Configurable merge strategy for handling overlapping rules (most_specific or merge_all)
• Symmetric wrapping via reverse_order_on_post for cleaner pre/post hook pairs
• Infrastructure filtering via server_name, server_id, gateway_id
• Two-level caching - static resolution (cached) + runtime filtering (per-request when clauses)

Configuration Structure

Plugins Section

Define plugins once with their base configuration. These are templates that can be overridden in hook rules.

plugins: - name: pii_filter kind: filter version: 1.0.0 author: security-team hooks: [tool_pre_invoke, tool_post_invoke] mode: enforce config: redaction_char: "*" log_redactions: true metadata: category: security compliance: [GDPR, CCPA, HIPAA] owner: security-team@company.com documentation: https://wiki.company.com/plugins/pii-filter - name: audit_logger kind: observability version: 1.0.0 author: compliance-team hooks: [tool_pre_invoke, tool_post_invoke, prompt_pre_invoke] mode: permissive metadata: category: observability retention_default: 90 - name: path_sanitizer kind: security version: 1.0.0 author: security-team hooks: [tool_pre_invoke] mode: enforce - name: rate_limiter kind: protection version: 1.0.0 author: ops-team hooks: [tool_pre_invoke] mode: enforce config: max_requests: 100 window_seconds: 60

Routes Section

Define routing rules for when and how plugins attach to entities. Enable routing with plugin_settings.enable_plugin_routing: true.

plugin_settings: enable_plugin_routing: true rule_merge_strategy: "most_specific" # or "merge_all"

Basic Examples

Tag-Based Matching

routes: # Apply PII filter to any tool with 'customer' or 'pii' tag - entities: [tool] tags: [customer, pii] reverse_order_on_post: true plugins: - name: pii_filter apply_to: fields: ["args.email", "args.ssn", "args.phone"] - name: audit_logger

Exact Name Match

routes: # Critical payment tool gets extra validation - entities: [tool] name: process_payment plugins: - name: payment_validator - name: fraud_detector config: threshold: 0.95 # Override default config - name: transaction_logger

Multiple Names or Tags

routes: - entities: [tool] name: [create_user, update_user, delete_user] plugins: - name: user_validator - name: audit_logger - entities: [tool, resource] tags: [sensitive] reverse_order_on_post: true plugins: - name: encryption_filter - name: audit_logger

Catch-All Rules

routes: # Apply to ALL tools (no name/tags filter) - entities: [tool] plugins: - name: general_tracker - name: performance_monitor # Apply to ALL prompts - entities: [prompt] plugins: - name: prompt_sanitizer

Advanced Filtering

Complex when Expressions

routes: # Combine tags with runtime conditions - entities: [tool] tags: [database, write-operation] when: "metadata.get('transaction_required') == true and server_name != 'read-replica'" plugins: - name: transaction_wrapper # Regex pattern matching - entities: [tool] when: "re.match(r'^(create_|update_)', name)" plugins: - name: input_validator # Resource URI filtering - entities: [resource] when: "payload.uri.endswith('.env') or payload.uri.endswith('.secrets')" plugins: - name: secret_redactor

HTTP-Level Plugins

routes: # Target specific HTTP hooks - hooks: [http_pre_request] plugins: - name: global_auth - name: request_id_injector # Combine hook filter with when clause - hooks: [http_pre_request] when: "payload.method == 'POST'" plugins: - name: rate_limiter config: max_requests: 100 window_seconds: 60 # Admin path protection - hooks: [http_pre_request] when: "payload.path.startswith('/admin/')" plugins: - name: admin_auth_checker - name: audit_logger

Hook Type Filtering

Target specific hooks for fine-grained control:

routes: # Only run on pre-invoke hooks - entities: [tool] hooks: [tool_pre_invoke] tags: [customer] plugins: - name: input_validator # Only run on post-invoke hooks - entities: [tool] hooks: [tool_post_invoke] tags: [customer] plugins: - name: response_sanitizer # Run on both pre and post hooks - entities: [tool] hooks: [tool_pre_invoke, tool_post_invoke] tags: [audit] plugins: - name: audit_logger

Implicit Priority

Plugins without explicit priority are auto-assigned priorities based on their list position (0, 1, 2, ...). This eliminates the need for explicit priorities in simple cases.

routes: - entities: [tool] tags: [api] plugins: - name: auth_check # priority: 0 (auto-assigned) - name: rate_limiter # priority: 1 (auto-assigned) - name: validator # priority: 2 (auto-assigned)

Explicit priorities override auto-assignment:

routes: - entities: [tool] tags: [critical] plugins: - name: circuit_breaker priority: 5 # explicit - name: validator # priority: 1 (auto-assigned) - name: audit_logger priority: 10 # explicit

Execution order: validator (1) → circuit_breaker (5) → audit_logger (10)

Infrastructure Filtering

Target specific infrastructure layers using server_name, server_id, or gateway_id filters. These are cached as part of the static resolution for performance.

routes: # Server-specific rules - entities: [tool] server_name: production-api plugins: - name: prod_rate_limiter - entities: [tool] server_name: [api-prod, api-staging] tags: [pii] plugins: - name: pii_filter # Gateway-specific rules - entities: [tool] gateway_id: gateway-us-east tags: [customer] plugins: - name: us_compliance_checker # Combined infrastructure + entity filtering - entities: [tool] tags: [customer] server_name: prod-api gateway_id: gateway-us-east plugins: - name: us_customer_compliance

HTTP-level vs Entity-level: Rules without entities apply at HTTP-level (before entity resolution). The system automatically filters plugins based on which hooks they support.

Rule Merging Strategies

When multiple rules match the same entity, the rule_merge_strategy setting controls how plugins are combined.

Strategy: most_specific (Default)

Only the most specific matching rules contribute plugins. Less specific rules are ignored.

Specificity scoring:

• Exact name match: 1000
• Tag match: 100
• Hook type filter: 50
• when expression: 10
• Entity type only: 0
• Infrastructure filters (server_name, etc.): add to base score

plugin_settings: rule_merge_strategy: "most_specific" # default routes: # Specificity: 0 (entity type only) - entities: [tool] plugins: - name: general_tracker # Specificity: 100 (tag match) - entities: [tool] tags: [customer] plugins: - name: pii_filter # Specificity: 1000 (name match) - entities: [tool] name: create_customer plugins: - name: customer_validator

For tool "create_customer" with tag "customer":

• All three rules match
• Only rules with highest specificity (1000) contribute
• Result: customer_validator only
• General and tag-based rules are excluded

Use case: Specific rules completely override general rules (simpler mental model, prevents plugin duplication).

Strategy: merge_all

All matching rules contribute plugins. Rules are ordered by explicit priority, then by specificity. Plugins are collected from all rules and sorted by plugin priority.

plugin_settings: rule_merge_strategy: "merge_all" routes: - entities: [tool] priority: 300 plugins: - name: general_tracker priority: 30 - entities: [tool] tags: [customer] priority: 200 plugins: - name: pii_filter priority: 20 - entities: [tool] name: create_customer priority: 100 plugins: - name: customer_validator priority: 10

For tool "create_customer" with tag "customer":

• All three rules match
• Rules ordered by priority: 100 → 200 → 300
• Plugins collected and sorted by plugin priority
• Result: customer_validator (10) → pii_filter (20) → general_tracker (30)

Use case: Layering plugins from multiple rules (e.g., base security + tag-specific + name-specific).

Multiple Instances with Different Configs

The same plugin can appear multiple times with different configurations, creating separate instances:

routes: - entities: [tool] tags: [api] plugins: - name: rate_limiter config: max_requests: 100 - entities: [tool] name: high_volume_api plugins: - name: rate_limiter config: max_requests: 1000

For tool "high_volume_api" with tag "api" (using merge_all):

• Two instances of rate_limiter are created (different config hashes)
• Both execute with their respective configs

Expression Context for when Clauses

The when clause has access to the following attributes and metadata:

Entity Context

• name (str): Entity name (e.g., tool name, prompt name, resource URI)
• entity_type (str): Entity type (tool, prompt, resource, agent, server, gateway)
• entity_id (str | None): Optional entity ID
• tags (list[str]): Entity tags
• metadata (dict): Entity metadata
• entity (dict): Complete entity object with fields:
  • entity.name: Same as name
  • entity.type: Same as entity_type
  • entity.id: Same as entity_id
  • entity.tags: Same as tags
  • entity.metadata: Same as metadata

Infrastructure Context

• server_name (str | None): Server name
• server_id (str | None): Server ID
• gateway_id (str | None): Gateway ID

Request Context

• args (dict): Request arguments (convenience accessor for payload.args)
• payload (dict): Complete request payload (varies by hook type - see below)
• user (str | None): User making the request
• tenant_id (str | None): Tenant ID
• agent (str | None): Agent identifier

Payload Fields by Hook Type

Tool hooks (tool_pre_invoke, tool_post_invoke):

• payload.name: Tool name
• payload.args: Tool arguments
• payload.headers: Request headers
• payload.result: Tool result (post-invoke only)

Prompt hooks (prompt_pre_invoke, prompt_post_invoke):

• payload.prompt_id: Prompt ID
• payload.args: Prompt arguments
• payload.result: Prompt result (post-invoke only)

Resource hooks (resource_pre_fetch, resource_post_fetch):

• payload.uri: Resource URI
• payload.metadata: Resource metadata
• payload.content: Resource content (post-fetch only)

Agent hooks (agent_pre_invoke, agent_post_invoke):

• payload.agent_id: Agent ID
• payload.args: Agent arguments
• payload.result: Agent result (post-invoke only)

HTTP hooks (http_pre_request, http_post_request):

• payload.method: HTTP method (GET, POST, etc.)
• payload.path: Request path
• payload.headers: Request headers
• payload.client_host: Client IP/host
• payload.query_params: Query parameters

Available Python Modules

• re: Regular expression module for pattern matching

Example Expressions

routes: # Entity context - entities: [tool] when: "name.startswith('create_') or name.startswith('update_')" plugins: [validator] # Tags and metadata - entities: [tool] when: "'pii' in tags and metadata.get('risk_level') == 'high'" plugins: [pii_filter] # Infrastructure context - entities: [tool] when: "server_name in ['prod-api', 'staging-api'] and gateway_id == 'us-east'" plugins: [regional_compliance] # Request arguments - entities: [tool] when: "args.get('size', 0) > 1000" plugins: [size_validator] # Payload fields - entities: [resource] when: "payload.uri.endswith('.env') or payload.uri.endswith('.secrets')" plugins: [secret_redactor] # HTTP payload - when: "payload.method == 'POST' and payload.path.startswith('/admin/')" plugins: [admin_auth] # Regex matching - entities: [tool] when: "re.match(r'^(create_|update_|delete_)', name)" plugins: [mutation_logger] # Complex conditions - entities: [tool] when: "'customer' in tags and args.get('email') and server_name == 'prod-api'" plugins: [customer_compliance]

Two-Level Caching

The system uses a two-level caching strategy for performance:

1. Static resolution (cached): Match rules by name, tags, and infrastructure filters (server_name, server_id, gateway_id). Cache key: (entity_type, entity_name, hook_type, server_name, server_id, gateway_id).

2. Runtime filtering (per-request): Evaluate when clauses with request context. Not cached - evaluated for each request.

This means static rules are resolved once and cached, but when clauses are evaluated on every request for maximum flexibility.

Corner Cases

Catch-All Rules

Rules with only entities (no name/tags/when) match all entities of that type:

routes: - entities: [tool] # Matches ALL tools plugins: - name: general_monitor

HTTP-Level Rules

Rules without entities apply at HTTP level and must have a when clause or infrastructure filter:

routes: # Valid: has 'when' clause - when: "payload.method == 'POST'" plugins: - name: post_handler # Valid: has gateway filter - gateway_id: gateway-prod plugins: - name: prod_auth # Invalid: no criteria at all - plugins: - name: some_plugin # ERROR: HTTP-level rules need 'when' or infrastructure filter

Symmetric Wrapping with reverse_order_on_post

For pre/post hook pairs, use reverse_order_on_post: true to automatically reverse plugin order on post-hooks:

routes: - entities: [tool] reverse_order_on_post: true plugins: - name: logger # Pre: first, Post: last - name: validator # Pre: second, Post: second - name: transformer # Pre: third, Post: first

Pre-hook order: logger → validator → transformer
Post-hook order: transformer → validator → logger (reversed)

This creates symmetric wrapping where the first pre-hook is the last post-hook.

Key Implementation Details

• Implicit priority: Plugins without explicit priority are auto-assigned based on list position (0, 1, 2, ...)
• Hook type filtering: Use hooks field to target specific hook types (e.g., [tool_pre_invoke], [http_pre_request])
• Fast path optimization: name, tags, and hooks use hash lookups; when expressions evaluated only at runtime
• Infrastructure filtering cached: server_name, server_id, gateway_id are part of static cache key
• Multiple instances: Same plugin with different configs creates separate instances (keyed by config hash)
• Expression context: when clauses have access to entity context (name, tags, metadata), infrastructure context (server_name, gateway_id), request context (args, payload, user), and Python modules (re for regex)
• Plugin configs are optional: Only specify when overriding defaults from plugin definition