5-Agent Security & Quality Analysis System

Discover how 5 specialized AI agents work together to automatically analyze merge requests for security risks, exposed secrets, code quality, database performance, and dependency consistency.

Comment

Inspiration

I was inspired by the challenge of catching security vulnerabilities and code quality issues that slip through manual code reviews. The GitLab AI Agents platform opened up possibilities for creating specialized agents that work together intelligently.

I realized I could build a multi-agent system where each agent focuses on a specific security or quality concern—from detecting exposed credentials to validating database performance.

My goal: Create an automated, intelligent code review system that catches security risks, secrets, quality issues, and deployment problems BEFORE they reach production. This reduces manual review burden and prevents costly security incidents.

What it does

A multi-agent AI system that I built to automatically analyze merge requests across 5 specialized dimensions:

🔐 Secret Detection Agent

  • Detects exposed AWS credentials, GitHub tokens, private keys
  • Finds hardcoded database passwords, API keys, JWT tokens
  • Blocks merges with critical secrets
  • Provides immediate remediation steps

⚠️ MR Risk Agent

  • Calculates risk scores (0-100) for code changes
  • Identifies sensitive file modifications (CI/CD, Docker, Kubernetes)
  • Detects dependency changes and breaking changes
  • Flags auth/permission code modifications
  • Triggers approval gates for high-risk changes

✅ Test Coverage Agent

  • Analyzes code coverage for new changes
  • Detects coverage drops > 5%
  • Supports JavaScript, Python, Java, Go test patterns
  • Recommends missing tests

🗄️ Database Performance Agent

  • Detects MongoDB query performance issues
  • Validates Kubernetes configurations
  • Prevents OOMKill errors by checking resource allocation
  • Ensures deployment readiness

📦 Dependency Consistency Agent

  • Validates all imports are in package.json/requirements.txt
  • Checks environment variables are documented
  • Prevents hardcoded credentials
  • Ensures deployment configs match local setup

Key Features:

  • Conditional triggering based on file changes
  • Intelligent approval gates based on severity
  • Detailed findings with actionable recommendations
  • Automated labeling for issue tracking
  • Blocks merges with critical security issues

How we built it

I leveraged the GitLab AI Agents platform to create a sophisticated multi-agent architecture:

Architecture:

  1. 5 Specialized Agents - Each agent has a specific focus:

    • Secret Detection Agent (security)
    • MR Risk Agent (risk analysis)
    • Test Coverage Agent (quality)
    • Database Performance Agent (deployment)
    • Dependency Consistency Agent (dependencies)

  2. Intelligent Flow Orchestration - Multiple flows I designed to trigger conditionally:

    • Security & Secrets Scan Flow (ALWAYS triggers)
    • Code Quality & Coverage Flow (triggers on code changes)
    • Database & Deployment Validation Flow (triggers on DB/K8s changes)
    • Dependency & Environment Validation Flow (triggers on manifest changes)

  3. Pattern-Based Detection - Each agent uses:

    • Regex patterns for credential detection
    • Entropy-based analysis for high-entropy strings
    • Semantic code analysis for risk assessment
    • File change analysis for impact evaluation

  4. Approval Gate System - Risk-based merge blocking:

    • Low Risk (0-34): Informational labels only
    • Medium Risk (35-64): Labels + security review requested
    • High Risk (65+): Labels + approval gate + security review required
    • Critical Secrets: Automatic merge block

Tech Stack:

  • GitLab AI Agents (agent orchestration)
  • YAML configuration (agent definitions & flows)
  • Python & JavaScript (test patterns)
  • Regex & semantic analysis (pattern matching)
  • GitLab CI/CD integration (automated triggering)

Development Process:

  1. Designed agent specifications and detection patterns
  2. Created YAML configurations for each agent
  3. Built flow orchestration logic
  4. Implemented conditional triggering
  5. Created comprehensive test cases
  6. Validated against real-world scenarios

Challenges we ran into

  1. Pattern Matching Accuracy

    • Challenge: Distinguishing real secrets from test/dummy values
    • Solution: I implemented false positive filtering and context analysis
    • Added documentation markers for intentional test secrets

  2. Multi-Agent Coordination

    • Challenge: Ensuring agents don't conflict or duplicate analysis
    • Solution: I designed clear responsibility boundaries for each agent
    • Implemented conditional triggering to avoid unnecessary runs

  3. Performance at Scale

    • Challenge: Analyzing large merge requests with many file changes
    • Solution: I implemented file-type filtering and scope limiting
    • Set appropriate timeouts (180s per agent)

  4. False Positives in Risk Scoring

    • Challenge: Avoiding over-flagging legitimate code changes
    • Solution: I weighted risk factors based on actual impact
    • Implemented threshold-based approval gates

  5. Environment Variable Documentation

    • Challenge: Ensuring all required vars are documented without exposing values
    • Solution: I created .env.example pattern with clear documentation
    • Validated against deployment configs

  6. Cross-Language Support

    • Challenge: Supporting Python, JavaScript, Java, Go patterns
    • Solution: I built language-specific detection patterns
    • Created comprehensive test cases for each language

  7. Balancing Security vs. Developer Experience

    • Challenge: Blocking merges without frustrating developers
    • Solution: I provided clear, actionable remediation steps
    • Implemented severity levels with proportional responses

Accomplishments that we're proud of

✅ Complete Multi-Agent System

  • Built 5 specialized agents with distinct responsibilities
  • Each agent provides detailed, actionable findings
  • Agents work together intelligently without conflicts

✅ Comprehensive Secret Detection

  • Detects 10+ types of credentials (AWS, GitHub, JWT, Stripe, etc.)
  • Includes entropy-based analysis for unknown secret patterns
  • False positive filtering to avoid blocking legitimate code

✅ Intelligent Risk Scoring

  • Calculates nuanced risk scores (0-100) based on multiple factors
  • Identifies sensitive file changes (CI/CD, Docker, Kubernetes)
  • Detects breaking changes and dependency issues
  • Triggers proportional approval gates

✅ Production-Ready Quality Checks

  • Validates test coverage across multiple languages
  • Detects coverage drops and missing tests
  • Prevents OOMKill errors in Kubernetes deployments
  • Validates database query performance

✅ Developer-Friendly Workflow

  • Conditional triggering reduces unnecessary agent runs
  • Clear, actionable recommendations for each finding
  • Automated labeling for issue tracking
  • Blocks only critical issues, allows low-risk changes

✅ Comprehensive Documentation

  • Detailed README with architecture diagrams
  • Clear agent specifications and detection patterns
  • Test cases demonstrating each agent's capabilities
  • Example configurations for different scenarios

✅ Hackathon Achievement

  • Built complete system in limited timeframe
  • Integrated with GitLab AI Agents platform
  • Created production-ready code
  • Demonstrated real-world security impact

What we learned

  1. AI Agents Excel at Specialized Tasks

    • Agents focused on single responsibilities are more effective
    • Combining multiple agents creates powerful emergent behavior
    • Clear agent boundaries prevent conflicts and duplication

  2. Pattern Matching Requires Context

    • Raw regex patterns generate false positives
    • Context analysis (test files, documentation) is crucial
    • Entropy-based detection complements pattern matching

  3. Risk Scoring is Nuanced

    • Not all code changes are equally risky
    • File type, change magnitude, and context matter
    • Proportional responses (labels vs. blocks) improve adoption

  4. Developer Experience Matters

    • Clear, actionable recommendations are essential
    • Blocking merges requires strong justification
    • Transparency in decision-making builds trust

  5. Multi-Language Support is Complex

    • Each language has different patterns and conventions
    • Test frameworks vary significantly
    • Dependency management differs across ecosystems

  6. Orchestration is Key

    • Conditional triggering prevents wasted resources
    • Flow-based architecture enables scalability
    • Clear handoffs between agents improve reliability

  7. Security & Usability Can Coexist

    • Strong security doesn't require poor UX
    • Detailed findings help developers understand issues
    • Automated remediation suggestions accelerate fixes

  8. GitLab AI Agents Platform is Powerful

    • Flexible agent architecture supports diverse use cases
    • YAML configuration enables rapid iteration
    • Integration with GitLab workflows is seamless

What's next for 5-Agent Security & Quality Analysis System

🚀 Short-term (Next 3 months)

  • Add support for more languages (Ruby, Go, Rust)
  • Implement machine learning for anomaly detection
  • Create custom rule builder for organization-specific policies
  • Add Slack/Teams notifications for critical findings
  • Build dashboard for security metrics and trends

📈 Medium-term (3-6 months)

  • Integrate with SAST/DAST tools (SonarQube, Snyk)
  • Add compliance checking (HIPAA, GDPR, PCI-DSS)
  • Implement auto-remediation for common issues
  • Create agent marketplace for community contributions
  • Build API for third-party integrations

🌟 Long-term (6+ months)

  • Expand to full CI/CD pipeline analysis
  • Add container image scanning
  • Implement supply chain security checks
  • Create AI-powered code suggestions for fixes
  • Build enterprise features (audit logs, role-based access)

🔧 Technical Improvements

  • Optimize agent performance for large repositories
  • Add caching to reduce redundant analysis
  • Implement distributed agent execution
  • Create agent versioning and rollback capabilities
  • Build comprehensive testing framework

📚 Community & Adoption

  • Open-source the agent templates
  • Create documentation and tutorials
  • Build community around custom agents
  • Establish best practices guide
  • Create certification program for agent developers

💼 Enterprise Features

  • Multi-team management and policies
  • Custom approval workflows
  • Advanced reporting and analytics
  • Integration with enterprise tools (Jira, ServiceNow)
  • Dedicated support and SLAs

Built With

Share this project:

Updates