With the rise of Large Language Models (LLMs), AI agents are transforming productivity across various fields. Typically seen as standalone tools, these agents can achieve even more when working together. This repository explores the collaboration between two distinct agents: the CamelAI Framework, which excels in generation and planning, and the Browser Use, specialized in GUI-based automation tasks. By connecting their strengths, we can create a powerful synergy where one agent's capabilities complement the other's limitations. This initial trial demonstrates how heterogeneous agents can work together seamlessly, unlocking new possibilities for complex tasks that require both creativity and precision.

BrowserUseToolkit (browser_use_toolkit.py) is a standard CamelAI toolkit implementation. It should be installed in CamelAI agent projects. This tool receives an instruction and sends it to the browser-use agent. Since GUI automation tasks can take a fairly long time, it submits the task and performs rolling polling to check if the task has finished and produced results.

BrowserUseService (browser_use_service.py) is the backend component responsible for executing web automation task. The service can only work on one specific task at a time. If other tasks are submitted, they will be pending until the current task is finished.

sequenceDiagram participant CamelAI_Agent as CamelAI Agent participant BrowserUseService as Browser Use Service participant BrowserUseAgent as Browser Use Agent CamelAI_Agent->>BrowserUseService: Submit Task with Instructions BrowserUseService-->>CamelAI_Agent: Return task_id BrowserUseService->>BrowserUseAgent: Submit Task for Automation loop Every 2 seconds CamelAI_Agent->>BrowserUseService: Query Status with task_id alt Task is not completed BrowserUseService-->>CamelAI_Agent: Status: In Progress else Task is completed BrowserUseAgent-->>BrowserUseService: Return Results BrowserUseService-->>CamelAI_Agent: Status: Completed BrowserUseService-->>CamelAI_Agent: Return Results end end 

You may need do some customization by yourself, to set up .env file, the browser configuration etc. The service default port is 4999. Following is an example .env file of browser_use_service.py. Place them in the same folder.

OPENAI_API_KEY=<YOUR_OPENAI_API_KEY> MODEL_NAME=gpt-4o-mini USERNAME=user PASSWORD=password 

Here, USERNAME and PASSWORD are sensitive_data will be passed to BrowserUse Agent.

sensitive_data = { 'x_name': os.environ['USERNAME'], 'x_password': os.environ['PASSWORD'] }

Please refer to the Browser Use page.

With pip (Python>=3.11):

pip install browser-use

install playwright:

playwright install

The code (gui_agent.py) demonstrates how to use the toolkit.