A comprehensive framework for economic simulations using Large Language Models (LLMs). The LLM Economist leverages state-of-the-art language models to create realistic, dynamic economic simulations with diverse agent populations for studying tax policy optimization and mechanism design.

• Multi-LLM Support: Compatible with OpenAI GPT, Google Gemini, Anthropic Claude, Meta Llama, and more
• Multiple Deployment Options: Local (vLLM, Ollama), cloud APIs (OpenAI, OpenRouter), and Google AI
• Diverse Economic Scenarios: Rational agents, bounded rationality, and democratic voting mechanisms
• Realistic Agent Personas: LLM-generated personas based on real demographic and occupational data
• Scalable Architecture: Support for 3-1000+ agents with efficient parallel processing
• Comprehensive Testing: Full test suite with real API integration testing
• Reproducible Research: Standardized experiment scripts and configuration management

The LLM Economist framework models economic systems as a two-level multi-agent reinforcement learning problem, implemented as a Stackelberg game where:

1. Tax Planner (Leader): Sets tax policies to maximize social welfare
2. Workers (Followers): Optimize labor allocation based on tax policies and individual utility functions

Key innovations include:

• In-context optimization for rational utility functions
• Synthetic demographic data for realistic agent diversity using real occupation, age, and gender statistics
• LLM-generated personas that create unique, realistic economic agents
• Mechanism design for positive societal influence

# Create and activate the environment conda create -n LLMEconomist python=3.11 -y conda activate LLMEconomist

pip install llm-economist

git clone https://github.com/sethkarten/LLMEconomist.git cd LLMEconomist pip install -e .

The framework supports multiple LLM providers. Install additional dependencies as needed:

# For local LLM serving pip install vllm ollama # For Google Gemini pip install google-generativeai # For development pip install -e .[dev]

Choose your preferred LLM provider and set the corresponding API key:

# OpenAI export OPENAI_API_KEY="your_openai_key" # OpenRouter (for multiple models) export OPENROUTER_API_KEY="your_openrouter_key" # Google Gemini export GOOGLE_API_KEY="your_google_key"

# Simple rational agents simulation python -m llm_economist.main --scenario rational --num-agents 5 --max-timesteps 500 # Bounded rationality simulation (note: currently uses 100% egotistical agents with personas) python -m llm_economist.main --scenario bounded --num-agents 10 --percent-ego 100 # Democratic voting simulation python -m llm_economist.main --scenario democratic --num-agents 15 --two-timescale 50

# OpenAI GPT-4 python -m llm_economist.main --llm gpt-4o --scenario rational # Local Llama via vLLM (requires local server) python -m llm_economist.main --llm meta-llama/Llama-3.1-8B-Instruct --service vllm --port 8000 # Claude via OpenRouter python -m llm_economist.main --llm anthropic/claude-3.5-sonnet --use-openrouter # Google Gemini python -m llm_economist.main --llm gemini-1.5-flash

LLMEconomist/ ├── llm_economist/ # Main package │ ├── agents/ # Agent implementations │ │ ├── worker.py # Worker agent logic │ │ ├── planner.py # Tax planner logic │ │ └── llm_agent.py # Base LLM agent class │ ├── models/ # LLM model integrations │ │ ├── openai_model.py # OpenAI GPT models │ │ ├── gemini_model.py # Google Gemini models │ │ ├── vllm_model.py # Local vLLM/Ollama models │ │ ├── openrouter_model.py # OpenRouter API │ │ └── base.py # Base model interface │ ├── utils/ # Utility functions │ │ ├── common.py # Common utilities │ │ └── bracket.py # Tax bracket utilities │ ├── data/ # Demographic data files │ └── main.py # Main entry point ├── experiments/ # Experiment scripts ├── examples/ # Usage examples │ ├── quick_start.py # Basic functionality tests │ └── advanced_usage.py # Simulation scenario tests ├── tests/ # Test suite └── README.md # This file 

Note: Currently, personas (used in bounded and democratic scenarios) only support egotistical utility types, so mixed utility types are only available with default personas.

OpenAI Models:

• gpt-4o - Most capable, highest cost
• gpt-4o-mini - Fast and cost-effective (recommended)

Via OpenRouter (requires OPENROUTER_API_KEY):

• meta-llama/llama-3.1-8b-instruct - Open source, good performance
• meta-llama/llama-3.1-70b-instruct - Larger Llama model
• anthropic/claude-3.5-sonnet - Excellent reasoning
• google/gemini-flash-1.5 - Fast Google model

Google Gemini (requires GOOGLE_API_KEY):

• gemini-1.5-pro - Most capable Gemini model
• gemini-1.5-flash - Fast and efficient (recommended)

vLLM (Recommended for local deployment):

# Start vLLM server vllm serve meta-llama/Llama-3.1-8B-Instruct --port 8000 # Use in simulation python -m llm_economist.main --llm meta-llama/Llama-3.1-8B-Instruct --service vllm --port 8000

Ollama (Easy local setup):

# Install and start Ollama ollama pull llama3.1:8b ollama serve # Use in simulation python -m llm_economist.main --llm llama3.1:8b --service ollama --port 11434

Run the experiments from the paper:

# All experiments python experiments/run_experiments.py --experiment all # Specific experiments python experiments/run_experiments.py --experiment rational python experiments/run_experiments.py --experiment bounded python experiments/run_experiments.py --experiment democratic python experiments/run_experiments.py --experiment llm_comparison python experiments/run_experiments.py --experiment scalability

# Chain of thought prompting python -m llm_economist.main --prompt-algo cot --llm gpt-4o # Input-output prompting (default) python -m llm_economist.main --prompt-algo io --llm gpt-4o-mini # Large scale simulation python -m llm_economist.main --num-agents 100 --max-timesteps 2000

The framework provides two types of examples:

For quick validation of imports, setup, and basic functionality:

# Test all basic functionality python examples/quick_start.py # Run specific basic tests python examples/quick_start.py --help

The quick start script validates:

• Package imports and dependencies
• Argument parser configuration
• API key detection
• Basic Args object creation
• Service configurations

For actual simulation testing with 20-timestep runs:

# Run all simulation scenarios python examples/advanced_usage.py # Test specific scenarios python examples/advanced_usage.py rational # OpenAI GPT-4o-mini python examples/advanced_usage.py bounded # Bounded rationality with personas python examples/advanced_usage.py democratic # Democratic voting mechanism python examples/advanced_usage.py fixed # Fixed workers with LLM planner # Test different LLM providers python examples/advanced_usage.py openrouter # OpenRouter API python examples/advanced_usage.py vllm # Local vLLM server python examples/advanced_usage.py ollama # Local Ollama python examples/advanced_usage.py gemini # Google Gemini # Show available scenarios python examples/advanced_usage.py --help

All advanced examples use 20 timesteps for thorough testing while remaining fast for development.

The examples are organized to provide clear separation of concerns:

• quick_start.py: Lightweight validation of basic functionality without running simulations

  • Tests imports and dependencies
  • Validates configuration setup
  • Checks API key availability
  • Fast execution (< 10 seconds)

• advanced_usage.py: Full simulation testing with real LLM APIs

  • 20-timestep economic simulations
  • All scenarios: rational, bounded, democratic, fixed workers
  • Multiple LLM providers: OpenAI, OpenRouter, vLLM, Ollama, Gemini
  • Realistic testing (2-10 minutes per scenario)

The framework includes comprehensive tests organized into three categories:

# Test basic functionality (imports, setup, configuration) pytest tests/test_quickstart.py -v # Test individual components python examples/quick_start.py # Direct basic functionality validation

# Test LLM model integrations pytest tests/test_models.py -v # Test simulation logic with mocking pytest tests/test_simulation.py -v # Test advanced usage scenarios (requires API keys) pytest tests/test_advanced_usage.py -v

# Test actual simulations with real APIs python examples/advanced_usage.py # All scenarios python examples/advanced_usage.py rational # Specific scenario

# Run all tests pytest -v # Run with coverage pytest --cov=llm_economist --cov-report=html

• API Keys: Advanced usage and integration tests require API keys:
  • OPENAI_API_KEY or ECON_OPENAI (required for most tests)
  • OPENROUTER_API_KEY (optional, for OpenRouter tests)
  • GOOGLE_API_KEY (optional, for Gemini tests)
• Real Integration: Advanced tests use actual LLM APIs to ensure end-to-end functionality
• Fast Execution: All tests use 20 timesteps or less for quick validation
• Local Servers: vLLM and Ollama tests require running local servers (will skip if not available)

The framework generates realistic agent personas using:

1. Demographic Sampling: Real occupation, age, and gender statistics from census data
2. LLM Generation: Each persona is uniquely generated based on sampled demographics
3. Economic Realism: Personas include realistic income levels, risk tolerance, and life circumstances

Example generated personas:

• "You are a 55-year-old female working as a licensed practical nurse... With over 30 years of experience, you prioritize savings for retirement and healthcare needs."
• "You are a 53-year-old male working as a welding worker... concerns about retirement savings keep you financially cautious."

To reproduce the experiments from the LLM Economist paper:

1. Environment Setup:

   git clone https://github.com/sethkarten/LLMEconomist.git cd LLMEconomist pip install -e . export WANDB_API_KEY="your_wandb_key" # For experiment tracking

2. LLM Setup (choose one):

   Option A: OpenAI (easiest):

   export OPENAI_API_KEY="your_key"

   Option B: Local vLLM (most cost-effective):

   # Start vLLM server with Llama 3.1 8B vllm serve meta-llama/Llama-3.1-8B-Instruct --tensor-parallel-size 1 --port 8000

# Rational agents  python experiments/run_experiments.py --experiment rational --wandb # Bounded rationality  python experiments/run_experiments.py --experiment bounded --wandb # Democratic voting  python experiments/run_experiments.py --experiment democratic --wandb # LLM comparison  python experiments/run_experiments.py --experiment llm_comparison --wandb # Scalability analysis  python experiments/run_experiments.py --experiment scalability --wandb 

Extend the framework with custom agent behaviors:

from llm_economist.agents.worker import Worker class CustomWorker(Worker): def compute_utility(self, income, rebate): # Custom utility function return your_custom_utility_logic(income, rebate)

Add support for new LLM providers:

from llm_economist.models.base import BaseLLMModel class CustomLLMModel(BaseLLMModel): def send_msg(self, system_prompt, user_prompt, temperature=None, json_format=False): # Implement your model's API return response, is_json

Enable detailed experiment tracking with Weights & Biases:

python -m llm_economist.main --wandb --scenario bounded --num-agents 20

API Key Errors:

# Make sure your API keys are set correctly echo $OPENAI_API_KEY echo $OPENROUTER_API_KEY echo $GOOGLE_API_KEY

Local Model Connection:

# Check if vLLM server is running curl http://localhost:8000/health # Check Ollama status ollama list

Memory Issues:

• Reduce --num-agents for large simulations
• Use gpt-4o-mini instead of gpt-4o for cost efficiency
• Adjust --history-len to reduce memory usage

Rate Limiting:

• Add delays between API calls
• Use local models (vLLM/Ollama) for unrestricted access
• Switch to OpenRouter for higher rate limits

Test Failures:

• Ensure API keys are set for quickstart tests
• Check network connectivity for cloud API tests
• Verify local model servers are running for local tests

If you use this framework in your research, please cite:

@article{karten2025llm, title={LLM Economist: Large Population Models and Mechanism Design in Multi-Agent Generative Simulacra}, author={Karten, Seth and Li, Wenzhe and Ding, Zihan and Kleiner, Samuel and Bai, Yu and Jin, Chi}, journal={arXiv preprint arXiv:2507.15815}, year={2025} }

This project is licensed under the MIT License - see the LICENSE file for details.