🚀 Choragi - The Autonomous AI Concert Orchestrator

Please Note: Just a quick technical note for the judges: the Live Negotiator's integration with Twilio and the Gemini Live API is fully functional. However, to be respectful of real businesses, I’ve hardcoded the dialer to route to my own phone for this demo rather than dialing the actual venues the venue-finder agent just found. This lets me safely demonstrate the AI's real-time voice capabilities for you.

💡 Inspiration

Music has an unparalleled, universal power to heal. It transcends language, mends broken spirits, and brings communities together in moments of pure, shared euphoria. We’ve all felt that electric energy at a live concert—standing in a crowd of thousands of strangers, singing the exact same lyric, finding solace, connection, and joy. Music is medicine for the soul.

But behind that magical, healing experience is a grueling and exhausting reality. Independent musicians and event organizers—the very people creating this magic—spend 90% of their time drowning in logistics. They are forced to become booking agents and marketers: spending days scouting venues, making stressful phone calls to negotiate rates, struggling to design professional promotional materials, and wrestling with ad campaigns.

We realized that the world needs more music, but artists need more time to actually create it. We thought, "What if we could build an engine that takes away the logistical burden?" This sparked the vision for Choragi. We wanted to fuse the absolute cutting edge of Google's generative AI with real-time telephony to turn months of grueling tour planning into a 5-minute autonomous sequence. Choragi exists so that artists can focus on the healing art of music, while AI handles the business of sharing it with the world.

⚙️ What it does

Choragi is a comprehensive, multi-agent AI Event Orchestration system that handles everything from the initial tour idea to the final ad campaign. Triggered by a simple, natural voice conversation, the system autonomously:

🔍 Scouts Venues: Automatically discovers potential concert spaces based on the target city and date.
📞 Negotiates Live: Uses real-time Native Audio AI to actually call the venue over the phone, converse with the manager, and secure the booking.
🎨 Directs Creatives: Generates photorealistic, 8K concert posters and cinematic, high-fidelity promotional video trailers for the artist.
🌐 Builds the Site: Autonomously constructs and deploys a live promotional website featuring the generated assets.
📈 Promotes the Show: Uses browser automation and Vision AI to navigate Google Ads and launch a targeted digital campaign.

🛠️ How we built it

Choragi is built on a robust, scalable microservices architecture powered by Java 17 and Spring Boot. We deployed a system of 6 highly specialized agents communicating seamlessly in the cloud.

Architecture & Scale

6 Microservices deployed on Google Cloud Run (ui-client, venue-finder, live-negotiator, creative-director, site-builder, digital-promoter).
Multi-Modal AI Integration utilizing Gemini 2.5 Flash Native Audio, Gemini 2.5 Pro Vision, Gemini Flash Image, and Vertex AI Veo 3.0 Fast.
Real-Time WebSocket Engine using STOMP/SockJS for real-time UI updates and ultra-low latency audio streaming.

🧩 Architectural Diagram

Choragi System Architecture

Technology Stack

Backend: Java, Spring Boot, REST APIs.
AI & Cloud: Google Vertex AI, Gemini Live API, Google Cloud Storage, Cloud Run, Application Default Credentials (ADC).
Telephony: Twilio WebSockets for real-time call bridging.
Automation: Microsoft Playwright for headless browser control.

🚧 Challenges we ran into

Building a pipeline this ambitious pushed us to the absolute limit, resulting in a few late-night debugging sessions:

Real-Time Audio Transcoding: Bridging Twilio's MuLaw audio format with Gemini's strict 16kHz PCM requirement on-the-fly required precise byte-level manipulation in Java. We also had to build a custom Voice Activity Detection (VAD) RMS algorithm to prevent background static from interrupting the AI.
Vertex AI Payload Strictness: Integrating the Long-Running Operations of Veo 3.0 Fast via raw REST APIs meant navigating strict undocumented payload requirements (like explicitly declaring durationSeconds and generateAudio flags) to avoid 500 Internal Server Errors.
Browser Automation in the Cloud: Running Playwright with Google Chrome locally is easy; running it inside a headless Linux container on Cloud Run without causing out-of-memory or single-process crashes required aggressive environment tuning.
WebSocket State Management: Keeping the frontend UI perfectly in sync with 5 background Spring Boot microservices required migrating heavy blocking tasks to asynchronous threads to prevent the UI from freezing.

🏆 Accomplishments that we're proud of

Live Telephony Integration: Successfully building a bridge that allows a generative AI model to hold a coherent, goal-oriented phone call with a human via Twilio.
Cinematic Programmatic Video: Successfully automating the generation of high-fidelity Veo 3.0 Fast videos and dumping them directly into public Cloud Storage buckets programmatically.
Flawless Orchestration: Getting 6 independent microservices to hand off a single task—from a simple voice command to a live Google Ad—without human intervention.

📚 What we learned

Building Choragi from scratch was an incredible leap. It forced us to grow beyond standard CRUD applications into the world of real-time streaming and cloud-native AI.

Google Cloud Ecosystem: We learned the true power of Google Cloud Run for serverless microservices, the security of Application Default Credentials (ADC), and how to directly interface with Vertex AI's REST endpoints when SDKs fall short.
Streaming Modalities: We mastered the Gemini BidiGenerateContent WebSocket API, realizing the crucial differences in requesting AUDIO versus TEXT response modalities, and how to intelligently parse streaming text buffers to trigger backend systems from a voice stream.
Resilience: We learned how to build robust "God Mode" fallbacks, ensuring that even if an AI hallucinates a phrase, the engineered system logic remains unbreakable.

🔮 What's next for Choragi

Ticketing Integration: Automatically generate and embed Ticketmaster or Eventbrite links into the generated websites.
Multi-Lingual Negotiation: Expand the Live Negotiator to automatically detect and negotiate in the native language of international venue owners.
Social Media Autonomous Posting: Have the Digital Promoter automatically slice the Veo generated videos and post them as TikToks and Instagram Reels.

🌟 Bonuses

Deep Google Cloud Usage: We fully utilized Google Cloud Run for our 6 microservices, Google Cloud Storage for public asset hosting, Vertex AI for Veo 3.0 Fast generation, and Google Secret Manager for secure credential injection.

🚀 Built with passion and late nights—automating the chaos of event planning so the world can focus on the healing power of live music.