Phish Eye

AI-powered real-time scam detection that sees, hears and speaks, protecting everyday people from phishing, fraud and voice cloning scams.

Comment

Inspiration

While working front-end at one of the telco companies in Australia, I saw firsthand the human cost of cybercrime. I met customers of every age who were left in tears after being hunted by scammers. As a Cybersecurity student, those tears felt like a personal call to action. I realized that while the industry protects corporations, everyday people are often left defenseless. I moved to the USA and saw the same global epidemic- billions of dollars lost to scams annually. I pledged to build a "Digital Guardian" to turn the tide.

What it does

PhishEye is an all-in-one AI scam protection platform. It uses a multi-agent system to provide real-time defense: ~Visual Detective: Scans emails/messages for psychological "scam-triggers." ~Live Sentinel: Analyzes live call transcripts to detect IRS impersonation or urgent payment demands. ~Call Guardian: Provides instant advice during suspicious interactions. ~Education Agent: Empowers users with grounded data from the FTC and FBI.

How we built it

The platform is built on Google Cloud and powered by Gemini 3 (Flash).

  1. Orchestration: We used the Agent Development Kit (ADK) to orchestrate four specialized AI agents.
  2. Backend: Hosted on Cloud Run for scalable, real-time processing.
  3. Grounding: Leveraging RAG (Retrieval-Augmented Generation) to ground agent responses in verified FBI and FTC scam data.
  4. Multimodal: Utilizing Gemini’s vision capabilities for the visual scanner and audio-to-text processing for call monitoring.

Challenges we ran into

One major hurdle was minimizing latency for real-time call analysis. We had to optimize the ADK Orchestrator to route data to the correct agent instantly so the user is warned before they make a mistake. Additionally, balancing technical accuracy with "Plain English" explanations for non-tech-savvy users (like the grandparents I met at the telco) required significant prompt engineering.

Accomplishments that we're proud of

~Built a fully working 4-agent ADK system with an Orchestrator routing between a Visual Detective, Live Sentinel, Call Guardian, and Educator Agent — all live on Google Cloud Run

~Created a multimodal experience that combines vision, voice, and text in one seamless app — anyone can use it with zero technical knowledge

~The Visual Scanner correctly identified a classic Netflix phishing email with HIGH confidence — catching the spoofed sender, credential harvesting, and fake urgency in seconds

~The Call Guardian caught a live IRS impersonation script instantly — flagging urgency tactics, gift card demands, and impersonation patterns in real time

~Built, designed, deployed, and shipped a complete product in under 48 hours — solo, as a student

~Made it completely free with no account required — because scam protection should be accessible to everyone

What we learned

~How to architect a multi-agent system using ADK with an orchestrator and specialist sub-agents that hand off to each other seamlessly

~How to deploy a Python ADK application to Google Cloud Run with lazy imports — the heavy dependency graph caused startup failures until I deferred initialization to first use

~How to build a UI that non-technical users — elderly people, immigrants, everyday people — can use without any instructions or tech knowledge

~Real-world scam patterns from working directly with fraud victims — which made the detection logic much more grounded and accurate

~How to combine Gemini vision, Google Cloud TTS, and the Web Speech API into one fluid multimodal experience

What's next for Phish Eye

~Multi-language call detection — protecting non-English speakers because scammers don't discriminate by language

~Real-time voice call monitoring — PhishEye listens to the call live without you having to type anything

~Browser extension — flags phishing links and suspicious emails directly in your browser

~Family protection mode — set up PhishEye to protect elderly family members automatically

~Mobile app — so protection is always in your pocket

~Partnerships with telecom companies and banks — to protect customers at scale

Built With

Share this project:

Updates