Skip to content

gojiplus/layoutlens

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

72 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

LayoutLens: AI-Powered Visual UI Testing

PyPI version Downloads Python 3.11+ Documentation

The Problem

Traditional UI testing is painful:

  • Brittle selectors break with every design change
  • Pixel-perfect comparisons fail on minor, acceptable variations
  • Writing test assertions requires deep technical knowledge
  • Cross-browser testing multiplies complexity
  • Generic analysis lacks domain expertise - accessibility, conversion optimization, mobile UX
  • Accessibility checks need specialized tools and expertise

The Solution

LayoutLens lets you test UIs the way humans see them - using natural language and domain expert knowledge:

# Basic analysis result = await lens.analyze("https://example.com", "Is the navigation user-friendly?") # Expert-powered analysis result = await lens.audit_accessibility("https://example.com", compliance_level="AA") # Returns: "WCAG AA compliant with 4.7:1 contrast ratio. Focus indicators visible..."

Instead of writing complex selectors and assertions, just ask questions like:

  • "Is this page mobile-friendly?"
  • "Are all buttons accessible?"
  • "Does the layout look professional?"

Get expert-level insights from built-in domain knowledge in accessibility, conversion optimization, mobile UX, and more.

βœ… 95.2% accuracy on real-world UI testing benchmarks

Quick Start

Installation

pip install layoutlens playwright install chromium # For screenshot capture

Basic Usage

from layoutlens import LayoutLens # Initialize (uses OPENAI_API_KEY env var) lens = LayoutLens() # Test any website or local HTML result = await lens.analyze("https://your-site.com", "Is the header properly aligned?") print(f"Answer: {result.answer}") print(f"Confidence: {result.confidence:.1%}")

That's it! No selectors, no complex setup, just natural language questions.

Key Functions

1. Analyze Pages

Test single pages with custom questions:

# Test local HTML files result = await lens.analyze("checkout.html", "Is the payment form user-friendly?") # Test with expert context from layoutlens.prompts import Instructions, UserContext instructions = Instructions( expert_persona="conversion_expert", user_context=UserContext( business_goals=["reduce_cart_abandonment"], target_audience="mobile_shoppers" ) ) result = await lens.analyze( "checkout.html", "How can we optimize this checkout flow?", instructions=instructions )

2. Compare Layouts

Perfect for A/B testing and redesign validation:

result = await lens.compare( ["old-design.html", "new-design.html"], "Which design is more accessible?" ) print(f"Winner: {result.answer}")

3. Expert-Powered Analysis

Domain expert knowledge with one line of code:

# Professional accessibility audit (WCAG expert) result = await lens.audit_accessibility("product-page.html", compliance_level="AA") # Conversion rate optimization (CRO expert) result = await lens.optimize_conversions("landing.html", business_goals=["increase_signups"], industry="saas") # Mobile UX analysis (Mobile expert) result = await lens.analyze_mobile_ux("app.html", performance_focus=True) # E-commerce audit (Retail expert) result = await lens.audit_ecommerce("checkout.html", page_type="checkout") # Legacy methods still work result = await lens.check_accessibility("product-page.html") # Backward compatible

4. Batch Testing

Test multiple pages efficiently:

results = await lens.analyze( sources=["home.html", "about.html", "contact.html"], queries=["Is it accessible?", "Is it mobile-friendly?"] ) # Processes 6 tests in parallel

5. High-Performance Async (3-5x faster)

# Async for maximum throughput result = await lens.analyze( sources=["page1.html", "page2.html", "page3.html"], queries=["Is it accessible?"], max_concurrent=5 )

6. Structured JSON Output

All results provide clean, typed JSON for automation:

result = await lens.analyze("page.html", "Is it accessible?") # Export to clean JSON json_data = result.to_json() # Returns typed JSON string print(json_data) # { # "source": "page.html", # "query": "Is it accessible?", # "answer": "Yes, the page follows accessibility standards...", # "confidence": 0.85, # "reasoning": "The page has proper heading structure...", # "screenshot_path": "/path/to/screenshot.png", # "viewport": "desktop", # "timestamp": "2024-01-15 10:30:00", # "execution_time": 2.3, # "metadata": {} # } # Type-safe structured access from layoutlens.types import AnalysisResultJSON import json data: AnalysisResultJSON = json.loads(result.to_json()) confidence = data["confidence"] # Fully typed: float

7. Domain Experts & Rich Context

Choose from 6 built-in domain experts with specialized knowledge:

# Available experts: accessibility_expert, conversion_expert, mobile_expert, # ecommerce_expert, healthcare_expert, finance_expert # Use any expert with custom analysis result = await lens.analyze_with_expert( source="healthcare-portal.html", query="How can we improve patient experience?", expert_persona="healthcare_expert", focus_areas=["patient_privacy", "health_literacy"], user_context={ "target_audience": "elderly_patients", "accessibility_needs": ["large_text", "simple_navigation"], "industry": "healthcare" } ) # Expert comparison analysis result = await lens.compare_with_expert( sources=["old-design.html", "new-design.html"], query="Which design converts better?", expert_persona="conversion_expert", focus_areas=["cta_prominence", "trust_signals"] )

CLI Usage

# Analyze a single page layoutlens https://example.com "Is this accessible?" # Analyze local files layoutlens page.html "Is the design professional?" # Compare two designs layoutlens page1.html page2.html --compare # Analyze with different viewport layoutlens site.com "Is it mobile-friendly?" --viewport mobile # JSON output for automation layoutlens page.html "Is it accessible?" --output json

CI/CD Integration

GitHub Actions

- name: Visual UI Test run: |  pip install layoutlens  playwright install chromium  layoutlens ${{ env.PREVIEW_URL }} "Is it accessible and mobile-friendly?"

Python Testing

import pytest from layoutlens import LayoutLens @pytest.mark.asyncio async def test_homepage_quality(): lens = LayoutLens() result = await lens.analyze("homepage.html", "Is this production-ready?") assert result.confidence > 0.8 assert "yes" in result.answer.lower()

Benchmark & Evaluation Workflow

LayoutLens includes a comprehensive benchmarking system to validate AI performance:

1. Generate Benchmark Results

# Run LayoutLens against test data python benchmarks/run_benchmark.py --api-key sk-your-key # With custom settings python benchmarks/run_benchmark.py \ --api-key sk-your-key \ --output benchmarks/my_results \ --no-batch \ --filename custom_results.json

2. Evaluate Performance

# Evaluate results against ground truth python benchmarks/evaluation/evaluator.py \ --answer-keys benchmarks/answer_keys \ --results benchmarks/layoutlens_output \ --output evaluation_report.json

3. Structured Benchmark Results

The benchmark runner outputs clean JSON for analysis:

# Example benchmark result structure { "benchmark_info": { "total_tests": 150, "successful_tests": 143, "failed_tests": 7, "success_rate": 0.953, "batch_processing_used": true, "model_used": "gpt-4o-mini" }, "results": [ { "html_file": "good_contrast.html", "query": "Is this page accessible?", "answer": "Yes, the page has good color contrast...", "confidence": 0.89, "reasoning": "WCAG guidelines are followed...", "success": true, "error": null, "metadata": {"category": "accessibility"} } ] }

4. Custom Benchmarks

Create your own test data and answer keys:

# Use the async API for custom benchmark workflows from layoutlens import LayoutLens async def run_custom_benchmark(): lens = LayoutLens() test_cases = [ {"source": "page1.html", "query": "Is it accessible?"}, {"source": "page2.html", "query": "Is it mobile-friendly?"} ] results = [] for case in test_cases: result = await lens.analyze(case["source"], case["query"]) results.append({ "test": case, "result": result.to_json(), # Clean JSON output "passed": result.confidence > 0.7 }) return results

Configuration

Simple configuration options:

# Via environment export OPENAI_API_KEY="sk-..." # Via code lens = LayoutLens( api_key="sk-...", model="gpt-4o-mini", # or "gpt-4o" for higher accuracy cache_enabled=True, # Reduce API costs cache_type="memory", # "memory" or "file" )

Resources

  • πŸ“– Full Documentation - Comprehensive guides and API reference
  • 🎯 Examples - Real-world usage patterns
  • πŸ› Issues - Report bugs or request features
  • πŸ’¬ Discussions - Get help and share ideas

Why LayoutLens?

  • Natural Language - Write tests like you'd describe the UI to a colleague
  • Domain Expert Knowledge - Built-in expertise in accessibility, CRO, mobile UX, and more
  • Rich Context Support - Business goals, user personas, compliance standards, and technical constraints
  • Zero Selectors - No more fragile XPath or CSS selectors
  • Visual Understanding - AI sees what users see, not just code
  • Async-by-Default - Concurrent processing for optimal performance
  • Simple API - One analyze method handles single pages, batches, and comparisons
  • Structured JSON Output - TypedDict schemas for full type safety in automation
  • Comprehensive Benchmarking - Built-in evaluation system with 95.2% accuracy
  • Production Ready - Used by teams for real-world applications

Making UI testing as simple as asking "Does this look right?"

Releases

No releases published

Packages

 
 
 

Contributors