Features

This release brings significant enhancements to performance, security, extended conversation capabilities, and powerful new tools.

• Performance Optimization: Parallel tool calling is now enabled by default, allowing multiple tools to run simultaneously rather than sequentially. This delivers substantial performance improvements, especially when working with Claude 4 models.

• Extended Conversation Support: New memory compactification automatically summarizes older messages in long conversations when token thresholds are exceeded, while preserving critical system messages. This allows users to maintain extended interactions without hitting token limits across all supported providers (OpenAI, Anthropic, Gemini).

• Enhanced Reasoning Capabilities: Added support for Anthropic's interleaved thinking beta headers. Implemented extended thinking for Claude 3.7 Sonnet and newer models with customizable reasoning effort levels (low, medium, high)

• Cross-LLM Tools:

  • Citations: Generate responses with specific citations from source documents, ideal for creating well-sourced summaries and comparisons. Works with either OpenAI or Anthropic models
  • Web Search: Easily integrate web search capabilities with standardized output handling across OpenAI, Anthropic, or Gemini providers
  • Cross-LLM Code Interpreter: Analyze CSV data by asking questions in natural language, with automatic code generation and execution across multiple LLM providers
  • Improved Report Generation: Enhanced with Claude 4 Sonnet's reasoning capabilities between tool calls, improving the system's ability to plan next steps and use tools more effectively during report generation.

Maintenance

• Better AI Models: The planning model has been upgraded to Claude-4-Sonnet across the application, providing more advanced analysis capabilities and improved query generation. Additionally, the system now uses Gemini Pro 2.5 as the default code interpreter, offering an optimal balance of speed and accuracy.
• Improved Security and Performance: Code execution now takes place in secure LLM provider environments rather than running locally. This architectural change significantly reduces latency and enhances reliability while maintaining a strong security posture.
• Streamlined Report Generation: The report synthesis process has been refactored to leverage the citations function from the defog-python library, making the system more flexible and laying groundwork for potential future integration with OpenAI models for citations.
• Library and Dependency Updates: The application now uses the latest version of the defog library and includes provider names in asynchronous chat communications, preparing the system for upcoming features while maintaining current functionality.
• Codebase Optimization: Unused legacy routes and code have been removed, reducing the application's footprint and improving overall maintainability. Dependencies have been updated to align with current requirements, ensuring a more efficient and robust system.

Release Notes (v0.1.5)

Features

Introspect

• Public Sharing for Oracle Reports: Users can now share Oracle reports with external stakeholders via public links that don't require authentication. A new visibility toggle allows you to easily switch reports between public and private status, making collaboration with clients and team members outside your organization seamless and secure.

Bug Fixes

This release includes important security and stability improvements across our repositories, with a focus on addressing critical vulnerabilities and enhancing thread safety.

Introspect

• Critical Security Enhancements: We've addressed two significant security vulnerabilities in the analysis tools functionality:
  • Fixed a remote code execution vulnerability in custom tools by implementing proper validation of input models and restricting potentially dangerous built-in functions. This strengthens the security of the testing endpoint without changing how you use the feature. PR #501
  • Resolved a code injection vulnerability (CWE-94) by restricting code execution capabilities and implementing validation for user-supplied code. This makes the application more secure when processing analysis requests while maintaining full functionality. PR #502

Defog-Python

• Improved Thread Safety: Database query helpers now properly manage connections and ensure they're safely closed after use. This prevents potential resource leaks and improves application stability, particularly in multi-threaded environments where multiple database queries are executed simultaneously. PR #89

These fixes enhance the overall security and reliability of our products while maintaining the same user experience you're accustomed to.

Maintenance Updates

Introspect

This release includes several important model upgrades and quality improvements to enhance the performance, reliability, and cost-effectiveness of the Introspect platform:

• AI Model Upgrades: The planning model has been updated from Claude-3.7-Sonnet to Claude-4-Sonnet across the application, providing users with improved reasoning capabilities and more accurate responses.

• Optimized Report Generation: The multi-agent report generation process now leverages o4-mini for initial report creation and gap analysis, while maintaining Claude 4 Sonnet for final output. This optimization delivers faster, more cost-effective, and more concise reports without sacrificing quality, improving your overall experience when generating analytical reports.

v0.1.4

Features

• Expanded AI Model Support: The library now supports OpenAI's and Gemini's newest models, including GPT-4.1, O4-mini, and the O3 series + gemini-pro-2.5

• Model Upgrades: We've replaced all instances of o3-mini with o4-mini throughout the codebase, delivering improved performance and capabilities for all model-dependent features.

• Better Clarifications: The clarification system now uses GPT-4.1 with an optimized prompt, resulting in more relevant and helpful clarifying questions when analyzing your inputs. This improvement helps the system better understand your intent and deliver more accurate results.

• Gemini Pro 2.5 Integration for Web Search: Users can now use Google's Gemini Pro 2.5 model for web search operations when a Gemini API key is provided. This delivers superior search results at a lower cost compared to previous options. We fall back to OpenAI's web search tool when no Gemini API key is available, ensuring continuous functionality.

• Frontend Code Interpreter Support: The frontend now includes code interpreter tool putputs, integrated into the thinking steps and inline citations. This allows users to better understand and visualize code-related operations performed by the agent.

• Frontend New Think Tool Support: The thinking steps UI introduces a dedicated "think tool" UI that provides deeper insights into the agent's reasoning process.

Bug Fixes

• Better LaTeX Rendering: Improves mathematical expression display by adding instructions to avoid dollar sign usage in LaTeX, preventing common formatting errors that previously disrupted formula presentation.

• Improved Citation System: Enhances the citation experience with better markdown formatting for inline citations, making references more readable and professionally presented in reports.

• Optimized Dashboard Visualization: Refines the expansion and collapse functionality in chart containers, providing a smoother and more intuitive visual experience when interacting with dashboards.

• Report Content Rendering Enhancements: Updates to Oracle Report components ensure more consistent display of SQL analysis content and citation references, creating a more polished and professional report presentation.

Features

• New Code Interpreter Tool: Adds advanced data analysis capabilities that enable users to perform sophisticated operations like statistical analyses and data cleaning directly within the application. This powerful addition streamlines complex data workflows without requiring external tools.
• UI Improvements:
  • Introduces a resizable history sidebar for Oracle reports with a draggable handle, allowing users to adjust the width to their preference. The system automatically remembers your preferred width across sessions for a consistent experience.
  • Adds drag-and-drop functionality to the dashboard, enabling users to reorder and hide items for a more personalized workspace that matches their specific needs and workflows.

These enhancements focus on giving users more control over their experience while adding powerful new capabilities that streamline complex data analysis tasks.

Features

This release introduces powerful new export options, enhanced mathematical content support, and improved data visualization capabilities across our platform.

Introspect

• New Podcast Transcript Export for Oracle Reports: Convert analytical reports into engaging conversational podcast transcripts between two hosts. This new export option will eventually make technical content more accessible and shareable in audio format (actually audio creation coming soon).

• LaTeX Support in Report Generation: Mathematical formulas and equations now render properly in reports thanks to new LaTeX formatting instructions. This enhancement allows for more professional presentation of complex mathematical content.

Agents UI Components

• Enhanced Data Visualization:
  • A new dashboard view transforms reports into navigable dashboards, making data exploration more intuitive.
  • Improved CSV export functionality ensures reliable exports even with complex data structures.

• Podcast Transcript Export Feature: Users can now convert Oracle reports to podcast transcripts with a simple click. The system provides clear visual feedback during export processing with a loading overlay and spinner.

• LaTeX Rendering Support: Mathematical formulas are now properly displayed in prediction reports with citations. Users can include LaTeX formulas in their queries for clearer presentation of mathematical concepts and equations.

Bug Fixes

This release includes important bug fixes across our platform components, enhancing stability, improving chart visualization, and ensuring consistent data representation across different time zones.

Agents UI Components

• Resolves timezone-related date display issues in charts, ensuring users in all regions see accurate date values. Previously, users in timezones east of GMT would experience incorrect year values in bar charts, which has now been fixed.

• Enhances chart functionality in Deep Research Reports through standardized chart handling and improved performance. The update introduces:

  • Simplified state management for faster chart rendering
  • Loading indicators for better user feedback during chart generation
  • Consistent chart display across different report types

These fixes collectively improve the reliability and user experience of our visualization tools, ensuring data is displayed accurately regardless of user location or client type.

Maintenance

This release includes important performance optimizations and dependency updates across our repositories to ensure stability, security, and improved user experience.

Agents UI Components

• Performance Optimization: Oracle components now implement memoization and optimized rendering patterns, significantly reducing unnecessary re-renders. This makes the Oracle interface more responsive and efficient, particularly when working with complex data or on lower-powered devices.

• Enhanced Compatibility: UUID generation has been centralized in a single utility function with fallback support for environments where native crypto.randomUUID() is unavailable. This improves cross-environment compatibility and introduces the ability to generate shortened UUIDs when needed for specific use cases.

Introspect

• Dependency Updates: Critical libraries have been updated to their latest versions, including the AUC library and another core dependency to version 1.10.4. These updates incorporate the latest security patches and performance improvements, ensuring the application remains secure and performs optimally.

These maintenance improvements work together to provide a more stable, secure, and responsive experience across our products while ensuring compatibility with a wide range of environments.

Features

This release introduces significant enhancements to our analysis capabilities, report presentation, and integration options across multiple repositories.

• Adding a Thinking Tool for Report Generation: A "thinking" tool has been added for better reasoning during report generation. This enables more thorough analysis and deeper reasoning. This enhancement delivers more comprehensive insights and higher quality content in generated reports.

• Interactive Citations: Citations in reports are now fully interactive, allowing users to explore source information directly within their workflow:

  • Users can click on citations to view the corresponding analysis with automatic scrolling to the relevant section
  • The "Dig Deeper" feature replaces the side panel with expandable inline content, making source information more accessible
  • Context-specific icons (SQL, PDF, or web search) and user-friendly text replace technical identifiers, making citations more intuitive

• Model Control Protocol (MCP) Support: The Defog Python library now supports the Model Control Protocol, enabling connections with external tools through a standardized interface. This integration works seamlessly with Anthropic, OpenAI, and Gemini models, allowing AI models to leverage external tools to enhance their capabilities. Users can initialize MCP clients to connect to servers and utilize tool-augmented responses in their applications.

Bug Fixes

This release includes important bug fixes across multiple repositories to improve reliability, compatibility, and error handling.

• Better Structured Output Handling: The SDK now includes robust fallback mechanisms when processing LLM responses that contain JSON with markdown artifacts or formatting inconsistencies. This ensures more reliable data processing and prevents parsing failures when working with structured outputs.

• Improved Compatibility with OpenAI-Compatible APIs: The client now gracefully handles cases where prompt token details are missing in responses from third-party OpenAI-compatible APIs. This enhancement prevents errors when using the Defog client with alternative AI providers that implement the OpenAI interface but don't include all cache-related information in their responses.

Infrastructure and Devex

• Dependency Updates and MCP Client Support: Critical dependencies have been updated to resolve application issues and add support for MCP clients. This improves overall application stability and expands compatibility with different client types, ensuring a smoother experience for users across various environments.

• Security and Dependency Updates: We've updated NextJS and agents-ui-components dependencies to address security vulnerabilities, ensuring a more secure application environment while improving overall user experience.

• Improved Component Architecture: The Oracle component has been refactored into smaller, more focused components with clear responsibilities. This architectural improvement enhances code maintainability without changing functionality, creating a more stable foundation for future enhancements.

• Centralized UUID Generation: We've implemented a unified approach to UUID generation with fallback support for environments where standard methods aren't available. This ensures consistent identification across different environments and improves application reliability.

These maintenance updates strengthen our codebase and infrastructure while maintaining a seamless experience for users. Though largely behind-the-scenes, these improvements contribute to a more robust, secure, and maintainable application.

Features

Backend

This release brings significant enhancements to our data analysis and reporting capabilities:

• Enhanced Report Generation with two major improvements:

  • Multi-agent Report Generation delivers higher quality reports through a three-phase process (data collection, evaluation, and synthesis), resulting in better organization, more comprehensive coverage, and improved logical flow.
  • Citations Functionality adds transparency to reports by clearly referencing information sources, allowing users to verify data origins and build greater confidence in analysis results.

• Improved PDF and File Management:

  • PDF Search in Fast Analyses enables AI-powered searching through PDF documents with citations, expanding data exploration capabilities.
  • Complete PDF File Management allows uploading, viewing, downloading, and deleting PDF files within projects through an intuitive interface.
  • Streamlined File Upload Experience with a unified endpoint that handles multiple file types more efficiently and sends file binaries directly instead of using base64 encoding.

• Enhanced Data Processing:

  • CSV and Excel File Cleaning automatically removes irrelevant elements, handles merged cells, standardizes null values, and converts data into SQL-friendly formats, making analysis more accurate.
  • Direct Database Export allows users to export CSV files directly to their selected database rather than always defaulting to the internal database.

• Improved Administration and Integration:

  • Revamped User Management with new fields, statuses, and improved validation for more efficient user administration.
  • New API Endpoints including a combined tables/files endpoint that simplifies integration by providing a complete view of database resources in a single request.

• AI Improvements:

  • OpenAI Agents Architecture introduces an experimental foundation for future AI-powered analysis capabilities with specialized analyst, evaluator, and report agents.
  • OpenAI as Default Web Search Tool replaces Gemini, providing significantly improved search results.

Frontend

This release delivers substantial improvements to the user interface and experience:

• Enhanced Report and Citation Features:

  • Interactive Report Citations allow users to click on citations to automatically select and scroll to corresponding analyses, creating a more intuitive workflow.
  • Citation Support in Analysis Results displays source information for data used in query responses, enhancing transparency and credibility.
  • PDF and Markdown Export Options enable saving and sharing reports in multiple formats without requiring additional software.

• Improved File Handling:

  • Multiple File Upload Support allows users to upload several files simultaneously instead of one at a time.
  • Real-time Upload Progress Tracking with progress bars and percentage indicators provides better visibility during uploads.
  • Built-in File Validation for extensions and MIME types simplifies implementation and provides specific feedback for invalid files.

• Enhanced User Interface:

  • Unified Oracle Interface merges query-data functionality into Oracle's embed page, providing a comprehensive data analysis experience.
  • Improved Oracle Search Bar with smooth animations when switching between 'Fast Analysis' and 'Deep Research' modes and an intuitive toggle control.
  • Keyboard Shortcuts for improved navigation: '/' to focus the search bar, 'cmd + m' to toggle mode, and 'cmd + k' to start a new question.
  • Better Dark Mode Support across the application, including SQL editor and thinking step questions.

• Resource Management Improvements:

  • Enhanced Resource Discovery with scrollable resource sections and automatic refresh after file uploads or project creation.
  • Oracle Sidebar Enhancements with hover functionality to display full report questions and color-coded status indicators.

These updates collectively deliver a more powerful, intuitive, and transparent data analysis platform with improved reporting capabilities and a more efficient user experience.

Bug Fixes

This release includes numerous bug fixes across our platforms, enhancing stability, improving user experience, and resolving key issues in data handling and interface interactions.

Backend

• Enhanced Data Import and Processing

  • Improves Excel and CSV file parsing to correctly identify column headers and data types, ensuring proper data structure during import and preventing null column names in databases.
  • Fixes PDF upload functionality throughout the application, allowing users to upload documents without requiring accompanying Excel/CSV files or database connections.
  • Enhances table detection accuracy with improved header identification and pattern recognition, particularly beneficial for small datasets and complex table structures.

• Database Management Improvements

  • Ensures proper deletion of old metadata when removing databases, preventing conflicts when reuploading the same database later.
  • Improves compatibility with non-PostgreSQL databases, particularly MySQL, with UI fixes and backend refactoring to support diverse database systems.
  • Fixes Redshift integration issues, ensuring correct schema display and improving the metadata extraction interface.

• Citation and Query Functionality

  • Enhances citation functionality by making sources clearer and more meaningful, helping users better understand which specific references are being used in analyses.
  • Improves golden query management in the alignment model interface, ensuring queries can be properly added and updated.

• Error Handling and System Reliability

  • Implements better error validation for report IDs, providing clear error messages when requested reports don't exist.
  • Improves error handling during file uploads by sending plain text error messages, making troubleshooting more straightforward.
  • Enhances installation reliability on Windows systems by properly handling line ending differences between operating systems.

Frontend

• Search and Input Improvements

  • Fixes search bar functionality to allow typing slash characters without triggering unwanted menu focus.
  • Enhances text area animations with smooth height adjustments based on content and reasonable maximum height limits.
  • Improves the SingleSelect component with better dropdown behavior, custom value handling, and more intuitive focus management.

• File Upload Enhancements

  • Resolves file upload issues across all search bars and modes in the Oracle embed.
  • Enables PDF file uploads in fast analysis mode, enhancing document analysis capabilities.
  • Fixes file extension handling to ensure proper processing of uploaded files.

• Analysis and Navigation Fixes

  • Corrects auto-scrolling to newly created analyses and improves the display order in tree views.
  • Fixes an issue where original questions would be lost during deep research clarification prompts.
  • Improves project name handling when creating new fast analyses.

• Visual and Layout Enhancements

  • Resolves CSS styling issues in Oracle question clarifications for proper display of search bars and file upload components.
  • Improves alert positioning to appear above search bars without disrupting layout.
  • Fixes multi-line question display in fast analysis mode to keep the submit button accessible on screen.

Documentation

This release brings significant improvements to our documentation across multiple repositories, enhancing both developer experience and AI assistant capabilities.

Backend

• Enhanced Developer Guidance: A new dedicated CLAUDE.md file provides comprehensive test instructions and guidance specifically for AI assistants working with the codebase, complementing the updated README.md with improved developer documentation.
• Docker Configuration Improvements: The docker-compose.yaml now includes named containers, making command execution and log access more intuitive during development and testing workflows.

Frontend

• AI Assistant Integration: Introduces a new CLAUDE.md file with detailed guidance for AI assistants, establishing clear conventions and workflow documentation to ensure higher quality AI-generated code contributions.

These documentation improvements streamline the onboarding process for both human developers and AI assistants, establishing clearer conventions and providing more accessible guidance for working with our repositories.

Maintenance

This release includes significant infrastructure and code quality improvements across our repositories, enhancing performance, reliability, and maintainability.

Backend

• Performance Optimization: The backend now uses uvloop instead of the default asyncio worker class, dramatically improving performance. File upload speeds are now significantly faster—large files that previously took over a minute now upload in seconds. Multiple workers have been added to prevent CPU-intensive tasks from blocking the service.

• Database Improvements: We've implemented automated database schema migrations that allow the application to update smoothly when new data fields are added, without disrupting existing data. The internal 'DbCreds' component has been renamed to 'Project' for better clarity and organization.

• Enhanced Error Handling: Detailed error messages have been added throughout the application, making it easier to identify and troubleshoot issues when SQL queries fail or database credentials are missing.

• Improved User Interface: Report citations have been reformatted to make them more user-friendly in the interface, ensuring a ...