Symbolicai

These changes expand SymbolicAI's capabilities with next-generation models from OpenAI, DeepSeek, and Anthropic.

Major Changes

New Models Support
- Added support for Claude 3.7 Sonnet with extended thinking capabilities
- Added support for OpenAI's o1 and o3-mini models with reasoning mode
- Added DeepSeek Reasoner model support

New Reasoning Features
- Implemented structured reasoning support across multiple LLM providers:
- Claude 3.7 with extended thinking (up to 64k tokens for thinking)
- OpenAI's o1/o3 models with reasoning mode
- DeepSeek Reasoner with explicit reasoning capabilities

Engine Improvements
- Refactored Anthropic Claude engines for improved response handling
- Added support for streaming responses with Claude models
- Improved token counting and context management
- Enhanced tool use support across different model providers

Architecture Changes
- Modularized request payload preparation with cleaner code structure
- Improved error handling for API interactions
- Added consistent handling for reasoning/thinking outputs

Developer Experience
- Better handling of max_tokens vs max_completion_tokens for OpenAI models
- More consistent self-prompting behavior
- Enhanced JSON response format support

Dependencies
- Added `loguru` (≥0.7.3) for improved logging
- Added `aiohttp` (≥3.11.13) for async HTTP requests

Version Update
- Increased version from 0.8.0 to 0.9.0

**Full Changelog**: https://github.com/ExtensityAI/symbolicai/compare/v0.8.0...v0.9.0

This release significantly improves the framework's configuration management, local model support, and validation capabilities while maintaining backward compatibility where possible. Users should review the new configuration system documentation when upgrading (see docs [here](https://symbolicai.readthedocs.io/en/latest/INSTALLATION.html#configuration-management)).

Major Features

New Priority-Based Configuration System
- Introduced a hierarchical configuration management system with three priority levels:
1. Debug Mode (Current Working Directory) - Highest priority
2. Environment-Specific Config (Python Environment) - Second priority
3. Global Config (Home Directory) - Lowest priority
- Added `symconfig` command to inspect current configuration setup
- Configurations now properly cascade and fall back based on priority

Enhanced Contract System
- Added new `contract` decorator for Design by Contract (DbC) pattern
- Supports both type and semantic validation
- Includes retry mechanisms and performance monitoring
- Added comprehensive performance statistics tracking for contract execution

Improved Local Model Support
- Extended support for local LLaMA.cpp models:
- Added embedding capabilities through local models
- Support for both Python bindings and direct C++ server
- Added batch processing for embeddings
- Enhanced server configuration options for local models

Package Management Improvements
- Enhanced `sympkg` with new features:
- Support for local package installation
- Git submodules initialization option
- Improved package update mechanism
- Added `--local-path` option for installing from local directories
- Added `--submodules` flag for Git repository operations

Breaking Changes
- Configuration file locations have changed due to new priority system
- Environment variables structure updated for speech-related settings
- Some API methods now return different types/structures
- Updated dependency requirements:
- numpy: Now supports up to 2.1.3
- openai: Minimum version increased to 1.60.0

New Features
- Added MetadataTracker for better usage tracking and statistics
- Enhanced token truncation system with smart percentage calculation
- Added new validation primitives for type and semantic checking
- Improved error handling and reporting
- Added new data models for structured input/output

Improvements
- Better handling of JSON validation and error correction
- Enhanced error messages and logging
- Improved documentation structure
- Better support for local development workflows
- Enhanced configuration management utilities

Dependencies
- Added new dependencies:
- nest-asyncio>=1.6.0
- rich>=13.9.4
- Optional dependency for LLaMA.cpp: llama-cpp-python[server]>=0.3.7

Documentation
- Reorganized API documentation structure
- Added comprehensive configuration management guide
- Improved package management documentation
- Added new examples and use cases
- Enhanced local engine documentation

Bug Fixes
- Fixed configuration cascade issues
- Improved error handling in package management
- Fixed token counting in various scenarios
- Addressed memory leaks in long-running processes
- Fixed various edge cases in validation systems

Developer Tools
- Added new `symconfig` command for configuration inspection
- Enhanced `symdev` and `sympkg` utilities
- Improved debugging capabilities
- Added performance monitoring tools

**Full Changelog**: https://github.com/ExtensityAI/symbolicai/compare/v0.7.4...v0.8.0

Major Features
- Added GUI-based installer for SymbolicAI with support for Windows, macOS (Universal Binary), and Linux
- Introduced Lean4 engine support for formal verification and theorem proving
- Added shell application (`app.py`) for easier CLI access
- Implemented plugin system with configurable prefix commands in shell

Engine Improvements
- Enhanced token handling with new truncation strategies for large inputs
- Added support for GPT-4 Turbo and Claude models with improved API integration
- Updated Anthropic Claude integration with latest API changes
- Enhanced Perplexity API integration for search capabilities
- Improved error handling and API key management across engines

Documentation & UI
- Updated installation and quickstart guides
- Replaced logo with new banner design
- Enhanced code examples and documentation clarity
- Updated Python package dependencies and version requirements

Core Functionality
- Improved conversation handling with better context management
- Enhanced document retrieval and indexing capabilities
- Added support for metadata preservation in document processing
- Improved memory management for conversations
- Enhanced vector database operations

Development & Build
- Added comprehensive build system for creating installers
- Improved icon handling and conversion utilities
- Enhanced Git integration for package management
- Updated environment configuration and dependency management

Bug Fixes & Optimizations
- Fixed conversation state handling
- Improved error handling in shell commands
- Enhanced token management for large inputs
- Fixed vector database cleanup operations
- Optimized memory usage in conversation handling

Configuration
- Updated minimum Python requirement to 3.10
- Added new configuration options for API endpoints
- Enhanced package dependency management
- Updated core dependencies to latest stable versions

Breaking Changes
- Changed minimum Python version requirement to 3.10
- Modified conversation API interface
- Updated API handling for newer model versions
- Changed package installation and import mechanisms

Release Notes

New Features
- **Support for Latest Models**:
- Added support for Claude-3-5-Haiku.
- Integrated the latest Anthropic models.
- Added support for Black Forest Labs' Flux image generation models.
- **Image Rendering Enhancements**:
- Introduced image rendering model configuration and defaults.
- Refactored the module name from `imagerendering` to `drawing engine` for better clarity.
- **Symsh Enhancements**:
- Added support for commands on Symsh.
- **Documentation Enhancements**:
- Added vision documentation.
- Included perplexity search documentation.
- **General Updates**:
- Enhanced readme with updated links to documentation and papers.
- Organized installation documentation with reference fixes.

Fixes
- Resolved a bug in Symsh when not using a stateful conversation.
- Fixed issues with the DocumentRetriever component and local vector database with persistence and indexing.
- Corrected code examples to enhance documentation clarity.

Refactoring and Clean-up
- Conducted minor refactorings and documentation updates for improved maintainability.
- Streamlined documentation pages with consolidated details.

Release Note

Updates

- Removed unused imports for optimized performance.
- Improved argument and keyword argument passing on engine creation.
- Introduced preview and representation improvements.
- Added beam search keyword to the huggingface engine.
- Introduced a custom property exception on reserved assignment.
- Added support for GPT-4o-mini.
- Optimized performance with the removal of the blip to avoid M3 incompatibility, thus it is now only optional.
- Added huggingface support through a local OpenAI-compatible FastAPI server.
- Refactored llama.cpp engine and added symserver.
- Revised async tests in llama.cpp.
- Implemented minor tweaks in symrun and removed OpenAI cost tracker and all related elements.
- Added anthropic dependency and support for Anthropic API, with minor refactoring.
- Added a check to disable symai warnings and marked cost tracking for deprecation.
- Switched to pytest for easier and more efficient testing.

Bug Fixes

- Fixed a major bug in the type system with operators, ensuring operator type safety.
- Fixed a race condition bug on symrun exit.
- Fixed tests-related issues.
- Fixed a warning if local is not used.

Removals

- Removed deprecated services.
- Removed print statements.

Security

- Addressed security issues noted by Dependabot.

Please note that this update also includes minor refactoring and code improvements and is part of our ongoing efforts to maintain and improve our platform's performance.

Release Notes

New Features:
- Support for similarity normalization and extended support for models and a cost tracker.
- Implementation of SAT Solver, added a linker for results and circular reference checks.
- Added a shape fix, torch type check for embedding, and generalized radial basis function (RBF) implementation for the distance function.
- Introduced an improved aggregator class with better error handling.
- Added parallel component support.
- Implemented a custom vector database (DB) and added defaults for non-API settings to favor local embedding models and vector DB.
- Introduced LLaVA support for in-memory images and format adaptation.
- Extended similarities and added a metric component to enhance tracking and comparison capabilities.

Enhancements:
- Added various kernel functions to expand computational capabilities.
- Expanded tensor and array support for typing, improving the handling of different data structures.
- Improved distance function implementations to enhance calculations.
- Improved type checking and conversion mechanisms, allowing for more robust data handling.
- Added an improved graph construction and visualization process.
- Improved memory and tagging management within Conversations.
- Added a new tutorial video to enhance user learning resources.
- Implemented better suited presets for smoother configuration.

Bug Fixes:
- Multiple bug fixes across the application, including shape-related issues, logic errors, and improved error messaging.
- Fixed a typo and conducted small refactoring in a few areas for code clarity.
- Fixed an indexing issue introduced after backend rework and documentation inaccuracies.
- Addressed a bug with embedding dimension handling and fixed query overloads.
- Rectified numerous minor issues that affected usability and stability.

Documentation and Infrastructure:
- Update README.md with new details and minor typing correction.
- Enhanced user experience with better error messages on file parse.
- Improved type passing and normalization of keyword arguments (kwargs) on expressions.
- Fixed issues regarding parameter passing to conversations.
- Improved shell script for app binary and handled symbolic sub-symbolic representation.
- Added Docker support to simplify the deployment process.

Refactoring:
- Major refactoring for initializing and purging processes, along with normalization of kwargs on expressions.
- Refactored code to improve dynamic context handling and facilitate type selection.
- Performed functional refactoring related to primitives and dynamic operator binding for efficiency and readability.

Miscellaneous Changes:
- Added local image support for the OCR engine.
- Provided more lenient declarative capabilities for callables and primitives.
- Enabled API key and model change arguments for flexible engine configurations.
- Resolved issues regarding path handling in various components such as the crawler and engine initialization.

---

**Note**: This changelog is a summary of the main changes and improvements made to the software, consolidated from the commits over the given period. Users are encouraged to update their versions to benefit from the new features and fixes introduced in this release.