Mindsdb

TL;DR

Updates
* The sdist has been improved to perform better builds.
* Updating the lightwood version has been accomplished to cater to enhanced features and improved performance.
* Git+http dependencies have been revised to include most recent and advantageous changes.

New Features
* Added SQLRetriever, which can now generate pgvector queries with specific metadata filters.
* Introduction of Async embeddings and asyncio support for embeddings for robust and efficient modeling purposes.

Fixes and Optimizations
* The 'extract' function in joins has been corrected for improved result accuracy.
* The async usage in contextual chunking has been fixed for better async performance.
* The issue of deploys being cancelled when one in the list fails is now resolved.
* DataFrame types are now converted to Ints correctly when inserting into a table.
* Inserting DataFrames with explicit NULL values into tables is now possible.
* Several memory optimizations have been implemented for enhanced system efficiency.
* The issue with the metrics decorator not being disabled when not initialized has been fixed.
* HTTP server type and options have been revised for better server management.

What's Changed

* Update sdist to build by ZoranPandovski in https://github.com/mindsdb/mindsdb/pull/10298
* Add SQLRetriever to Generate pgvector Queries With Metadata Filters by tmichaeldb in https://github.com/mindsdb/mindsdb/pull/10299
* Allow Inserting DataFrames With Explicit NULL Values Into Tables by tmichaeldb in https://github.com/mindsdb/mindsdb/pull/10302
* stop deploys being cancelled when one in the list fails by hamishfagg in https://github.com/mindsdb/mindsdb/pull/10308
* Convert DataFrame Types to Ints Correctly When Inserting Into Table by tmichaeldb in https://github.com/mindsdb/mindsdb/pull/10309
* added asyncio support for embeddings by QuantumPlumber in https://github.com/mindsdb/mindsdb/pull/10316
* Async embeddings by QuantumPlumber in https://github.com/mindsdb/mindsdb/pull/10317
* Render frame of window function by ea-rus in https://github.com/mindsdb/mindsdb/pull/10310
* Update lightwood version by ea-rus in https://github.com/mindsdb/mindsdb/pull/10307
* fix async usage in contextual chunking by dusvyat in https://github.com/mindsdb/mindsdb/pull/10320
* Memory optimizations by StpMax in https://github.com/mindsdb/mindsdb/pull/10303
* Disable metrics decorator if not initialized by StpMax in https://github.com/mindsdb/mindsdb/pull/10321
* Set HTTP server type and options via by StpMax in https://github.com/mindsdb/mindsdb/pull/10313
* Fix 'extract' function in joins by ea-rus in https://github.com/mindsdb/mindsdb/pull/10312
* Update git+http dependencies by ZoranPandovski in https://github.com/mindsdb/mindsdb/pull/10325

**Full Changelog**: https://github.com/mindsdb/mindsdb/compare/v24.12.3.0...v24.12.4.0

TL;DR

This release contains significant changes in a number of sectors:

**Documentation:**
Documentation has received several updates detailing the lack of file size limit, information about thread safety in data integrations, custom configuration with HTTP authentication, and updates to the REST API for chatbot.

**Features and Enhancements:**
A custom vllm embedding class has been added for langchain_embedding_handler. Support has been provided for Nomic embedding text formatting. In addition, chatbot creation and updates can now utilize database names via API. A feature for smart release notes has been implemented.

**Bug Fixes and Improvements:**
Installation error related to httpfs in Amazon S3 integration has been fixed. The redshift handler issue has been resolved. Typos in the README file have been amended. A situation where Vector Database Tables were being dropped while deleting the Knowledge Base has been addressed.

**Other Notable Changes:**
The Redirect URI has been improved for executing the MS Authentication flow. The class has been improved and now permits logging to files. It is now possible to see types in the information schema. There is now feature to enable copy storage while creating databases via the API.

What's Changed

* docs - update no file size limit by martyna-mindsdb in https://github.com/mindsdb/mindsdb/pull/10275
* add custom vllm embedding class for langchain_embedding_handler by dusvyat in https://github.com/mindsdb/mindsdb/pull/10271
* docs: added thread_safe info to data integrations by martyna-mindsdb in https://github.com/mindsdb/mindsdb/pull/10267
* Fixed httpfs Installation Error in Amazon S3 Integration by MinuraPunchihewa in https://github.com/mindsdb/mindsdb/pull/10278
* Enabled the Use of Database Name in Creating and Updating Chat-bots via API by MinuraPunchihewa in https://github.com/mindsdb/mindsdb/pull/10277
* docs for custom config - http auth by martyna-mindsdb in https://github.com/mindsdb/mindsdb/pull/10272
* docs - rest api for chatbots update by martyna-mindsdb in https://github.com/mindsdb/mindsdb/pull/10282
* Fix redshift handler by ea-rus in https://github.com/mindsdb/mindsdb/pull/10283
* Add support for Nomic embedding text formatting by dusvyat in https://github.com/mindsdb/mindsdb/pull/10284
* Add smart release notes by hamishfagg in https://github.com/mindsdb/mindsdb/pull/10280
* Added the Redirect URI in Executing the MS Authentication Flow by MinuraPunchihewa in https://github.com/mindsdb/mindsdb/pull/10281
* fix typos in README by cliffordp in https://github.com/mindsdb/mindsdb/pull/10285
* Don't Drop Vector Database Table When Deleting Knowledge Base by tmichaeldb in https://github.com/mindsdb/mindsdb/pull/10292
* Improve class and allow logging to file by StpMax in https://github.com/mindsdb/mindsdb/pull/10287
* Added a More Reliable Means of Generating Redirect URI for MS Authentication Flow by MinuraPunchihewa in https://github.com/mindsdb/mindsdb/pull/10296
* Show types in information schema by ea-rus in https://github.com/mindsdb/mindsdb/pull/10288
* Enabled Copy Storage for Creating Database via API by MinuraPunchihewa in https://github.com/mindsdb/mindsdb/pull/10294

New Contributors

* cliffordp made their first contribution in https://github.com/mindsdb/mindsdb/pull/10285

**Full Changelog**: https://github.com/mindsdb/mindsdb/compare/v24.12.2.1...v24.12.3.0

lightwood version updated: https://github.com/mindsdb/lightwood/pull/1244

Changes: https://github.com/mindsdb/mindsdb/compare/v24.12.2.1...v24.12.2.2

What's Changed

* Add User Input to Map Reduce Summarizer Chain by tmichaeldb in https://github.com/mindsdb/mindsdb/pull/10269
* Don't Register Vectors for PGVector on Every Query by tmichaeldb in https://github.com/mindsdb/mindsdb/pull/10270
* Hide sensitive info from logging by StpMax in https://github.com/mindsdb/mindsdb/pull/10265
* docs: http auth lifetime param by martyna-mindsdb in https://github.com/mindsdb/mindsdb/pull/10266
* Sample db docs by martyna-mindsdb in https://github.com/mindsdb/mindsdb/pull/10273
* Fix: 'files' database is not created automatically by ea-rus in https://github.com/mindsdb/mindsdb/pull/10274



**Full Changelog**: https://github.com/mindsdb/mindsdb/compare/v24.12.2.0...v24.12.2.1

What's Changed

* Summary preprocessor by QuantumPlumber in https://github.com/mindsdb/mindsdb/pull/10232
* fix: Broken Link(for slack) syntax by AkashJana18 in https://github.com/mindsdb/mindsdb/pull/10238
* docs: langchain provider setup by martyna-mindsdb in https://github.com/mindsdb/mindsdb/pull/10240
* Set model status to 'error' if training process is missed by StpMax in https://github.com/mindsdb/mindsdb/pull/10217
* Refactor document preprocessor and enhance logging for debug by dusvyat in https://github.com/mindsdb/mindsdb/pull/10239
* Upgrade multipart by ZoranPandovski in https://github.com/mindsdb/mindsdb/pull/10245
* Text chunk model by QuantumPlumber in https://github.com/mindsdb/mindsdb/pull/10247
* Speed up of first query time by ea-rus in https://github.com/mindsdb/mindsdb/pull/10230
* Add VLLM embedding tests and improve configuration logic by dusvyat in https://github.com/mindsdb/mindsdb/pull/10249
* Bumped `scikit-learn` to 1.5.0 by MinuraPunchihewa in https://github.com/mindsdb/mindsdb/pull/10234
* Add `max_restart_interval_seconds` config option by StpMax in https://github.com/mindsdb/mindsdb/pull/10241
* LLM function: get api key from LLM_FUNCTION_API_KEY by ea-rus in https://github.com/mindsdb/mindsdb/pull/10252
* docs: llm() function updates by martyna-mindsdb in https://github.com/mindsdb/mindsdb/pull/10203
* Fixed the Errors Associated with Importing Handler Modules by MinuraPunchihewa in https://github.com/mindsdb/mindsdb/pull/10233
* added list sheets and select sheets for Excel files by parthiv11 in https://github.com/mindsdb/mindsdb/pull/10246
* Slack Integration Improvements by MinuraPunchihewa in https://github.com/mindsdb/mindsdb/pull/10227
* docs: custom config - restart by martyna-mindsdb in https://github.com/mindsdb/mindsdb/pull/10257
* Ensure Embeddings model is used from knowledge Base by dusvyat in https://github.com/mindsdb/mindsdb/pull/10258
* Added Unit Tests for Slack Tables and Fixed a Few Bugs by MinuraPunchihewa in https://github.com/mindsdb/mindsdb/pull/10261
* Make HTTP session lifetime configurable via env vars by StpMax in https://github.com/mindsdb/mindsdb/pull/10259
* Snowflake fixes for TPC benchmarks by ea-rus in https://github.com/mindsdb/mindsdb/pull/10251
* Slack Documentation Improvements by MinuraPunchihewa in https://github.com/mindsdb/mindsdb/pull/10250
* Test render using parser by ea-rus in https://github.com/mindsdb/mindsdb/pull/10263
* Added MindsDB Java SDK as community SDK by md-abid-hussain in https://github.com/mindsdb/mindsdb/pull/10100
* Fix: Table not found for column (don't check alias of a query) by ea-rus in https://github.com/mindsdb/mindsdb/pull/10254
* docs: files into integrations by martyna-mindsdb in https://github.com/mindsdb/mindsdb/pull/10256
* Fix TS with CTE by ea-rus in https://github.com/mindsdb/mindsdb/pull/10235


New Contributors

* md-abid-hussain made their first contribution in https://github.com/mindsdb/mindsdb/pull/10100

**Full Changelog**: https://github.com/mindsdb/mindsdb/compare/v24.12.1.0...v24.12.2.0

What's Changed

* Add auto-heal to auto-restart unhealthy containers in docker-compose by hamishfagg in https://github.com/mindsdb/mindsdb/pull/10200
* Change type of columns `llm_log.input` and `llm_log.output` to JSON by StpMax in https://github.com/mindsdb/mindsdb/pull/10224
* docs: knowledge base sql docs updates by martyna-mindsdb in https://github.com/mindsdb/mindsdb/pull/10204



**Full Changelog**: https://github.com/mindsdb/mindsdb/compare/v24.11.4.2...v24.12.1.0