R2r

Adds in contextual enrichment, which you can learn more about here: https://r2r-docs.sciphi.ai/documentation/contextual-enrichment

What's Changed
* Reintroduce contextual enrichment by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1832
* fix graph collection filtering by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1834
* fix sim. calc and return by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1835
* fix relationship filters by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1836
* Bump LiteLLM by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1837


**Full Changelog**: https://github.com/SciPhi-AI/R2R/compare/v3.3.28...v3.3.29

Introduces [deduplication](https://r2r-docs.sciphi.ai/documentation/deduplication), automatic extraction and deduplication, as well as a number of bug fixes.

What's Changed
* Feature/add auth callback by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1809
* Add await to ingestion by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1811
* Improve XML Sanitation by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1812
* Introduce LM Studio by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1813
* Bug fix: passing document id to chunks by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1815
* tweak graph permissions by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1817
* force overwrite of graphrag prompts by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1818
* tweak for prod fix by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1819
* Add 'processing' status to document extraction by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1821
* Feature/add auto extract by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1822
* Document Entity Deduplication by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1823
* Admin only for logs websocket by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1826
* Fix document count on collections by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1827
* Feature/fix default client add conversation title by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1830
* Disable automatic extraction and deduplication for local models by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1831


**Full Changelog**: https://github.com/SciPhi-AI/R2R/compare/3.3.27...v3.3.28

A number of bug fixes, introduction of more user management configurations, and refined graph processes to better support local LLMs.


What's Changed
* Feature/fix zerox ingestion by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1659
* Dev minor by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1665
* fix provider defaults issue by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1667
* fix retrieval by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1669
* Fix agent responses for conversation by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1668
* Fix serialization error for Collection UUID going into Hatchet workflow by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1670
* Feature/add document search back by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1672
* fix owner id references by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1673
* Patch/fix filtered deletion logic by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1677
* Add statement_cache_size parameter to allow connections to Supabase by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1676
* up by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1678
* 20,539% speed up in collections overview by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1680
* Nolan/collections migration by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1681
* up by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1679
* Feature/test graphs and indices by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1683
* up by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1684
* Feature/fix multi user search by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1686
* Add document summary to extraction process by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1682
* Nolan/collections summary by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1685
* Feature/clustering as a service by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1687
* Patch/rename to cluster as a service by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1688
* cleanup scale settings by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1689
* Feature/improve collection logic add regr tests by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1692
* merge in nolan + main by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1694
* up by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1695
* checkin limits implementation by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1696
* up by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1697
* delete test cruft by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1700
* up by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1701
* Feature/add systems test for limits by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1703
* Implementing Missing R2R Parsers, Image Ingestion by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1702
* Regenerate lock by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1704

* Add Poppler by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1706
* Include compose.full_with_replicas by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1707
* up by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1709
* Test cloud rate limit by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1711
* Fix rate limit bug on Websockets by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1712
* Improved types by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1710
* Move dict.update (shallow copy) to deep_update by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1714
* Remove chunking strategy from test by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1715
* Conversations Updates by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1716
* Add NaCl Crypto Provider, User API Keys by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1713
* Allow setting conversation name in conversation.create by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1717
* Allow superuser to delete any document by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1719
* Community deletion by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1720
* Ensure name and bio are returned with user, clean up typing across auth by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1722
* Add Permissions to Conversations by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1723
* Move JS SDK to camel case by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1724
* Update JS route examples by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1725
* Fix messages passing in conversations by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1726
* Fix Crypto Providers in Factory by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1727
* Make verify password robust for existing deployments by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1729
* Allow for setting public routes by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1730
* Fix service level type errors by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1732
* Improved CLI by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1708
* Fix lock by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1734
* Feature/add api key support by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1735
* up by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1736
* Feature/change default url by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1738
* Add llms.txt to support modern AI-assisted coding workflows by yigitkonur in https://github.com/SciPhi-AI/R2R/pull/1731
* Run pre-commit by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1741
* Improved typing, Remove old methods by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1743
* Remove V2 SDK by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1744
* Export Functionality by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1740
* Add typing to StoreType by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1745
* Further cleanup, remove old deduplication logic by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1746
* Increase Hatchet step timeouts by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1750
* Add cleanup to documents in tests by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1751
* More test fixes by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1752
* Fix filter logic bugs by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1753
* Revert graph filters by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1754
* Add migration script for user limits overrides by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1755
* Update CLI auth error message, move backoff down to 3 retries by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1759
* Fix document download for JS client by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1761
* save token counts by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1762
* Feature/configurable api base by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1760
* add user limit tracking by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1763
* Fix CSV ordering, run pre-commit by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1764
* RFC 5987 Encoding on content-disposition headers by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1765
* Reset password and verification email to public routes by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1766
* Resend Verification Email HS by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1767
* Fix JS test—expect to throw 400 by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1768
* Clean up bugs around user deletion by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1769
* Remove users.register in favor of users.create by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1770
* Fix deletion of documents with no chunks by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1771
* Add R2R_SECRET_KEY and missing import by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1772
* commit by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1773
* up by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1774
* change default bmp by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1775
* Remove dependency on pyproject.toml in site package by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1777
* Fix site package installation location by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1779
* Fix Dockerfile config location by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1780
* Only log POST and DELETE requests in the request_log table by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1781
* Feature/centralized limit calc and chunk limits by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1782
* Feature/fix password reset js sdk by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1783
* Move from user_id to owner_id for contextual enrichment by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1784
* fix ingestion by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1785
* Update Readme by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1786
* up by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1787
* bump poetry lock by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1788
* Fix R2RExceptions, run pre-commit, remove password validation by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1789
* NaN Embedding Dimension Support by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1792
* up by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1793
* User Registration with Profile Fields by logerzerox in https://github.com/SciPhi-AI/R2R/pull/1790
* Password change email notification by logerzerox in https://github.com/SciPhi-AI/R2R/pull/1791
* Add support for Gemini by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1794
* get not post by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1795
* add logic for github auth by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1796
* up by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1797
* Adding auto-gen email fallback for github oauth by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1798
* fix oauth collection bug by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1799
* impl user max collections by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1800
* up by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1801
* cap streaming chunk size by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1802
* Feature/fix app config loadout by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1804
* Improve graph workflow, add better support for local LLMs by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1803
* upsert document info infront of hatchet by emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/1806
* Add LM Studio Support by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1805
* Move Never from typing to typing_extensions by NolanTrem in https://github.com/SciPhi-AI/R2R/pull/1808

New Contributors
* yigitkonur made their first contribution in https://github.com/SciPhi-AI/R2R/pull/1731

**Full Changelog**: https://github.com/SciPhi-AI/R2R/compare/v3.3.0...3.3.27

With the release of v3.3.0, R2R offers a completely RESTful API that covers everything you need for production RAG applications. The biggest change is our Git-like knowledge graph architecture, but we've also unified all the core objects you need to build real applications.

**📚 Complete API Coverage:**

1️⃣ **Content & Knowledge**
• `Documents`: Upload files, manage content, and track extraction status
• `Chunks`: Access and search vectorized text segments
• `Graphs`: Git-like knowledge graphs with:
↳ Entities & Relationships
↳ Automatic community detection
↳ Independent graphs per collection

2️⃣ **Infrastructure**
• `Indices`: Manage vector indices for search optimization
• `Collections`: Organize documents and share access
• `Users`: Built-in auth and permission management
• `Conversations`: Track chat history and manage branches

3️⃣ **Retrieval & Generation**
• `RAG`: Configurable retrieval pipeline with hybrid search
• `Agents`: Conversational interfaces with knowledge graph integration
• `Search`: Vector, keyword, and knowledge graph search

**💻 Quick Example:**
python
from r2r import R2RClient
client = R2RClient("http://localhost:7272")

Document level extraction
client.documents.extract(document_id)

Collection level graph management
client.graphs.pull(collection_id)

Advanced RAG with everything enabled
response = client.retrieval.rag(
"Your question here",
search_settings={
"use_hybrid_search": True,
"graph_settings": {"enabled": True}
}
)


All these components work together seamlessly - just configure what you need and R2R handles the rest. Perfect for teams building serious RAG applications.

🔗 Check the API: https://r2r-docs.sciphi.ai/api-and-sdks/introduction

We'd love feedback from folks building in production!

Among a number of bug fixes, scaling, and performance improvements, there are a few notable features and considerations.

**Migrations**
With this release, we introduce a number of changes that require migrations. To run migrations, you can update to the latest version of R2R with `r2r update`, upon serving R2R (e.g. `r2r serve --docker`) the schema will be checked, and you will be prompted to upgrade your database with `r2r db upgrade.`

**Advanced GraphRAG**
We're rolling out [prompt tuning](https://r2r-docs.sciphi.ai/cookbooks/advanced-graphrag#prompt-tuning) and [contextual embeddings](https://r2r-docs.sciphi.ai/cookbooks/advanced-graphrag#contextual-chunk-enrichment), which both offer significant improvements.