Neo4j-graphrag

Added

- Added the `run_with_context` method to `Component`. This method includes a `context_` parameter, which provides information about the pipeline from which the component is executed (e.g., the `run_id`). It also enables the component to send events to the pipeline's callback function.

Fixed

- Added `enforce_schema` parameter to `SimpleKGPipeline` for optional schema enforcement.

Added

- Added optional schema enforcement as a validation layer after entity and relation extraction.
- Introduced a linear hybrid search ranker for HybridRetriever and HybridCypherRetriever, allowing customizable ranking with an `alpha` parameter.
- Introduced SearchQueryParseError for handling invalid Lucene query strings in HybridRetriever and HybridCypherRetriever.
- Components can now be called with the `run_with_context` method that gets an extra `context_` argument containing information about the pipeline it's run from: the `run_id`, `task_name` and a `notify` function that can be used to send `TASK_PROGRESS` events to the same callback as the pipeline events.

Fixed

- Fixed config loading after module reload (usage in jupyter notebooks)

Changed

- Qdrant retriever now fallbacks on the point ID if the `external_id_property` is not found in the payload.
- Updated a few dependencies, mainly `pypdf`, `anthropic` and `cohere`.

Added

- Utility functions to retrieve metadata for vector and full-text indexes.
- Support for effective_search_ratio parameter in vector and hybrid searches.
- Introduced upsert_vectors utility function for batch upserting embeddings to vector indexes.
- Introduced `extract_cypher` function to enhance Cypher query extraction and formatting in `Text2CypherRetriever`.
- Introduced Neo4jMessageHistory and InMemoryMessageHistory classes for managing LLM message histories.
- Added examples and documentation for using message history with Neo4j and in-memory storage.
- Updated LLM and GraphRAG classes to support new message history classes.

Changed

- Refactored index-related functions for improved compatibility and functionality.
- Added deprecation warnings to upsert_vector, upsert_vector_on_relationship functions in favor of upsert_vectors.
- Added deprecation warnings to async_upsert_vector, async_upsert_vector_on_relationship functions notifying developers that they will be removed in a future release.
- Added support for database, timeout, and sanitize arguments in schema functions.

Fixed

- Resolved an issue with an incorrectly hard coded node alias in the `_handle_field_filter` function.

Added

- Ability to add event listener to get notifications about Pipeline progress.
- Added py.typed so that mypy knows to use type annotations from the neo4j-graphrag package.
- Support for creating enhanced schemas with detailed property statistics.
- New utility functions for schema formatting and value sanitization.
- Updated unit and integration tests to cover enhanced schema functionality.

Changed

- Changed the default behaviour of `FixedSizeSplitter` to avoid words cut-off in the chunks whenever it is possible.
- Refactored schema creation code to reduce duplication and improve maintainability.

Fixed

- Removed the `uuid` package from dependencies (not needed with Python 3).
- Fixed a bug in the `AnthropicLLM` class preventing it from being used in `GraphRAG` pipeline.

Fixed

- Fix a bug where the `OllamaEmbedder` would return a `list[list[float]]` instead of the expected `list[float]`.

Fixed

Dependencies

- PyYAML dependency was missing and has been added.
- Weaviate was unintentionally added as a mandatory dependency in previous version, this behavior has been reverted.
- PyPDF and fsspec are not optional anymore so that SimpleKGPipeline examples can run out of the box (they just require the independent installation of openai python package if using OpenAI).