Argilla

* feat: small UI improvements by leiyre in https://github.com/argilla-io/argilla/pull/4770
* feat:update UI for settings page by leiyre in https://github.com/argilla-io/argilla/pull/4767
* Fix: "cannot import name 'formatargspec' from 'inspect'" with Python 3.11 by walter-hernandez in https://github.com/argilla-io/argilla/pull/4693
* 🐛 Ranking component not showing rankings by damianpumar in https://github.com/argilla-io/argilla/pull/4775
* Adding LlamaIndex docs to integrations by ignacioct in https://github.com/argilla-io/argilla/pull/4803
* docs: use FeedbackDataset in HF example by sdiazlor in https://github.com/argilla-io/argilla/pull/4805
* docs: clarification/typo in tutorial by sdiazlor in https://github.com/argilla-io/argilla/pull/4810
* Log if a dataset is deleted by paulbauriegel in https://github.com/argilla-io/argilla/pull/4752
* ✨ Search text filtering by field by damianpumar in https://github.com/argilla-io/argilla/pull/4771
* ✨ Add text search for fields by damianpumar in https://github.com/argilla-io/argilla/pull/4831
* ✨ Fix shift issue and Letter S on issue reported by damianpumar in https://github.com/argilla-io/argilla/pull/4836
* 🚑 Fix issue for intentional submission by damianpumar in https://github.com/argilla-io/argilla/pull/4840
* ci: Mono repo setup by frascuchon in https://github.com/argilla-io/argilla/pull/4742
* fix: add branches and tags to argilla-server.yml GitHub workflow by jfcalvo in https://github.com/argilla-io/argilla/pull/4854
* fix: GitHub action names with typos by jfcalvo in https://github.com/argilla-io/argilla/pull/4850
* fix: remove non necessary conditional to build argilla-server docker images by jfcalvo in https://github.com/argilla-io/argilla/pull/4855
* chore: update datasets.py by eltociear in https://github.com/argilla-io/argilla/pull/4842
* docs: Fix typo Argila -> Argilla by louisguitton in https://github.com/argilla-io/argilla/pull/4870
* fix: add error code when searching for a record missing specific vector by jfcalvo in https://github.com/argilla-io/argilla/pull/4856
* 🐛 Fix highlight multiple fields by damianpumar in https://github.com/argilla-io/argilla/pull/4866
* feat: add support for value zero on rating questions by jfcalvo in https://github.com/argilla-io/argilla/pull/4864
* fix(import): remove non-existent server module by frascuchon in https://github.com/argilla-io/argilla/pull/4874
* 🐛 Fix pre selection by damianpumar in https://github.com/argilla-io/argilla/pull/4872
* support for Python 3.12 by nicoloboschi in https://github.com/argilla-io/argilla/pull/4837
* Search bar and highlight docs by nataliaElv in https://github.com/argilla-io/argilla/pull/4882
* feat: UI Metadata info component by leiyre in https://github.com/argilla-io/argilla/pull/4851
* [IMPROVEMENT] Update pip when building docker image by frascuchon in https://github.com/argilla-io/argilla/pull/4907
* [BUGFIX] Filter record metadata value based on metadata property policies by frascuchon in https://github.com/argilla-io/argilla/pull/4906
* feat: UI - metadata adjustments by leiyre in https://github.com/argilla-io/argilla/pull/4905
* [REVIEW] Add missing entries in CHANGELOG files by frascuchon in https://github.com/argilla-io/argilla/pull/4910

New Contributors
* walter-hernandez made their first contribution in https://github.com/argilla-io/argilla/pull/4693
* eltociear made their first contribution in https://github.com/argilla-io/argilla/pull/4842
* louisguitton made their first contribution in https://github.com/argilla-io/argilla/pull/4870
* nicoloboschi made their first contribution in https://github.com/argilla-io/argilla/pull/4837

**Full Changelog**: https://github.com/argilla-io/argilla/compare/v1.28.0...v1.29.0

🔆 Release highlights

Improved suggestions

https://github.com/argilla-io/argilla/assets/126158523/380004e0-28cb-409f-b11c-71d0e3b6e8bf

Multiple scores support for `MultiLabelQuestion` and `RankingQuestion`

`MultiLabelQuestion` and `RankingQuestion` now take one score per suggested label / value, making the scores easier to interpret. Learn more about suggestions and their scores [here](https://docs.argilla.io/en/develop/practical_guides/create_update_dataset/suggestions_and_responses.html#format-suggestions).

> [!WARNING]
> If you upgrade to this version all previous scores in suggestions for MultiLabelQuestion, RankingQuestion and SpanQuestion will turn to NULL, as they will not be valid in the new schema. Please, make sure you upload scores again if you want to use them.

See scores next to its label / value

Scores are now shown next to its label / value in all questions. This makes them more visible and easier to interpret.

Suggestions first - 🌟 Community request: https://github.com/argilla-io/argilla/issues/4647

Now you can order labels in `MultiLabelQuestion` so that suggestions are always shown first. This will help you make sure that the most relevant labels are always at hand. Plus, if you’ve added scores to your labels, these will be ordered in descending order. To enable this, go to the Dataset Settings page > Questions and enable “Suggestions first” for the desired question.

`SpanQuestion` improvements

https://github.com/argilla-io/argilla/assets/126158523/fad7b9ca-3890-45ed-acc8-5b038a81db06

Pre-selection highlight

We’ve improved the way selections are shown. You can now see a highlight that represents what the final selection will look like while you’re dragging your mouse. This will help you with the selection speed and show you the difference between the token vs character selection.
> [!NOTE]
> Remember that character-level spans are activated by holding `Shift` while doing the selection.

New label selector

We’ve improved the way the label selector works in the `SpanQuestion` when overlapping spans are enabled so it’s easier to add or correct labels. Simply click on the desired span to activate the selector and click on the label(s) that you want to add or remove.

Persistent storage warning

We’ve added a warning for Argilla instances deployed on Hugging Face Spaces to alert of data loss when the persistent storage is not enabled.

To learn more about this warning and how to disable it, go to [our docs](https://docs.argilla.io/en/develop/getting_started/installation/deployments/huggingface-spaces.html#setting-up-persistent-storage).

[Changelog 1.28.0](https://github.com/argilla-io/argilla/compare/v1.27.0...v1.28.0)

Added

- Added suggestion multi score attribute. ([4730](https://github.com/argilla-io/argilla/pull/4730))
- Added order by suggestion first. ([4731](https://github.com/argilla-io/argilla/pull/4731))
- Added multi selection entity dropdown for span annotation overlap. ([4735](https://github.com/argilla-io/argilla/pull/4735))
- Added pre selection highlight for span annotation. ([4726](https://github.com/argilla-io/argilla/pull/4726))
- Added banner when persistent storage is not enabled. ([4744](https://github.com/argilla-io/argilla/pull/4744))
- Added support on Python SDK for new multi-label questions `labels_order` attribute. ([4757](https://github.com/argilla-io/argilla/pull/4757))

Changed

- Changed the way how Hugging Face space and user is showed in sign in. ([4748](https://github.com/argilla-io/argilla/pull/4748))

Fixed

- Fixed Korean character reversed. ([4753](https://github.com/argilla-io/argilla/pull/4753))

Fixed

- Fixed requirements for version of wrapt library conflicting with Python 3.11 ([4693](https://github.com/argilla-io/argilla/pull/4693))

**Full Changelog**: https://github.com/argilla-io/argilla/compare/v1.27.0...v1.28.0

🔆 Release highlights

Overlapping spans

We are finally releasing a much expected feature: overlapping spans. This allows you to draw more than one span over the same token(s)/character(s).

https://github.com/argilla-io/argilla/assets/126158523/3aeb6c6c-b348-4b3d-be67-483636c76293

To try them out, set up a `SpanQuestion` with the argument `allow_overlap=True` like this:

python
dataset = rg.FeedbackDataset(
fields = [rg.TextField(name="text")]
questions = [
rg.SpanQuestion(
name="spans",
labels=["label1", "label2", "label3"],
field="text"
)
]
)


Learn more about configuring this and other question types [here](https://docs.argilla.io/en/latest/practical_guides/create_update_dataset/create_dataset.html#define-questions).

Global progress bars

We’ve included a new column in our home page that offers the global progress of your datasets, so that you can see at a glance what datasets are closer to completion.

<img width="1624" alt="Captura de pantalla 2024-04-17 a las 14 27 32" src="https://github.com/argilla-io/argilla/assets/126158523/d0cfa75f-6744-4e5c-be49-6e9eac5cd58e">

These bars show progress by grouping records based on the status of their responses:

- Submitted: Records where all responses have the `submitted` status.
- Discarded: Records where all responses have the `discarded` status.
- Conflicting: Records with at least one `submitted` and one `discarded` response.
- Left: All other records that have no `submitted` or `discarded` responses. These may be in `pending` or `draft` .

Suggestions got a new look

We’ve improved the way suggestions are shown in the UI to make their purpose clearer: now you can identify each suggestion with a sparkle icon ✨ .

The behavior is still the same:

- suggested values will appear pre-filled responses and marked with the sparkle icon.
- make changes the the incorrect suggestions, then save as a draft or submit.
- the icon will stay to mark the suggestions so you can compare the final response with the suggested one.

Increased label limits

We’ve increased the limit of labels you can use in Label, Multilabel and Span questions to 500. If you need to go beyond that number, you can set up a custom limit using the following environment variables:

- `ARGILLA_LABEL_SELECTION_OPTIONS_MAX_ITEMS` to set the limits in label and multi label questions.
- `ARGILLA_SPAN_OPTIONS_MAX_ITEMS` to set the limit in span questions.

> [!WARNING]
> The UI has been optimized to support up to 1000 labels. If you go beyond this limit, the UI may not be as responsive.

Learn more about this and other environment variables [here](https://docs.argilla.io/en/latest/getting_started/installation/configurations/server_configuration.html#environment-variables).

Argilla auf Deutsch!

Thanks to our contributor paulbauriegel you can now use Argilla fully in German! If that is the main language of your browser, there is nothing you need to do, the UI will automatically detect that and switch to German.

Would you like to translate Argilla to your own language? Reach out to us and we'll help you!

[Changelog 1.27.0](https://github.com/argilla-io/argilla/compare/v1.26.1...v1.27.0)

Added

- Added Allow overlap spans in the `FeedbackDataset` ([4668](https://github.com/argilla-io/argilla/pull/4668))
- Added `allow_overlapping` parameter for span questions. ([4697](https://github.com/argilla-io/argilla/pull/4697))
- Added overall progress bar on `Datasets` table ([4696](https://github.com/argilla-io/argilla/pull/4696))
- Added German language translation ([4688](https://github.com/argilla-io/argilla/pull/4688))

Changed

- New UI design for suggestions ([4682](https://github.com/argilla-io/argilla/pull/4682))

Fixed

- Improve performance for more than 250 labels ([4702](https://github.com/argilla-io/argilla/pull/4702))

New Contributors
* stevengans made their first contribution in https://github.com/argilla-io/argilla/pull/4646
* tim-win made their first contribution in https://github.com/argilla-io/argilla/pull/4672
* strickvl made their first contribution in https://github.com/argilla-io/argilla/pull/4675
* paulbauriegel made their first contribution in https://github.com/argilla-io/argilla/pull/4688
* davanstrien made their first contribution in https://github.com/argilla-io/argilla/pull/4687

**Full Changelog**: https://github.com/argilla-io/argilla/compare/v1.26.1...v1.27.0

[1.26.1](https://github.com/argilla-io/argilla/compare/v1.26.0...v1.26.1)

Added

- Added support for automatic detection of RTL languages. ([4686](https://github.com/argilla-io/argilla/pull/4686))


**Full Changelog**: https://github.com/argilla-io/argilla/compare/v1.26.0...v1.26.1

🔆 Release highlights
Spans question
We've added a new type of question to Feedback Datasets: the `SpanQuestion`. This type of question allows you to highlight portions of text in a specific field and apply a label. It is specially useful for token classification (like NER or POS tagging) and information extraction tasks.

https://github.com/argilla-io/argilla/assets/126158523/d3821d49-6da0-4488-99e2-068d7411268a

With this type of question you can:

✨ Provide suggested spans with a confidence score, so your team doesn't need to start from scratch.

⌨️ Choose a label using your mouse or with the keyboard shortcut provided next to the label.

🖱️ Draw a span by dragging your mouse over the parts of the text you want to select or if it's a single token, just double-click on it.

🪄 Forget about mistakes with token boundaries. The UI will snap your spans to token boundaries for you.

🔎 Annotate at character-level when you need more fine-grained spans. Hold the `Shift` key while drawing the span and the resulting span will start and end in the exact boundaries of your selection.

✔️ Quickly change the label of a span by clicking on the label name and selecting the correct one from the dropdown.

🖍️ Correct a span at the speed of light by simply drawing the correct span over it. The new span will overwrite the old one.

🧼 Remove labels by hovering over the label name in the span and then click on the 𐢫 on the left hand side.

Here's an example of what your dataset would look like from the SDK:

python
import argilla as rg
from argilla.client.feedback.schemas import SpanValueSchema

connect to your Argilla instance
rg.init(...)

create a dataset with a span question
dataset = rg.FeedbackDataset(
fields=[rg.TextField(name="text"),
questions=[
rg.SpanQuestion(
name="entities",
title="Highlight the entities in the text:",
labels={"PER": "Person", "ORG": "Organization", "EVE": "Event"}, or ["PER", "ORG", "EVE"]
field="text", the field where you want to do the span annotation
required=True
)
]
)

create a record with suggested spans
record = rg.FeedbackRecord(
fields={"text": "This is the text of the record"}
suggestions = [
{
"question_name": "entities",
"value": [
SpanValueSchema(
start=0, position of the first character of the span
end=10, position of the character right after the end of the span
label="ORG",
score=1.0
)
],
"agent": "my_model",
}
]
)

add records to the dataset and push to Argilla
dataset.add_records([record])
dataset.push_to_argilla(...)


To learn more about this and all the other questions available in Feedback Datasets, check out our documentation on:
- [Defining questions](https://docs.argilla.io/en/latest/practical_guides/create_update_dataset/create_dataset.html#define-questions)
- [Working with suggestions and responses](https://docs.argilla.io/en/latest/practical_guides/create_update_dataset/suggestions_and_responses.html)
- [Annotating Feedback Datasets](https://docs.argilla.io/en/latest/practical_guides/annotate_dataset.html#feedback-dataset)

[Changelog 1.26.0](https://github.com/argilla-io/argilla/compare/v1.25.0...v1.26.0)

Added

- If you expand the labels of a `single or multi` label Question, the state is maintained during the entire annotation process. ([4630](https://github.com/argilla-io/argilla/pull/4630))
- Added support for span questions in the Python SDK. ([4617](https://github.com/argilla-io/argilla/pull/4617))
- Added support for span values in suggestions and responses. ([4623](https://github.com/argilla-io/argilla/pull/4623))
- Added `span` questions for `FeedbackDataset`. ([4622](https://github.com/argilla-io/argilla/pull/4622))
- Added `ARGILLA_CACHE_DIR` environment variable to configure the client cache directory. ([4509](https://github.com/argilla-io/argilla/pull/4509))

Fixed

- Fixed contextualized workspaces. ([4665](https://github.com/argilla-io/argilla/pull/4665))
- Fixed prepare for training when passing `RankingValueSchema` instances to suggestions. ([4628](https://github.com/argilla-io/argilla/pull/4628))
- Fixed parsing ranking values in suggestions from HF datasets. ([4629](https://github.com/argilla-io/argilla/pull/4629))
- Fixed reading description from API response payload. ([4632](https://github.com/argilla-io/argilla/pull/4632))
- Fixed pulling (n\*chunk_size)+1 records when using `ds.pull` or iterating over the dataset. ([4662](https://github.com/argilla-io/argilla/pull/4662))
- Fixed client's resolution of enum values when calling the Search and Metrics api, to support Python >=3.11 enum handling. ([4672](https://github.com/argilla-io/argilla/pull/4672))

New Contributors
* davidefiocco made their first contribution in https://github.com/argilla-io/argilla/pull/4639

**Full Changelog**: https://github.com/argilla-io/argilla/compare/v1.25.0...v1.26.0

🔆 Release highlights

Reorder labels
`admin` and `owner` users can now change the order in which labels appear in the question form. To do this, go to the `Questions` tab inside Dataset Settings and move the labels until they are in the desired order.

https://github.com/argilla-io/argilla/assets/126158523/40f382a5-35c6-4bea-b15c-f001f539940d

Aligned SDK status filter
The `missing` status has been removed from the SDK filters. To filter records that don't have responses you will now need to use the `pending` status like so:

python
filtered_dataset = dataset.filter_by(response_status="pending")

Learn more about how to use this filter in [our docs](https://docs.argilla.io/en/latest/practical_guides/filter_dataset.html#by-status)

Pandas 2.0 support

We’ve removed the limitation to use pandas `<2.0.0` so you can now use Argilla with pandas v1 or v2 safely.

[Changelog 1.25.0](https://github.com/argilla-io/argilla/compare/v1.24.0...v1.25.0)

> [!NOTE]
> For changes in the argilla-server module, visit the argilla-server [release notes](https://github.com/argilla-io/argilla-server/releases/tag/v1.25.0)


Added

- Reorder labels in `dataset settings page` for single/multi label questions ([4598](https://github.com/argilla-io/argilla/pull/4598))
- Added pandas v2 support using the python SDK. ([4600](https://github.com/argilla-io/argilla/pull/4600))

Removed

- Removed `missing` response for status filter. Use `pending` instead. ([4533](https://github.com/argilla-io/argilla/issues/4533))

Fixed

- Fixed FloatMetadataProperty: value is not a valid float ([4570](https://github.com/argilla-io/argilla/pull/4605))
- Fixed redirect to `user-settings` instead of 404 `user_settings` ([4609](https://github.com/argilla-io/argilla/pull/4609))

New Contributors
* julien-c made their first contribution in https://github.com/argilla-io/argilla/pull/4582
* 7flash made their first contribution in https://github.com/argilla-io/argilla/pull/4504

**Full Changelog**: https://github.com/argilla-io/argilla/compare/v1.24.0....v1.25.0