[0.14.0](https://github.com/recognai/rubrix/compare/v0.13.3...v0.14.0) (2022-05-10)
Async version of `rb.log`
You can now use the parameter `background` in the `rb.log` method to log records without blocking the main process. The main use case is monitoring production pipelines to do prediction monitoring. Here's an example with BentoML (you can find the full example in the updated [Monitoring guide](https://rubrix.readthedocs.io/en/v0.14.0/guides/monitoring.html#Using-rb.log-in-background-mode)):
python
from bentoml import BentoService, api, artifacts, env
from bentoml.adapters import JsonInput
from bentoml.frameworks.spacy import SpacyModelArtifact
import rubrix as rb
import spacy
nlp = spacy.load("en_core_web_sm")
env(infer_pip_packages=True)
artifacts([SpacyModelArtifact("nlp")])
class SpacyNERService(BentoService):
api(input=JsonInput(), batch=True)
def predict(self, parsed_json_list):
result, rb_records = ([], [])
for index, parsed_json in enumerate(parsed_json_list):
doc = self.artifacts.nlp(parsed_json["text"])
prediction = [{"entity": ent.text, "label": ent.label_} for ent in doc.ents]
rb_records.append(
rb.TokenClassificationRecord(
text=doc.text,
tokens=[t.text for t in doc],
prediction=[
(ent.label_, ent.start_char, ent.end_char) for ent in doc.ents
],
)
)
result.append(prediction)
rb.log(
name="monitor-for-spacy-ner",
records=rb_records,
tags={"framework": "bentoml"},
background=True,
verbose=False
) By using the background=True, the model latency won't be affected
return result
Confidence scores in Token Classification (NER)
To store entity predictions you can attach a score using the last position of the entity tuple `(label, char_start, char_end, score)`. Let's see an example:
python
import rubrix as rb
text = "Rubrix is a data science tool"
record = rb.TokenClassificationRecord(
text=text,
tokens=text.split(" "),
prediction=[("PRODUCT", 0, 6, 0.99)]
)
rb.log(record, "ner_with_scores")
Then, in the web application, you and your team can use the score filter to find potentially problematic entities, like in the screenshot below:
<img width="1587" alt="Screenshot 2022-05-12 at 11 49 43" src="https://user-images.githubusercontent.com/1107111/168043415-ea52354d-24aa-407f-a34e-474f30b55883.png">
If you want to see this in action, check this blog post by David Berenstein:
https://www.rubrix.ml/blog/concise-concepts-rubrix/
Rule metrics sidebar
We have a fresh new sidebar for the weak labeling mode, where you can see your overall rule metrics as you define new rules.
This sidebar should help you quickly understand your progress:
<img width="1572" alt="Screenshot 2022-05-12 at 11 52 10" src="https://user-images.githubusercontent.com/1107111/168043851-abcfa5d4-44c0-4c1b-b2e0-ac2146a76875.png">
See the updated user guide here: https://rubrix.readthedocs.io/en/v0.14.0/reference/webapp/define_rules.html
Features
* **1132:** introduce async/background version of rb.log ([1391](https://github.com/recognai/rubrix/issues/1391)) ([900307e](https://github.com/recognai/rubrix/commit/900307e8fd73427753676499ba8821643bcec252)), closes [#1132](https://github.com/recognai/rubrix/issues/1132)
* **1247:** label models predict method returns DatasetForTextClassification ([1442](https://github.com/recognai/rubrix/issues/1442)) ([42ca1be](https://github.com/recognai/rubrix/commit/42ca1be3b109ba248fe85a761170075e4622dad2)), closes [#1247](https://github.com/recognai/rubrix/issues/1247)
* **1379:** show prediction score in NER ([1389](https://github.com/recognai/rubrix/issues/1389)) ([0bdccd2](https://github.com/recognai/rubrix/commit/0bdccd2c07e354b33f693453db3e767ec2978a53)), closes [#1379](https://github.com/recognai/rubrix/issues/1379) [#1451](https://github.com/recognai/rubrix/issues/1451)
* **961:** rules metrics in sidebar ([1377](https://github.com/recognai/rubrix/issues/1377)) ([261f53a](https://github.com/recognai/rubrix/commit/261f53adb35a95666a8a3978f881b9045ff77e09)), closes [#961](https://github.com/recognai/rubrix/issues/961) [#1408](https://github.com/recognai/rubrix/issues/1408)
* **home:** improve table actions and styles ([1384](https://github.com/recognai/rubrix/issues/1384)) ([f09746e](https://github.com/recognai/rubrix/commit/f09746e7cb30ef7250575589dcf316fe6b01efb1)), closes [#1355](https://github.com/recognai/rubrix/issues/1355) [#1333](https://github.com/recognai/rubrix/issues/1333)
Bug Fixes
* **1407:** fix visualization in 1024px viewport ([1420](https://github.com/recognai/rubrix/issues/1420)) ([46f8d4d](https://github.com/recognai/rubrix/commit/46f8d4d33149dcfefeb175e7e798ec2acca47dd6)), closes [#1441](https://github.com/recognai/rubrix/issues/1441)
* **1458:** token classifier visualization in Safari ([1459](https://github.com/recognai/rubrix/issues/1459)) ([01cc492](https://github.com/recognai/rubrix/commit/01cc49236d3a517f1743b1654824940617810518)), closes [#1458](https://github.com/recognai/rubrix/issues/1458)