Cannlytics

*Published 2023-09-22*

- Continued work on the organization screens in the app.
- Made significant progress on data collection and app.
- WA CCRS sales and strains data curation.
- Designed a data dashboard console and refactored app imports.
- Development of data dashboard and Cannlytics data archive.
- Added AI models, datasets, and sales curation features.
- Worked on API with an OpenAPI specification.
- Made updates to the data archive and developed cannabis license curation algorithms.
- Fixed GIS data and implemented data algorithms for multiple states.
- Resumed development of data app, specifically on subscription management and dashboard features.
- Implemented download state license data functionality.
- Outlined and began generating code for AI data tools.
- Refactored the data app dashboard and worked on CoADoc user interface.
- Implemented receipt parser UI, API, and stabilized website requirements.
- Continued work on licensing features, lab results data availability, and data app map features.
- Began work on FL COA parsing algorithms and made lab results accessible through the data app.
- Implemented lab result search and made various data app improvements.
- Began the algorithm to archive CA producer lab results.
- Refactored the website and the data app.
- Worked on Kaycha Labs COA parsing and refactored documentation.
- Worked on COA PDF parsing and carried out tests.
- Refactored `CannPatent` - Cannabis Strain Identifier.
- Refactored various components of the app, including data app, archive, tests, API, and AI.
- Implemented QR code scan and began parsing COA with AI.
- Integrated COA AI with data app through the API and published it.
- Implemented & tested `BudSpender` AI receipt parser and UI.
- Worked on COA and receipt parsers.
- Focused on uploading receipts and COAs and improving CRUD operations.
- Began implementing payments, tokens, and subscription plans in the app.

*Released 2023-07-04*

- Added starter Flutter architecture for the app.
- Refactored app while preserving core functionality.
- Refactor of app onboarding and sign-in logic/names.
- Architecture for screens & started implementing routing.
- Refactoring of various app screens and added API service.
- Refactored app menu and began implementing app models.
- Added footer and PNG images for the app.
- Various styling and theming updates, including dashboard, footer, and menu style.
- Implementation and refactoring of Metrc service functions for the app.
- Added license functionality and implemented UI controllers for various app screens.
- Continuous work on the app's Metrc screens, account screens, organization screens, and licenses.
- Refactored app routes and started implementing model CRUDs.
- Implemented Metrc actions in the app and various refactors.
- Added app authentication features for smooth sign-in.
- Implemented app location sorting and started adding search functionality.
- Improved app locations table functionality and other enhancements.
- Implemented checkboxes for location form and added location type selection features.
- Improved app's data curation and organization screen features.
- Refactor of app facilities and improvements to `CoADoc` algorithms.
- Refactored Metrc API authentication.
- Implemented Metrc sync and refactored app theme.
- Made updates to app theme and colors.
- Implemented the 'join organization' feature in the app.
- Released app to production.

Cannlytics `v0.0.13` is a major improvement over prior versions that, while still not recommended for large-scale production use, is becoming stable and increasingly useful in development, testing, and staging environments.

Features include:

- `ai`: Minor work has begun on `ai` tools, demonstrating how tools such as `openai` can be used from the Cannlytics API.
- `api`: The Cannlytics API has been improved to facilitate onboarding of Metrc licenses. `ai` API endpoints have also been added.
- `data.ccrs`: Tools to wrangle data from the [Cannabis Compliance and Regulatory System (CCRS)](https://lcb.wa.gov/ccrs), Washington State's traceability system for cannabis. The data is available to the public upon a public records request.
- `data.coas`: The `CoADoc` COA parsing tool continues to be improved and demonstrate its usefulness. The tool still has much room for improvement and future work may see the incorporation of `ai` into tools such as `CoADoc`.
- `metrc`: The `metrc` module has been nearly completed, minus certain endpoints that are not yet standard across states.

A very special thank you to Cannlytics contributors, in this release:
candy-o

This pull request incorporates improvements found during a rigorous testing of `CoADoc`: the parsing of [a dataset of public cannabis lab test results](https://cannlytics.page.link/rawgarden) [published by Raw Garden](https://rawgarden.farm/lab-results/). There's a whole lot of cleaning to be done on this data! For example, there's a `value` of `'<LLoQ'`, and I found 200+ more! If anyone else wants to help look for odd values in any columns or datasets, then below are version `1.0.0` of the Raw Garden lab result data.

1. [`'rawgarden_details'`](https://cannlytics.page.link/?link=https://firebasestorage.googleapis.com/v0/b/cannlytics.appspot.com/o/data%252Flab_results%252Frawgarden%252Fdetails.csv?alt%3Dmedia%26token%3De5b5273a-049a-4092-98d7-90a62ef399a3)
2. [`'rawgarden_results'`](https://cannlytics.page.link/?link=https://firebasestorage.googleapis.com/v0/b/cannlytics.appspot.com/o/data%252Flab_results%252Frawgarden%252Fresults.csv?alt%3Dmedia%26token%3Ddd868e72-edde-4278-9725-b33368a35d54)
3. [`'rawgarden_values'`](https://cannlytics.page.link/?link=https://firebasestorage.googleapis.com/v0/b/cannlytics.appspot.com/o/data%252Flab_results%252Frawgarden%252Fvalues.csv?alt%3Dmedia%26token%3D5d427468-c33e-4e45-ae40-efd10fdca644)

Example usage:

py
import pandas as pd

Read results.
df = pd.read_csv('https://cannlytics.page.link/?link=https://firebasestorage.googleapis.com/v0/b/cannlytics.appspot.com/o/data%252Flab_results%252Frawgarden%252Fresults.csv?alt%3Dmedia%26token%3Ddd868e72-edde-4278-9725-b33368a35d54')

Look at all of the unique values.
unique_values = list(df['value'].unique())
errors = []
for value in unique_values:
try:
pd.to_numeric(value)
except:
errors.append(value)
print(errors)


Furthermore, the new utility function, `create_hash`, found in `cannlytics.utils.utils.py`, needs to be incorporated into `CoADoc`'s `standardize` and perhaps `save` / `aggregate` methods. For now, users can perform hashing for their data as follows.

py
from cannlytics.utils.utils import create_hash
import pandas as pd

Read details.
df = pd.read_csv('https://cannlytics.page.link/?link=https://firebasestorage.googleapis.com/v0/b/cannlytics.appspot.com/o/data%252Flab_results%252Frawgarden%252Fdetails.csv?alt%3Dmedia%26token%3De5b5273a-049a-4092-98d7-90a62ef399a3')

Create hashes for a given DataFrame of details.
df= df.where(pd.notnull(df), None)
df['results_hash'] = df['results'].apply(lambda x: create_hash(x))
df['sample_hash'] = df.loc[:, df.columns != 'sample_hash'].apply(
lambda x: create_hash(x.to_dict()),
axis=1,
)
datafile_hash = create_hash(df)


Ultimately, this version provides improvements to the `CoADoc` class that were discovered during the parsing of more than 2,600 COAs for [Raw Garden products](https://rawgarden.farm/lab-results/). Big thanks to candy-o for testing the parsing!

:tada::partying_face: :tada: Big release today - :disguised_face: [CoADoc](https://cannlytics.com/data/coas) is live!

Big thanks to CandyO who spearheaded testing and who's diligence and attention to detail critically helped with countless improvements. Also, huge thanks to [The CESC](thecesc.org) who supplied the training data that made everything possible. We are blessed for the whole Cannabis Data Science Team, thank you for your help making this tool a reality :pray:

This tool will need extensive testing, as the production environment is always slightly different that development. Right off the bat, it is trickier to parse QR codes due to the needed dependencies. I've adjusted [this Dockerfile](https://github.com/cannlytics/cannlytics/blob/main/website/Dockerfile) to run CoADoc in the same container as the Cannlytics Website. If anyone wants to experiment running CoADoc in a Docker container, then that would be super, super helpful. In particular, the tricky dependency is `zbar`. [Here is a reference](https://stackoverflow.com/questions/48792965/importerror-unable-to-find-zbar-shared-library-on-flask) to a similar issue.

Once battle-tested, CoADoc can parse your CoAs in 3 ways:

1. Through the [user interface](https://cannlytics.com/data/coas).
2. Through the [API](https://github.com/cannlytics/cannlytics/blob/main/api/data/coa_data.py).
3. With the [Python SDK](https://pypi.org/project/cannlytics).

Hopefully you find use for the tool, the resulting lab result, and potentially business opportunities for onboarding new labs and LIMS. The sky is the limit :rocket: Thank you all!

This is the first publication of the unified Cannlytics platform, including API, console, documentation, and website. Many features are still under construction.