Cdp-backend

Council Data Project is a backend, frontend, and cookiecutter deployment for creating a whole database, storage system, and website, for archiving, exploring, and tracking municipal council action.

This library, `cdp-backend` maintains the pipelines, database models, infrastructure configuration, etc.

This version of `cdp-backend` is meant to fix the currently broken infrastructure deployment system. We are ripping out all of Pulumi and using gcloud, gsutil, and firebase CLI tooling to achieve the same result (which are all first-party tools so _should be_ better supported).

:warning: :warning:
This release is a "breaking" change for instances. Instance maintainers will want to update the instance with `make update-from-cookiecutter`.
:warning: :warning:

What's Changed
* admin/upgrade-packaging-standards by evamaxfield in https://github.com/CouncilDataProject/cdp-backend/pull/201
* bugfix/infra-stack by evamaxfield in https://github.com/CouncilDataProject/cdp-backend/pull/202


**Full Changelog**: https://github.com/CouncilDataProject/cdp-backend/compare/v3.1.3...v3.2.0

WebVTT Captions can have end of sentence markings in the middle of a sentence. Parse the captions per line instead of per caption to solve.

**Full Changelog**: https://github.com/CouncilDataProject/cdp-backend/compare/v3.1.1...v3.1.2

We couldn't deploy any new instances due to a bug with Firestore index creation. This changes makes it so that indexes are only created one after another.

See more details here: https://github.com/pulumi/pulumi-google-native/issues/528

As a result, stack deployment takes much longer unfortunately.

What's Changed
* bugfix/create-firestore-indexes-one-after-another by JacksonMaxfield in https://github.com/CouncilDataProject/cdp-backend/pull/194


**Full Changelog**: https://github.com/CouncilDataProject/cdp-backend/compare/v3.1.0...v3.1.1

We have made it a new minor version!

This release include two changes:
1. Better / safer WebVTT parsing (specifically to address Boston closed caption files) -- thanks to isaacna
2. The index pipeline has been split into two. This is a breaking change and deployment maintainers should update their deployment repositories with the `make update-from-cookiecutter` to pull in these changes.

What's Changed
* bugfix/fix-newline-webvtt-sentence-segmentation by isaacna in https://github.com/CouncilDataProject/cdp-backend/pull/193
* feature/reduce-and-fan-ngram-index by JacksonMaxfield in https://github.com/CouncilDataProject/cdp-backend/pull/189


**Full Changelog**: https://github.com/CouncilDataProject/cdp-backend/compare/v3.0.16...v3.1.0

A welcome addition of attempting multiple URLs when storing supporting files and also supporting m3u8 playlist video formats so that we can more generally handle Legistar data portals.

What's Changed
* feature/add-supporting-file-resiliency by JacksonMaxfield in https://github.com/CouncilDataProject/cdp-backend/pull/187
* feature/support-processing-m3u8-video-playlist-uris by JacksonMaxfield in https://github.com/CouncilDataProject/cdp-backend/pull/188


Contribution from liamphmurphy

**Full Changelog**: https://github.com/CouncilDataProject/cdp-backend/compare/v3.0.15...v3.0.16

This release is largely admin work.

What's Changed
* Drops py37
* Adds py310
* Cleans up and upgrades deps
* Deps have wider supported bounds where possible to allow for downstream cdp-data easier install
* Switches doc theme to furo
* License and copyright year and holder changes (Jackson Maxfield Brown -> CDP Contribs)
* Explicitly close local dask cluster during parallel pipeline runs


**Full Changelog**: https://github.com/CouncilDataProject/cdp-backend/compare/v3.0.14...v3.0.15