<!-- Release notes generated using configuration in .github/release.yml at main -->
What's Changed
Breaking Changes
3.0.0 is a major release, which means there are some breaking changes. You can see a full list of all PRs included in this release which have implications for breaking changes [here](https://github.com/move-coop/parsons/pulls?q=base%3Amajor-release+is%3Amerged+updated%3A%3E2023-08-31+).
1) We now parse boolean types by default instead of coercing them into strings by default. We detect boolean column types when copying a parsons Table to a database and create a boolean column in the database. If you want to maintain the old behavior, to convert the boolean columns in the table to strings before uploading it to the database, like this: `table = table.convert(['bool', 'columns', 'here', ...], str)` More: https://github.com/move-coop/parsons/pull/943
2) We've made some major updates to the BigQuery and GoogleCloudStorage connecters:
GoogleBigQuery
The `GoogleBigQuery` connector was written with compatibility in mind, and utilizes many of the same functions as the Amazon `Redshift` connector in order to minimize the user experience between the two cloud service providers. `GoogleBigQuery` is authenticated with a service account JSON file, which can be generated in the GCP user interface and stored locally.
There are several subtle differences between `GoogleBigQuery` and `Redshift`, most notably in the `.query()` function, which runs asynchronously in Google. We recommend using the `.delete_table()` function rather than sending a `DELETE TABLE` SQL query through the `.query()` function for this reason, as the connector will raise an exception when the asynchronous task completes and the table no longer exists; alternatively, the user can pass in `.query(sql=sql, return_values=False)` to prevent this exception from raising.
In addition to the familiar `.copy()` function, the `GoogleBigQuery` connector includes a `.copy_large_compressed_file_from_gcs()` function to handle large files in cloud storage, such as the voter file. BigQuery streams large uncompressed files in batches, but cannot do so when a file in compressed. This function decompresses the file in question using the correct compression type parameter (`gzip` is default but `zip` is also accepted), copies the file to BigQuery, then deletes the decompressed file from cloud storage.
GoogleCloudStorage
Similarly, the `GoogleCloudStorage` connector provides an API to view and manipulate blobs in cloud storage with compatibility to Amazon's `S3` connector. Users can create new storage buckets, load blobs into buckets, list their contents, acquire blob metadata, and download blobs from cloud storage to their local environments. This connector handles the decompression steps outlined above in the `. copy_large_compressed_file_from_gcs()` function outlined above, and also includes helpful utilities to aid in moving data to and from Google Cloud Storage.
New Connectors
Big thanks to cmdelrio for adding the new MobileCommons connector (https://github.com/move-coop/parsons/pull/896) and to austinweisgrau for the new Catalist Match API connector (https://github.com/move-coop/parsons/pull/912)!
Other Changes
* Zoom Polls by IanRFerguson in https://github.com/move-coop/parsons/pull/886
* Check for empty tables in zoom poll results by Jason94 in https://github.com/move-coop/parsons/pull/897
* 741 : Deprecate Slack chat.postMessage `as_user` argument and allow for new authorship arguments by angloyna in https://github.com/move-coop/parsons/pull/891
* 816 Airtable.get_records() fields argument can be either str or list by angloyna in https://github.com/move-coop/parsons/pull/892
* Nir's actionnetwork changes by NirTatcher in https://github.com/move-coop/parsons/pull/900
* Fix airtable.insert_records table arg by codygordon in https://github.com/move-coop/parsons/pull/907
* Add canales s3 functions by sharinetmc in https://github.com/move-coop/parsons/pull/885
* Enhancement: Action Network Connector: Added unpack_statistics param in get_messages method by matthewkrausse in https://github.com/move-coop/parsons/pull/917
* Adding rename_columns method to Parsons Table by mkrausse-ggtx in https://github.com/move-coop/parsons/pull/923
* Add http response to update_mailer by sjwmoveon in https://github.com/move-coop/parsons/pull/924
* Enable passing arbitrary additional fields to NGPVAN people match API by austinweisgrau in https://github.com/move-coop/parsons/pull/916
* Install dependencies for tests based on package install by austinweisgrau in https://github.com/move-coop/parsons/pull/951
New Contributors
Cheers to our newest contributors! 🎉 Thanks so much for your help.
* NirTatcher made their first contribution in https://github.com/move-coop/parsons/pull/900
* matthewkrausse made their first contribution in https://github.com/move-coop/parsons/pull/917
* dexchan made their first contribution in https://github.com/move-coop/parsons/pull/920
**Full Changelog**: https://github.com/move-coop/parsons/compare/v2.1.0...v3.0.0