Twarc

v1.4.0 adds a new output format `csv-excel` that modifies some of the columns to prevent newlines from being present in quoted strings. While it's valid to have newlines within quoted column values, they [cause some problems](https://github.com/DocNow/twarc/pull/208) for Excel. Thanks to justinlittman for diagnosing and resolving the issue.

This bugfix release fixes an install problem on Windows that was causing the twarc.py command line to fail to execute. It also includes a fix for not often used subcommands like `places` that were failing because they weren't encoding JSON properly.

A small bugfix release to correct a missing import.

This release brings CSV output to twarc.

twarc search blacklivesmatter --format csv > blacklivematter.csv

If you prefer you can output directly to a named file:

twarc search blacklivesmatter --format csv --outupt blacklivesmatter.csv

You can also split the output across numbered files if you want. For example this will rotate files every 10,000 rows:

twarc search blacklivesmatter --format csv --split 10000 --output blacklivesmatter.csv

Which will write files like:

blacklivesmatter-001.csv
blacklivesmatter-002.csv
blacklivesmatter-003.csv
...

The big change in this release is that twarc will now emit tweets in *extended* mode by default when it is feching tweets from Twitter's REST API. This means the information previously available in `.text` will now be found in `.full_text`.

On the other hand, data collected with twarc's `filter` and `sample` commands from Twitter's streaming APIs continue to use `.text` but have the additional `.extended_tweet` stanza when tweets require it.

For all the grizzly details please see this [announcement](https://dev.twitter.com/overview/api/upcoming-changes-to-tweets) from Twitter. You also might be interested in reading [this post](https://gwu-libraries.github.io/sfm-ui/posts/2017-03-31-extended-tweets) from the Social Feed Manager project.

If you have data pipelines of any kind built with twarc we highly recommend that you test that things are working properly before install twarc in a production environment. If you would prefer to get data the old way please use `--tweet_mode compat`. Unfortunately there is no way to change the mode for the streaming APIs.

This is a small bugfix release for the replies command which didn't emit an understandable error when used on the command line with a non-existent tweet identifier.