Bigchaindb

Changed

Upgraded PyMongo to version 3.6 (which is compatible with MongoDB 3.6, 3.4 [and more](https://docs.mongodb.com/ecosystem/drivers/driver-compatibility-reference/python-driver-compatibility)). [Pull request 2298](https://github.com/bigchaindb/bigchaindb/pull/2298)

Fixed

When deploying a node using our Docker Compose file, it didn't expose port 46656, which is used by Tendermint for inter-node communications, so the node couldn't communicate with other nodes. We fixed that in [pull request 2299](https://github.com/bigchaindb/bigchaindb/pull/2299)

Notes

We ran all our tests using MongoDB 3.6 and they all passed, so it seems safe to use BigchainDB with MongoDB 3.6 from now on.

Changed

To resolve [issue 2279](https://github.com/bigchaindb/bigchaindb/issues/2279), we made some changes to the `bigchaindb-abci` package (which is our fork of `py-abci`) and told BigchainDB to use the new version (`bigchaindb-abci==0.4.5`). [Pull request 2281](https://github.com/bigchaindb/bigchaindb/pull/2281).

Changed

The Kubernetes liveness probe for the BigchainDB StatefulSet was improved to check the Tendermint /status endpoint in addition to the Tendermint /abci_info endpoint. [Pull request 2275](https://github.com/bigchaindb/bigchaindb/pull/2275)

Fixed

[Pull request 2270](https://github.com/bigchaindb/bigchaindb/pull/2270) resolved [issue 2269](https://github.com/bigchaindb/bigchaindb/issues/2269).

Notes

There's a new [page in the docs about storing files in BigchainDB](https://docs.bigchaindb.com/en/latest/store-files.html). [Pull request 2259](https://github.com/bigchaindb/bigchaindb/pull/2259)

Changed

* Upgraded BigchainDB Server code to use the latest version of Tendermint: version 0.19.2. Pull requests [2249](https://github.com/bigchaindb/bigchaindb/pull/2249), [2252](https://github.com/bigchaindb/bigchaindb/pull/2252) and [2253](https://github.com/bigchaindb/bigchaindb/pull/2253)
* Made some fixes to `py-abci` (an external Python package) and used our fixed version with BigchainDB. Those fixes resolved several known issues, including [issue 2182](https://github.com/bigchaindb/bigchaindb/issues/2182) and issues with large transactions in general. Note: At the time of writing, our fixes to `py-abci` hadn't been merged into the main `py-abci` repository or its latest release on PyPI; we were using our own special `bigchaindb-abci` package (which included our fixes). Pull requests [2250](https://github.com/bigchaindb/bigchaindb/pull/2250) and [2261](https://github.com/bigchaindb/bigchaindb/pull/2261)
* If BigchainDB Server crashes and then comes back, Tendermint Core doesn't try to reconnect to it. That's just how Tendermint Core works. We revised our Kubernetes-based production deployment template to resolve that issue: BigchainDB Server and Tendermint Core are now in the same Kubernetes StatefulSet; if the connection between them ever goes down, then Kubernetes restarts the whole StatefulSet. [Pull request 2242](https://github.com/bigchaindb/bigchaindb/pull/2242)

Fixed

Re-enabled multi-threading. [Pull request 2258](https://github.com/bigchaindb/bigchaindb/pull/2258)

Known Issues

Tendermint changed how it responds to a request to store data (via the [Tendermint Broadcast API](https://tendermint.com/docs/tendermint-core/using-tendermint.htmlbroadcast-api)) between version 0.12 and 0.19.2. We started modifying the code of BigchainDB Server to account for those changes in responses (in [pull request 2239](https://github.com/bigchaindb/bigchaindb/pull/2239)), but we found that there's a difference between what the Tendermint documentation _says_ about those responses and how Tendermint actually responds. We need to determine Tendermint's intent before we can finalize that pull request.

Notes

We were focused on getting the public BigchainDB Testnet stable during the development of BigchainDB 2.0 Alpha 3, and we think we largely succeeded. Because of that focus, we delayed the deployment of an internal test network until later. It would have had the same instabilities as the public BigchainDB Testnet anyway. In the future, we'll always test a new version of BigchainDB on our internal test network before deploying it on the public BigchainDB Testnet. (That wasn't possible this time around, because there was no old/stable version of BigchainDB 2.n to run on the public BigchainDB Testnet while we tested BigchainDB 2.[n+1] internally.)

Added

An implementation of [BEP-8 (BigchainDB Enhancement Proposal 8)](https://github.com/bigchaindb/BEPs/tree/master/8), which makes sure a node can recover from a system fault (e.g. a crash) into a consistent state, i.e. a state where the data in the node's local MongoDB database is consistent with the data stored in the blockchain. Pull requests [2135](https://github.com/bigchaindb/bigchaindb/pull/2135) and [2207](https://github.com/bigchaindb/bigchaindb/pull/2207)

Changed

When someone uses the HTTP API to send a new transaction to a BigchainDB network using the [POST /api/v1/transactions?mode={mode}](https://docs.bigchaindb.com/projects/server/en/master/http-client-server-api.htmlpost--api-v1-transactions?mode=mode) endpoint, they now get back a more informative HTTP response, so they can better-understand what happened. This is only when mode is `commit` or `sync`, because `async` _means_ that the response is immediate, without waiting to see what happened to the transaction. [Pull request 2198](https://github.com/bigchaindb/bigchaindb/pull/2198)

Known Issues

* If BigchainDB Server crashes and then is restarted, Tendermint Core won't try to reconnect to BigchainDB Server and so all operations requiring that connection won't work. We only understood this recently. We'll write a blog post explaining what we intend to do about it.
* The known issues in 2.0 Alpha (listed below) are still there.

There were _many_ changes between 1.3 and 2.0 Alpha, too many to list here. We wrote a series of blog posts to summarize most changes, especially those that affect end users and application developers:

* [BigchainDB 2.0 is Byzantine Fault Tolerant](https://blog.bigchaindb.com/bigchaindb-2-0-is-byzantine-fault-tolerant-5ffdac96bc44)
* [Some HTTP API Changes in the Next Release](https://blog.bigchaindb.com/some-http-api-changes-in-the-next-release-49612a537b0c)
* [Three Transaction Model Changes in the Next Release](https://blog.bigchaindb.com/three-transaction-model-changes-in-the-next-release-dadbac50094a)
* [Changes to the Server Command Line Interface in BigchainDB 2.0](https://blog.bigchaindb.com/changes-to-the-server-command-line-interface-in-bigchaindb-2-0-e1d6576e7155)
* _A forthcoming post about changes in BigchainDB Server configuration settings_

External Contributors

* r7vme contributed [pull request 1984](https://github.com/bigchaindb/bigchaindb/pull/1984) which fixed a bug in our Kubernetes-based production deployment template.

Known Issues

We intend to resolve these issues before releasing the final BigchainDB 2.0:

* There's a known Heisenbug that (sometimes) arises and we found that making the BigchainDB webserver single-threaded prevents that bug from causing problems. We intend to resolve that bug, but in the meantime our temporary workaround is to change the _default_ webserver configuration settings to single-threaded mode, i.e. `BIGCHAINDB_SERVER_WORKERS=1` and `BIGCHAINDB_SERVER_THREADS=1`.
* Issues sometimes happen when a large transaction is sent to a BigchainDB network.