Spruned

- bugfixing on the getblock api

- add database migrations. This is tricky and may fail. In case, delete the storage and sync headers again (I hope not).
- improve the efficiency of SQLite DB (fewer calls, less size) by changing data serialization and queries. reduces the size by roughly a half.
- improve the efficiency on the P2P side when new block headers are notified (should fix race conditions of issue 85 )
- persist and index raw transactions when fetched via getblock (reduce network calls)
- improve the efficiency of the getblock verbose calls by using the per-block transactions index, avoid to load the entire block data
- increase the cache agent speed by using the per-block transactions index instead of the leveldb iterator

There are a lot of changes that may break the code, but they are very needed and useful.

The basic idea is to improve data indexing so that it can also be used in the future for other purposes (layerthree).

The release is available on pypi and docker hub.

- fixed a memory leak in the mempool volatile repository
- deprecated python 3.5.2, min version is now python 3.5.3
- added support for python 3.7.2
- requirements update

Also available on https://cloud.docker.com/u/gdassori/repository/docker/gdassori/spruned

- mempool support (see notes)
- quorum based fee estimation improvements (see notes)
- electrum protocol bump to v1.4
- Github anchored docker distribution (https://hub.docker.com/r/gdassori/spruned, see notes)
- jsonrpc floats formatting bugfix (now with fixed precision)
- rpcport bugfix (was 8332 also in testnet unless explicit)

notes

Mempool:
The code was in since a while, zeromq already shared some observers. Looks stable enough to be released.

$ spruned --mempool-size 100

enables the mempool with 100Mbytes buffer.

Fee estimation:
The previous quorum service had its own connectionpool. That was slow. The fee estimation service now uses the same connectionpool as the other API.

Docker:
I have seen many spruned containers on dockerhub, but I'd like to maintain a Github-anchored container on my username (gdassori, the same as Github).

This new release comes with a major update: zeromq support.
All the Core 0MQ topics are supported: hashblock, rawblock, hashtx, rawtx.

Summary:
- new checkpoints: up to block 568150 (00000000000000000001ae8ead7f279a3f7038967a147a0fb35acb83ff16fd82)
- new feature: zeromq
- new electrum testnet servers (public servers are unreliable, I'm going to support also the testnet network)
- jsonrpc: gettxout floats was returned as strings
- jsonrpc: getblockchaininfo difficulty&chainwork are mocks but the syntax is now valid (was null values)
- arguments: --daemon is deprecated, the argument is now useless and will be removed in next versions.

- lot of bugfixing all around
- a robusts P2P connectionpool with a 100% getblock hitrate
- an electrum servers quorum-based fee estimation
- proxies support
- tor support
- less memory usage for low end devices

An extended RPC API set:

== Blockchain ==
getbestblockhash
getblock "blockhash" ( verbosity )
getblockchaininfo
getblockcount
getblockhash height
getblockheader "hash" ( verbose )
gettxout "txid" n ( include_mempool )

== Rawtransactions ==
getrawtransaction "txid" ( verbose )
sendrawtransaction "hexstring" ( allowhighfees )

== Util ==
estimatefee nblocks
estimatesmartfee conf_target ("estimate_mode")
uptime

== Network ==
getpeerinfo
getnetworkinfo

== Wallet ==
validateaddress

== Partially emulated for compatibility ==
getchaintxstats
getmininginfo
getnettotals