Rolls-blockchain

Changed

- This is a hard fork/breaking change from RC6/7/8. The current plan is to drop the flag at noon pacific time, today 3/16.
- Using the real prefarm keys for this test

Fixed

- Found and fixed another green flag related issue
- Fixed an issue with weight proofs where all sub-epochs were sampled, and the size of the weight proof kept growing
- Fixed an issue with install-gui.sh, where npm audit fix was failing. (Thanks Depado!)
- Migration with ROLLS_ROOT set does not crash rolls init

Added

- This is a hard fork/breaking change from RC6/7. TROLLS Coins will **not** be moved forward but your plots and keys and parts of your configuration do. When you install this version before 10AM PDST on 3/16/2021 it will load up, start finding peers, and otherwise wait for the flag drop at that time to start farming. This is likely to be the last dress rehearsal for mainnet launch. Our [3/15/2021 blog post](https://www.pecanrolls.net/2021/03/15/mainnet-update.html) has more details on the current mainnet launch plan.
- The GUI now has a tooltip that directs users to the explanation of the plot filter.
- The GUI now has a tooltip to explain the "Disable bitfield plotting" option. Thanks shaneo257 for the idea.
- The GUI now has a tooltip to explain Hierarchical Deterministic keys next to Receive Address on the Wallet page.

Changed

- We now use Python 3.9 to build MacOS installers.
- Harvester now catches another error class and continues to harvest. Thanks to xorinox for this PR.
- We now use a smaller weight proof sample size to ease the load on smaller machines when syncing.
- Starting the GUI from Linux will now also error out if `npm run build` is run outside the venv. Huge thanks to dkackman for that PR.
- `rolls farm summary` will now display TROLLS or ROLLS as appropriate.
- We added more time to our API timeouts and improved logging around times outs.

Fixed

- We no longer use the transaction cache to look up transactions for new transactions as that was causing a wallet sync bug.
- Sometimes the GUI would not pick up the fingerprint for the plotting key.
- `rolls farm summary` displayed some incorrect amounts.
- Weight proofs were timing out.
- Changes to farming rewards target addresses from the GUI were not being saved for restart correctly.
- Signage points, recent deficit blocks, and slots for overflow challenge blocks had minor issues.

Changed

- Our green flag test blockchain launch worked but it uncovered a flaw in our installer versions. This release is a bug fix release to address that flaw. You should read the RC6 changes below if this is your first time installing since RC5.
- Thanks to dkackman for implementing an early exit of the GUI if you run `npm run build` without being in the `venv`.
- `rolls netspace` now defaults to 1000 blocks to mirror the GUI.
- The installer build process was spruced up some.

Fixed

- Setting difficulty way too low on the testnet_6 launch revealed a Timelord edge case. The full node was hardcoding the default difficulty if block height is < EPOCH_BLOCKS. However there were many overlapping blocks, so none of the blocks reached the height, and therefore the timelord infused the wrong difficulty.
- Fixed a race condition in the Timelord, where it took time to update the state, so it ignored the new_peak_timelord form the full_node, which should have reset the timelord to a good state.
- Wallet notoriously showed "not synced" when it was in sync.
- Installers were not correctly placing root TLS certificates into the bundle.
- Weight proofs had a logic typo.
- There was a typo in `rolls netspace`. Thanks altendky.
- There was a typo in `rolls plots`. Thanks adamfiddler.

Known Issues

- Some users can't plot in the GUI in MacOS Big Sur - especially on M1. See issue [1189](https://github.com/strandedathome/rolls-blockchain/issues/1189)

Added

- This is a hard fork/breaking change from RC5. TROLLS Coins will **not** be moved forward but your plots and keys and parts of your configuration do. We will be testing the final mainnet release strategy with the launch of RC6. For the test, those who are comfortable running the dev branch will update and start up their farms. All harvesters and plots will load and until the green flag drops, peers will be gossiped so your farm can establish good network connectivity. When the flag drops, each node will pull down the signed genesis challenge and start farming. Block 1 will be broadcast to anyone who hasn't seen the flag drop yet. The only difference for mainnet is that there will be 1.0 installers and a main branch release more than 24 hours before the real green flag.
- There is now basic plot queueing functionality in the GUI. By default, plotting works as it has in the past. However you can now name a queue in Step 2 Advanced Options. Chose something like `first`. Everything you add to the `first` queue will start up like it has in the past but now you can go through the steps again and create a queue named `second` and it will immediately start plotting as if it is unaware of and parallel with `first`. A great use case is that you would set `first` to plot 5 plots sequentially and then you'd set `second` to plot 5 sequentially and that would give you two parallel queues of 5 plot's each. We will be returning to plotting speed and UI soon. Thanks jespino for this clever work around for now.
- There is now an option on the Farm page to manage your farming rewards receive addresses. This makes it easy to send your farming rewards to an offline wallet. This also checks your existing rewards addresses and warns if you do not have the matching private key. That is expected if you are using an offline wallet of course.
- Functionally has been added to the farmer rpc including checking and changing your farming rewards target addresses.
- Added the ability to translate material-ui components like `Row 1 of 10`. Thanks jespino.
- Arch linux support has been added to `sh install.sh`. Thanks jespino.
- Update FullBlock to Allow Generator References - a list of block heights of generators to be made available to the block program of the current block at generator runtime. This sets the stage for smart coins calling existing "libraries" already on the chain to lower fees and increase the scale of complex smart coins.

Changed

- Remove `rolls plots "-s" "--stripe_size"` and the strip size setting in the Advanced section of the GUI. We now always use the best default of 64K for the GUI and cli.
- `rolls keys add` takes secret words a prompt on the command line or stdin instead of command line arguments for security.
- Version 1.0.1 of chiavdf was added. This brought MPIR on Windows to the most recent release. Additionally we removed inefficient ConvertIntegerToBytes() and ConvertBytesToInt() functions, use GMP library's mpz_export/mpz_import for big integers and simple helper functions for built-in integer types. The latter are taken from chiavdf. We now require compressed forms to be encoded canonically when deserializing. This should prevent potential grinding attacks where some non-canonical encodings of a compressed form could be used to change its hash and thus the next challenges derived from it. Canonically encoded compressed forms must be reduced and must produce the same string when deserialized and serialized again.
- Version 1.0 of our BLS signature library is included. We brought Relic, gmp and MPIR up to their most recent releases. We again thank the Dash team for their fixes and improvements.
- We now hand build Apple Silicon native binary wheels for all rolls-blockchain dependencies and host them at [https://pypi.chia.net/simple](https://pypi.chia.net/simple). We are likely to hand build a MacOS ARM64 dmg available and certainly will for 1.0. You can install natively on M1 now with the `git clone` developer method today. Just make sure Python 3.9 is installed. `python3 --version` works.
- The GUI now shows you which network you are connected to on the Full Node page. It will also wait patiently for the green flag to drop on a network launch.
- In the GUI you can only plot k=32 or larger with the single exception of k=25 for testing. You will have to confirm choosing k=25 however. Thanks to jespino for help on this and limiting the cli as well.
- The restore smart wallets from backup prompt has been improved to better get the intent across and that it can be skipped.
- At the top of the plotting wizard we have added text pointing out that you can plot without being in sync or on the internet.
- Wallet no longer automatically creates a new hierarchical deterministic wallet receive address on each start. You can and still should choose a new one with the `NEW ADDRESS` button for each new transaction for privacy.
- The network maximum k size is now set to k=50. We think that may be more storage than atoms in the solar system so it should be ok. But we will probably be hated for it in 200 years...
- The formula for computing iterations is simplified, so that only one division is necessary, and inverting the (1-x) into just x.
- There are new timestamp consensus rules. A block N must have a greater timestamp than block N-1. Also, a block's timestamp cannot be more than 5 minutes in the future. Note that we have decided that work factor difficulty resets are now going to be 24 hours on mainnet but are still shorter on testnet.
- A List[Tuple[uint16, str]] is added to the peer network handshake. These are the capabilities that the node supports, to add new features to the protocol in an easy - soft fork - manner. The message_id is now before the data in each message.
- Peer gossip limits were set.
- Generators have been re-worked in CLVM. We added a chialisp deserialization puzzle and improved the low-level generator. We reduce the accepted atom size to 1MB during PecanRollsLisp native deserialization.
- When processing mempool transactions, Coin IDs are now calculated from parent coin ID and amount
- We implemented rate limiting for full node. This can and will lead to short term bans of certain peers that didn't behave in expected ways. This is ok and normal, but strong defense against many DDOS attacks.
- `requirements-dev.txt` has been removed in favor of the CI actions and test scripts.
- We have moved to a new and much higher scalability download.pecanrolls.net to support the mainnet launch flag and additional download demand.
- To always get the latest testnet and then mainnet installers you can now use a latest URL: [Windows](https://download.pecanrolls.net/latest/Setup-Win64.exe) and [MacOS x86_64](https://download.pecanrolls.net/latest/Setup-MacOS.dmg).
- PecanRolls wheels not on Pypi and some dependecies not found there also are now on pypi.pecanrolls.net.
- Additional typing has been added to the Python code with thanks to jespino.
- Cryptography and Keyring have been bumped to their current releases.
- PRs and commits to the rolls-blockchain-gui repository will automatically have their locales updated.

Fixed

- The Farm page will now no longer get stuck at 50 TROLLS farmed.
- `rolls farm` has had multiple bugs and spelling issues addressed. Thanks to alfonsoperez, soulmerge and olivernyc for your contributions.
- `rolls wallet` had various bugs.
- Various weight proof improvements.
- Some users on Big Sur could not plot from the GUI as the log window would be stuck on "Loading."
- We believe we have fixed the chain stall/confused Timelord bug from ~ 13:00 UTC 3/10/21. We've added additional recovery logic as well.
- Logs from receiving a duplicate compacted Proof of Time are much more human friendly.
- We believe that the install/migrate process was bringing forward bad farming rewards receive addresses. We have attempted to stop that by only migrating RC3 and newer configurations. You can make sure you are not effected by using the Manage Farming Rewards tool mentioned above or putting a known good wallet receive address in both `rolls_target_address` sections of config.yaml.
- Wallet cached transactions incorrectly in some cases.

Added

- The RC5 release is a new breaking change/hard fork blockchain. Plots and keys from previous chains will work fine on RC5 but balances of TROLLS will not come forward.
- We now support a "green flag" chain launch process. A new version of the software will poll download.pecanrolls.net/notify/ for a signed json file that will be the genesis block of the chain for that version. This will allow unattended start at mainnet.
- Bluebox Timelords are back. These are Timelords most anyone can run. They search through the historical chain and find large proofs of times and compact them down to their smallest representation. This significantly speeds up syncing for newly started nodes. Currently this is only supported on Linux and MacOS x86_64 but we will expand that. Any desktop or server of any age will be fast enough to be a useful Bluebox Timelord.
- Thanks to jespino there is now `rolls farm summary`. You can now get almost exactly the same farming information on the CLI as the GUI.
- We have added Romanian to the GUI translations. Thank you to bicilis on [Crowdin](https://crowdin.com/project/rolls-blockchain). We also added a couple of additional target languages. Klingon anyone?
- `rolls wallet` now takes get_address to get a new wallet receive address from the CLI.
- `rolls plots check` will list out all the failed plot filenames at the end of the report. Thanks for the PR go to eFishCent.
- Chialisp and the clvm have had the standard puzzle updated and we replaced `((c P A))` with `(a P A)`.

Changed

- Testnets and mainnet now set their minimum `k` size and enforce it. RC5 testnet will reject plots of size less than k=32.
- Sub slots now require 16 blocks instead of 12.
- Thanks to xdustinface of Dash, the BlS Signature library has been updated to 0.9 with clean ups and some speed ups. This changed how the G2 infinity element was handled and we now manage it inside of rolls-blockchain, etc., instead of in blspy.
- We have updated the display of peer nodes and moved adding a peer to it's own pop up in the GUI.
- Block searching in the GUI has been improved.
- jespino added i18n support and refactored how locales are loaded in the GUI. Additionally he moved more strings into the translation infrastructure for translators.
- In chiavdf we changed n-Wesolowski proofs to include B instead of y in segments. Proof segments now have the form (iters, B, proof) instead of (iters, y, proof). This reduces proof segment size from 208 to 141 bytes.
- The new chiavdf proof format is not compatible with the old one, however zero-Wesolowski proofs are not affected as they have zero proof segments and consist only of (y, proof).
- We made two HashPrime optimizations in chiavdf. This forces numbers being tested for primality to be odd and avoids an unnecessary update of the sprout vector by stopping after the first non-zero value. This is a breaking change as it changes the prime numbers generated from a given seed. We believe this is the final breaking change for chiavdf.
- chiabip158 was set to a gold 1.0 version.
- Comments to Chialisp and clvm source have been updated for all of the Chialisp changes over the proceeding three weeks.
- And thanks yet again to jespino for a host of PRs to add more detailed typing to various components in rolls-blockchain.
- aiohttp was updated to 3.7.4 to address a low severity [security issue](https://github.com/advisories/GHSA-v6wp-4m6f-gcjg).
- calccrypto/uint128_t was updated in the Windows chiapos implementation. Chiapos required some changes its build process to support MacOS ARM64.

Fixed

- Harvester would crash if it encountered more than 16,000 plot files or 256 directories.
- Nodes that were interrupted by a network crash or standby on a laptop were not syncing upon reconnection in RC4.
- Sync issues could stop syncing from restarting and could lead to a peer host that you could not remove.
- Adding Click changed the behavior of `rolls keys add -m`. The help now makes it clear that the 24 word mnemonic needs to be surrounded by a pair of quotes.
- Python root CA certificates have issues so we have added the Mozilla certificate store via curl.se and use that to connect to backup.pecanrolls.net via https, for example.
- The difficulty adjustment calculation was simplified.
- All of the rolls sub repositories that were attempting to build MacOS Universal wheels were only generating x86_64 wheels internally. We have moved back to only generating x86_64 MacOS wheels on CI.
- However, we have updated and test compiled all PecanRolls dependencies on Apple Silicon and will be making available a test .dmg for MacOS ARM64 shortly.
- Various weight proof edge cases have been fixed.
- Various typos and style clean ups were made to the Click CLI implementation. `rolls -upnp f` was added to disable uPnP.
- `rolls plots check` shouldn't crash when encountering plots that cause RuntimeError. PR again thanks to eFishCent.
- Coloured coin announcements had a bug that would allow counterfeiting.

Fixed

- This is a bug fix release for RC3. There was an unexpected interaction between the GUI and the Click cli tool regarding Windows that made GUI plotting not start on all GUIs.