Nautilus-trader

Released on 3rd November 2023 (UTC).

Enhancements
- Improved internal latency for live engines by using `loop.call_soon_threadsafe(...)`
- Improved `RedisCacheDatabase` client connection error handling with retries
- Added `WebSocketClient` connection headers, thanks ruthvik125 and twitu
- Added `support_contingent_orders` config option for venues (to simulate venues which do not support contingent orders)
- Added `StrategyConfig.manage_contingent_orders` config option (to automatically manage **open** contingent orders)
- Added `FuturesContract.activation_utc` property which returns a `pd.Timestamp` tz-aware (UTC)
- Added `OptionContract.activation_utc` property which returns a `pd.Timestamp` tz-aware (UTC)
- Added `CryptoFuture.activation_utc` property which returns a `pd.Timestamp` tz-aware (UTC)
- Added `FuturesContract.expiration_utc` property which returns a `pd.Timestamp` tz-aware (UTC)
- Added `OptionContract.expiration_utc` property which returns a `pd.Timestamp` tz-aware (UTC)
- Added `CryptoFuture.expiration_utc` property which returns a `pd.Timestamp` tz-aware (UTC)

Breaking Changes
- Renamed `FuturesContract.expiry_date` to `expiration_ns` (and associated params) as `uint64_t` UNIX nanoseconds
- Renamed `OptionContract.expiry_date` to `expiration_ns` (and associated params) as `uint64_t` UNIX nanoseconds
- Renamed `CryptoFuture.expiry_date` to `expiration_ns` (and associated params) as `uint64_t` UNIX nanoseconds
- Changed `FuturesContract` Arrow schema
- Changed `OptionContract` Arrow schema
- Changed `CryptoFuture` Arrow schema
- Transformed orders will now retain the original `ts_init` timestamp
- Removed unimplemented `batch_more` option for `Strategy.modify_order`
- Removed `InstrumentProvider.venue` property (redundant as a provider may have many venues)
- Dropped support for Python 3.9

Fixes
- Fixed `ParquetDataCatalog` file writing template, thanks limx0
- Fixed Binance all orders requests which would omit order reports when using a `start` param
- Fixed managed GTD orders past expiry cancellation on restart (orders were not being canceled)
- Fixed managed GTD orders cancel timer on order cancel (timers were not being canceled)
- Fixed `BacktestEngine` logging error with immediate stop (caused by certain timestamps being `None`)
- Fixed `BacktestNode` exceptions during backtest runs preventing next sequential run, thanks for reporting cavan-black
- Fixed `BinanceSpotPersmission` value error by relaxing typing for `BinanceSpotSymbolInfo.permissions`
- Interactive Brokers adapter various fixes, thanks rsmb7z

---

Released on 22nd October 2023 (UTC).

A major feature of this release is the `ParquetDataCatalog` version 2, which represents months of
collective effort thanks to contributions from Brad limx0, twitu, ghill2 and davidsblom.

This will be the final release with support for Python 3.9.

Enhancements
- Added `ParquetDataCatalog` v2 supporting built-in data types `OrderBookDelta`, `QuoteTick`, `TradeTick` and `Bar`
- Added `Strategy` specific order and position event handlers
- Added `ExecAlgorithm` specific order and position event handlers
- Added `Cache.is_order_pending_cancel_local(...)` (tracks local orders in cancel transition)
- Added `BinanceTimeInForce.GTD` enum member (futures only)
- Added Binance Futures support for GTD orders
- Added Binance internal bar aggregation inference from aggregated trades or 1-MINUTE bars (depending on lookback window)
- Added `BinanceExecClientConfig.use_gtd` config option (to remap to GTC and locally manage GTD orders)
- Added package version check for `nautilus_ibapi`, thanks rsmb7z
- Added `RiskEngine` min/max instrument notional limit checks
- Added `Controller` for dynamically controlling actor and strategy instances for a `Trader`
- Added `ReportProvider.generate_fills_report(...)` which provides a row per individual fill event, thanks r3k4mn14r
- Moved indicator registration and data handling down to `Actor` (now available for `Actor`)
- Implemented Binance `WebSocketClient` live subscribe and unsubscribe
- Implemented `BinanceCommonDataClient` retries for `update_instruments`
- Decythonized `Trader`

Breaking Changes
- Renamed `BookType.L1_TBBO` to `BookType.L1_MBP` (more accurate definition, as L1 is the top-level price either side)
- Renamed `VenueStatusUpdate` -> `VenueStatus`
- Renamed `InstrumentStatusUpdate` -> `InstrumentStatus`
- Renamed `Actor.subscribe_venue_status_updates(...)` to `Actor.subscribe_venue_status(...)`
- Renamed `Actor.subscribe_instrument_status_updates(...)` to `Actor.subscribe_instrument_status(...)`
- Renamed `Actor.unsubscribe_venue_status_updates(...)` to `Actor.unsubscribe_venue_status(...)`
- Renamed `Actor.unsubscribe_instrument_status_updates(...)` to `Actor.unsubscribe_instrument_status(...)`
- Renamed `Actor.on_venue_status_update(...)` to `Actor.on_venue_status(...)`
- Renamed `Actor.on_instrument_status_update(...)` to `Actor.on_instrument_status(...)`
- Changed `InstrumentStatus` fields/schema and constructor
- Moved `manage_gtd_expiry` from `Strategy.submit_order(...)` and `Strategy.submit_order_list(...)` to `StrategyConfig` (simpler and allows re-activiting any GTD timers on start)

Fixes
- Fixed `LimitIfTouchedOrder.create` (`exec_algorithm_params` were not being passed in)
- Fixed `OrderEmulator` start-up processing of OTO contingent orders (when position from parent is open)
- Fixed `SandboxExecutionClientConfig` `kw_only=True` to allow importing without initializing
- Fixed `OrderBook` pickling (did not include all attributes), thanks limx0
- Fixed open position snapshots race condition (added `open_only` flag)
- Fixed `Strategy.cancel_order` for orders in `INITIALIZED` state and with an `emulation_trigger` (was not sending command to `OrderEmulator`)
- Fixed `BinanceWebSocketClient` reconnect behavior (reconnect handler was not being called due event loop issue from Rust)
- Fixed Binance instruments missing max notional values, thanks for reporting AnthonyVince and thanks for fixing filipmacek
- Fixed Binance Futures fee rates for backtesting
- Fixed `Timer` missing condition check for non-positive intervals
- Fixed `Condition` checks involving integers, was previously defaulting to 32-bit and overflowing
- Fixed `ReportProvider.generate_order_fills_report(...)` which was missing partial fills for orders not in a final `FILLED` status, thanks r3k4mn14r

---

Released on 2nd September 2023 (UTC).

Enhancements
None

Breaking Changes
None

Fixes
- Fixed `OrderBookDelta.clear` method (where the `sequence` field was swapped with `flags` causing an overflow)
- Fixed `OrderManager` OTO contingency handling on fills
- Fixed `OrderManager` duplicate order canceled events (race condition when processing contingencies)
- Fixed `Cache` loading of initialized emulated orders (were not being correctly indexed as emulated)
- Fixed Binance order book subscriptions for deltas at full depth (was not requesting initial snapshot), thanks for reporting doublier1

---

Released on 26th August 2023 (UTC).

This release includes a large breaking change to quote tick bid and ask price property and
parameter naming. This was done in the interest of maintaining our generally explicit naming
standards, and has caused confusion for some users in the past. Data using 'bid' and 'ask' columns should
still work with the legacy data wranglers, as columns are renamed under the hood to accommodate
this change.

Enhancements
- Added `ActorExecutor` with `Actor` API for creating and running threaded tasks in live environments
- Added `OrderEmulated` event and associated `OrderStatus.EMULATED` enum variant
- Added `OrderReleased` event and associated `OrderStatus.RELEASED` enum variant
- Added `BacktestVenueConfig.use_position_ids` config option (default `True` to retain current behavior)
- Added `Cache.exec_spawn_total_quantity(...)` convenience method
- Added `Cache.exec_spawn_total_filled_qty(...)` convenience method
- Added `Cache.exec_spawn_total_leaves_qty(...)` convenience method
- Added `WebSocketClient.send_text`, thanks twitu
- Implemented string interning for `TimeEvent`

Breaking Changes
- Renamed `QuoteTick.bid` to `bid_price` including all associated parameters (for explicit naming standards)
- Renamed `QuoteTick.ask` to `ask_price` including all associated parameters (for explicit naming standards)

Fixes
- Fixed execution algorithm `position_id` assignment in `HEDGING` mode
- Fixed `OrderMatchingEngine` processing of emulated orders
- Fixed `OrderEmulator` processing of exec algorithm orders
- Fixed `ExecutionEngine` processing of exec algorithm orders (exec spawn IDs)
- Fixed `Cache` emulated order indexing (were not being properly discarded from the set when closed)
- Fixed `RedisCacheDatabase` loading of transformed `LIMIT` orders
- Fixed a connection issue with the IB client, thanks dkharrat and rsmb7z

---

Released on 31st July 2023 (UTC).

Enhancements
- Implemented string interning with the [ustr](https://github.com/anderslanglands/ustr) crate, thanks twitu
- Added `SyntheticInstrument` capability, including dynamic derivation formulas
- Added `Order.commissions()` convenience method (also added to state snapshot dictionaries)
- Added `Cache` position and order state snapshots (configure via `CacheConfig`)
- Added `CacheDatabaseConfig.timestamps_as_iso8601` to persist timestamps as ISO 8601 strings
- Added `LiveExecEngineConfig.filter_position_reports` to filter position reports from reconciliation
- Added `Strategy.cancel_gtd_expiry` to cancel managed GTD order expiration
- Added Binance Futures support for modifying `LIMIT` orders
- Added `BinanceExecClientConfig.max_retries` config option (for retrying order submit and cancel requests)
- Added `BinanceExecClientConfig.retry_delay` config option (the delay between retry attempts)
- Added `BinanceExecClientConfig.use_reduce_only` config option (default `True` to retain current behavior)
- Added `BinanceExecClientConfig.use_position_ids` config option (default `True` to retain current behavior)
- Added `BinanceExecClientConfig.treat_expired_as_canceled` option (default `False` to retain current behavior)
- Added `BacktestVenueConfig.use_reduce_only` config option (default `True` to retain current behavior)
- Added `MessageBus.is_pending_request(...)` method
- Added `Level` API for core `OrderBook` (exposes the bid and ask levels for the order book)
- Added `Actor.is_pending_request(...)` convenience method
- Added `Actor.has_pending_requests()` convenience method
- Added `Actor.pending_requests()` convenience method
- Added `USDP` (Pax Dollar) and `TUSD` (TrueUSD) stablecoins
- Improved `OrderMatchingEngine` handling when no fills (an error is now logged)
- Improved Binance live clients logging
- Upgraded Cython to v3.0.0 stable

Breaking Changes
- Moved `filter_unclaimed_external_orders` from `ExecEngineConfig` to `LiveExecEngineConfig`
- All `Actor.request_*` methods no longer take a `request_id`, but now return a `UUID4` request ID
- Removed `BinanceExecClientConfig.warn_gtd_to_gtd` (now always an `INFO` level log)
- Renamed `Instrument.native_symbol` to `raw_symbol` (you must manually migrate or flush your cached instruments)
- Renamed `Position.cost_currency` to `settlement_currency` (standardize terminology)
- Renamed `CacheDatabaseConfig.flush` to `flush_on_start` (for clarity)
- Changed `Order.ts_last` to represent the UNIX nanoseconds timestamp of the last _event_ (rather than fill)

Fixes
- Fixed `Portfolio.net_position` calculation to use `Decimal` rather than `float` to avoid rounding errors
- Fixed race condition on `OrderFactory` order identifiers generation
- Fixed dictionary representation of orders for `venue_order_id` (for three order types)
- Fixed `Currency` registration with core global map on creation
- Fixed serialization of `OrderInitialized.exec_algorithm_params` to spec (bytes rather than string)
- Fixed assignment of position IDs for contingent orders (when parent filled)
- Fixed `PENDING_CANCEL` -> `EXPIRED` as valid state transition (real world possibility)
- Fixed fill handling of `reduce_only` orders when partially filled
- Fixed Binance reconciliation which was requesting reports for the same symbol multiple times
- Fixed Binance Futures native symbol parsing (was actually Nautilus symbol values)
- Fixed Binance Futures `PositionStatusReport` parsing of position side
- Fixed Binance Futures `TradeReport` assignment of position ID (was hard-coded to hedging mode)
- Fixed Binance execution submitting of order lists
- Fixed Binance commission rates requests for `InstrumentProvider`
- Fixed Binance `TriggerType` parsing 1154, thanks for reporting davidblom603
- Fixed Binance order parsing of invalid orders in execution reports 1157, thanks for reporting graceyangfan
- Extended `BinanceOrderType` enum members to include undocumented `INSURANCE_FUND`, thanks for reporting Tzumx
- Extended `BinanceSpotPermissions` enum members 1161, thanks for reporting davidblom603

---

Released on 16th June 2023 (UTC).

The Betfair adapter is broken for this release pending integration with the new Rust order book.
We recommend you do not upgrade to this version if you're using the Betfair adapter.

Enhancements
- Integrated Interactive Brokers adapter v2 into platform, thanks rsmb7z
- Integrated core Rust `OrderBook` into platform
- Integrated core Rust `OrderBookDelta` data type
- Added core Rust `HttpClient` based on `hyper`, thanks twitu
- Added core Rust `WebSocketClient` based on `tokio-tungstenite`, thanks twitu
- Added core Rust `SocketClient` based on `tokio` `TcpStream`, thanks twitu
- Added `quote_quantity` parameter to determine if order quantity is denominated in quote currency
- Added `trigger_instrument_id` parameter to trigger emulated orders from alternative instrument prices
- Added `use_random_ids` to `add_venue(...)` method, controls whether venue order, position and trade IDs will be random UUID4s (no change to current behavior)
- Added `ExecEngineConfig.filter_unclaimed_external_orders` config option, if unclaimed order events with an `EXTERNAL` strategy ID should be filtered/dropped
- Changed `BinanceHttpClient` to use new core HTTP client
- Defined public API for data, can now import directly from `nautilus_trader.model.data` (denest namespace)
- Defined public API for events, can now import directly from `nautilus_trader.model.events` (denest namespace)

Breaking Changes
- Upgraded `pandas` to v2
- Removed `OrderBookSnapshot` (redundant as can be represented as an initial CLEAR followed by deltas)
- Removed `OrderBookData` (redundant)
- Renamed `Actor.handle_order_book_delta` to `handle_order_book_deltas` (to more clearly reflect the `OrderBookDeltas` data type)
- Renamed `Actor.on_order_book_delta` to `on_order_book_deltas` (to more clearly reflect the `OrderBookDeltas` data type)
- Renamed `inverse_as_quote` to `use_quote_for_inverse` (ambiguous name, only applicable for notional calcs on inverse instruments)
- Changed `Data` contract (custom data), [see docs](https://nautilustrader.io/docs/latest/concepts/advanced/data.html)
- Renamed core `LogMessage` to `LogEvent` to more clearly distinguish between the `message` field and the event struct itself (aligns with [vector](https://vector.dev/docs/about/under-the-hood/architecture/data-model/log/) language)
- Renamed core `LogEvent.timestamp_ns` to `LogEvent.timestamp` (affects field name for JSON format)
- Renamed core `LogEvent.msg` to `LogEvent.message` (affects field name for JSON format)

Fixes
- Updated `BinanceAccountType` enum members and associated docs
- Fixed `BinanceCommonExecutionClient` iteration of `OrderList` orders
- Fixed heartbeats for `BinanceWebSocketClient` (new Rust client now responds with `pong` frames)
- Fixed Binance adapter typing for `orderId`, `fromId`, `startTime` and `endTime` (all are ints), thanks for reporting davidsblom
- Fixed `Currency` equality to be based on the `code` field (avoiding equality issues over FFI), thanks for reporting Otlk
- Fixed `BinanceInstrumentProvider` parsing of initial and maintenance margin values

---