Nautilus-trader

Released on 22nd October 2024 (UTC).

Enhancements
- Added `TardisCSVDataLoader` for loading data from Tardis format CSV files as either legacy Cython or PyO3 objects
- Added `Clock.timestamp_us()` method for UNIX timestamps in microseconds (μs)
- Added support for `bbo-1s` and `bbo-1m` quote schemas for Databento adapter (1990), thanks faysou
- Added validation for venue `book_type` configuration vs data (prevents an issue where top-of-book data is used when order book data is expected)
- Added `compute_effective_deltas` config option for `PolymarketDataClientConfig`, reducing snapshot size (default `False` to retain current behavior)
- Added rate limiter for `WebSocketClient` (1994), thanks Pushkarm029
- Added in the money probability field to GreeksData (1995), thanks faysou
- Added `on_signal(signal)` handler for custom signal data
- Added `nautilus_trader.common.events` module with re-exports for `TimeEvent` and other system events
- Improved usability of `OrderBookDepth10` by filling partial levels with null orders and zero counts
- Improved Postgres config (2010), thanks filipmacek
- Refined `DatabentoInstrumentProvider` handling of large bulks of instrument definitions (improved parent symbol support)
- Standardized Betfair symbology to use hyphens instead of periods (prevents Betfair symbols being treated as composite)
- Integration guide docs fixes (1991), thanks FarukhS52

Internal Improvements
- Ported `Throttler` to Rust (1988), thanks Pushkarm029 and twitu
- Ported `BettingInstrument` to Rust
- Refined `RateLimiter` for `WebSocketClient` and add tests (2000), thanks Pushkarm029
- Refined `WebSocketClient` to close existing tasks on reconnect (1986), thanks davidsblom
- Removed mutable references in `CacheDatabaseAdapter` trait in Rust (2015), thanks filipmacek
- Use Rust rate limiter for dYdX websockets (1996, 1999), thanks davidsblom
- Improved error logs for dYdX websocket subscriptions (1993), thanks davidsblom
- Standardized log and error message syntax in Rust
- Continue porting `SimulatedExchange` and `OrderMatchingEngine` to Rust (1997, 1998, 2001, 2003, 2004, 2006, 2007, 2009, 2014), thanks filipmacek

Breaking Changes
- Removed legacy `TardisQuoteDataLoader` (now redundant with new Rust implemented loader)
- Removed legacy `TardisTradeDataLoader` (now redundant with new Rust implemented loader)
- Custom signals are now passed to `on_signal(signal)` instead of `on_data(data)`
- Changed `Position.to_dict()` `commissions` value type to `list[str]` (was an optional `str` of a list of strings)
- Changed `Position.to_dict()` `avg_px_open` value type to `float`
- Changed `Position.to_dict()` `avg_px_close` value type to `float | None`
- Changed `Position.to_dict()` `realized_return` value type to `float | None`
- Changed `BettingInstrument` Arrow schema fields `event_open_date` and `market_start_time` from `string` to `uint64`

Fixes
- Fixed `SocketClient` TLS implementation
- Fixed `WebSocketClient` error handling on writer close, thanks for reporting davidsblom
- Fixed resubscribing to orderbook in batched mode for dYdX (1985), thanks davidsblom
- Fixed Betfair tests related to symbology (1988), thanks limx0
- Fixed check for `OmsType` in `OrderMatchingEngine` position ID processing (2003), thanks filipmacek
- Fixed `TardisCSVDataLoader` snapshot5 and snapshot25 parsing (2005), thanks Pushkarm029
- Fixed Binance clients venue assignment, we should use the `client_id` params (which match the custom client `name`) to communicate with the clients, and use the same `'BINANCE'` venue identifiers
- Fixed `OrderMatchingEngine` incorrectly attempting to process monthly bars for execution (which will fail, as no reasonable `timedelta` is available), thanks for reporting frostRed
- Fixed handling `MONTH` aggregation for `cache.bar_types()` (sorting required an internal call for the bar intervals `timedelta`), thanks for reporting frostRed

---

Released on 5th October 2024 (UTC).

Enhancements
- Added `mode` parameter to `ParquetDataCatalog.write_data` to control data writing behavior (1976), thanks faysou
- Added batch cancel for short terms orders of dYdX (1978), thanks davidsblom
- Improved OKX configuration (1966), thanks miller-moore
- Improved option greeks (1964), thanks faysou

Internal Improvements
- Implemented order book delta processing for `SimulatedExchange` in Rust (1975), thanks filipmacek
- Implemented bar processing for `SimulatedExchange` in Rust (1969), thanks filipmacek
- Implemented remaining getter functions for `SimulatedExchange` in Rust (1970), thanks filipmacek
- Implemented rate limiting for dYdX websocket subscriptions (1977), thanks davidsblom
- Refactored reconnection handling for dYdX (1983), thanks davidsblom
- Refined `DatabentoDataLoader` internals to accommodate usage from Rust
- Added initial large test data files download and caching capability

Breaking Changes
None

Fixes
- Fixed out of order row groups in DataFusion filter query (1974), thanks twitu
- Fixed `BacktestNode` data sorting regression causing clock non-decreasing time assertion error
- Fixed circular imports for `Actor`, thanks limx0
- Fixed OKX HTTP client signatures (1966), thanks miller-moore
- Fixed resubscribing to orderbooks for dYdX (1973), thanks davidsblom
- Fixed generating cancel rejections for dYdX (1982), thanks davidsblom
- Fixed `WebSocketClient` task cleanup on disconnect (1981), thanks twitu
- Fixed `Condition` method name collisions with C `true` and `false` macros, which occurred during compilation in profiling mode

---

Released on 27th September 2024 (UTC).

This will be the final release with support for Python 3.10.

The `numpy` version requirement has been relaxed to >= 1.26.4.

Enhancements
- Added Polymarket decentralized prediction market integration
- Added OKX crypto exchange integration (1951), thanks miller-moore
- Added `BinaryOption` instrument (supports Polymarket integration)
- Added `LiveExecutionEngine.inflight_check_retries` config option to limit in-flight order query attempts
- Added `Symbol.root()` method for obtaining the root of parent or composite symbols
- Added `Symbol.topic()` method for obtaining the subscription topic of parent or composite symbols
- Added `Symbol.is_composite()` method to determine if symbol is made up of parts with period (`.`) delimiters
- Added `underlying` filter parameter for `Cache.instruments(...)` method
- Added `reduce_only` parameter for `Strategy.close_position(...)` method (default `True` to retain current behavior)
- Added `reduce_only` parameter for `Strategy.close_all_positions(...)` method (default `True` to retain current behavior)
- Implemented flush with truncate Postgres function for `PostgresCacheDatabase` (1928), thanks filipmacek
- Implemented file rotation for `StreamingFeatherWriter` with internal improvements using `Clock` and `Cache` (1954, 1961), thanks graceyangfan
- Improved dYdX execution client to use `RetryManager` for HTTP requests (1941), thanks davidsblom
- Improved Interactive Brokers adapter to use a dynamic IB gateway `container_image` from config (1940), thanks rsmb7z
- Improved `OrderBookDeltas` streaming and batching based on the `F_LAST` flag
- Standardized underscore thousands separators for backtest logging
- Updated Databento `publishers.json`

Internal Improvements
- Implemented `OrderTestBuilder` to assist testing in Rust (1952), thanks filipmacek
- Implemented quote tick processing for SimulatedExchange in Rust (1956), thanks filipmacek
- Implemented trade tick processing for SimulatedExchange in Rust (1956), thanks filipmacek
- Refined `Logger` to use unbuffered stdout/stderr writers (1960), thanks twitu

Breaking Changes
- Renamed `batch_size_bytes` to `chunk_size` (more accurate naming for number of data points to process per chunk in backtest streaming mode)
- Standardized Stop-Loss (SL) and Take-Profit (TP) parameter ordering for `OrderFactory.bracket(...)` including: `tp_time_in_force`, `tp_exec_algorithm_params`, `tp_tags`, `tp_client_order_id`

Fixes
- Fixed `LoggingConfig` issue for `level_file` when used with `use_pyo3=True` (was not passing through the `level_file` setting), thanks for reporting xt2014
- Fixed composite bar requests (1923), thanks faysou
- Fixed average price calculation for `ValueBarAggregator` (1927), thanks faysou
- Fixed breaking protobuf issue by pinning `protobuf` and `grpcio` for dYdX (1929), thanks davidsblom
- Fixed edge case where exceptions raised in `BacktestNode` prior to engine initialization would not produce logs, thanks for reporting faysou
- Fixed handling of internal server error for dYdX (1938), thanks davidsblom
- Fixed `BybitWebSocketClient` private channel authentication on reconnect, thanks for reporting miller-moore
- Fixed `OrderFactory.bracket(...)` parameter ordering for `sl_time_in_force` and `tp_time_in_force`, thanks for reporting marcodambros
- Fixed `Cfd` instrument Arrow schema and serialization
- Fixed bar subscriptions on TWS/GW restart for Interactive Brokers (1950), thanks rsmb7z
- Fixed Databento parent and continuous contract subscriptions (using new symbol root)
- Fixed Databento `FuturesSpread` and `OptionSpread` instrument decoding (was not correctly handling price increments and empty underlyings)
- Fixed `FuturesSpread` serialization
- Fixed `OptionSpread` serialization

---

Released on 9th September 2024 (UTC).

Enhancements
- Added order book deltas triggering support for `OrderEmulator`
- Added `OrderCancelRejected` event generation for dYdX adapter (1916), thanks davidsblom
- Refined handling of Binance private key types (RSA, Ed25519) and integrated into configs
- Implemented cryptographic signing in Rust (replacing `pycryptodome` for Binance)
- Removed the vendored `tokio-tungstenite` crate (1902), thanks VioletSakura-7

Breaking Changes
None

Fixes
- Fixed `BinanceFuturesEventType` by adding new `TRADE_LITE` member, reflecting the Binance update on 2024-09-03 (UTC)

---

Released on 7th September 2024 (UTC).

Enhancements
- Added dYdX integration (1861, 1868, 1873, 1874, 1875, 1877, 1879, 1880, 1882, 1886, 1887, 1890, 1891, 1896, 1901, 1903, 1907, 1910, 1911, 1913, 1915), thanks davidsblom
- Added composite bar types, bars aggregated from other bar types (1859, 1885, 1888, 1894, 1905), thanks faysou
- Added `OrderBookDeltas.batch` for batching groups of deltas based on record flags (batch until `F_LAST`)
- Added `OrderBookDeltas` batching support for `ParquetDataCatalog` (use `data_cls` of `OrderBookDeltas` to batch with the same flags method as live adapters)
- Added `RetryManagerPool` to abstract common retry functionality for all adapters
- Added `InstrumentClose` functionality for `OrderMatchingEngine`, thanks limx0
- Added `BacktestRunConfig.dispose_on_completion` config option to control post-run disposal behavior for each internal backtest engine (default `True` to retain current behavior)
- Added `recv_window_ms` config option for `BinanceExecClientConfig`
- Added `sl_time_in_force` and `tp_time_in_force` parameters to `OrderFactory.bracket(...)` method
- Added custom `client_order_id` parameters to `OrderFactory` methods
- Added support for Binance RSA and Ed25519 API key types (1908), thanks NextThread
- Added `multiplier` parameter for `CryptoPerpetual` (default 1)
- Implemented `BybitExecutionClient` retry logic for `submit_order`, `modify_order`, `cancel_order` and `cancel_all_orders`
- Improved error modeling and handling in Rust (1866), thanks twitu
- Improved `HttpClient` error handling and added `HttpClientError` exception for Python (1872), thanks twitu
- Improved `WebSocketClient` error handling and added `WebSocketClientError` exception for Python (1876), thanks twitu
- Improved `WebSocketClient.send_text` efficiency (now accepts UTF-8 encoded bytes, rather than a Python string)
- Improved `customdataclass` decorator with `date` field and refined `__repr__` (1900, 1906, 1909), thanks faysou
- Improved standardization of `OrderBookDeltas` parsing and records flags for crypto venues
- Refactored `RedisMessageBusDatabase` to tokio tasks
- Refactored `RedisCacheDatabase` to tokio tasks
- Upgraded `tokio` crate to v1.40.0

Breaking Changes
- Renamed `heartbeat_interval` to `heartbeat_interval_secs` (more explicitly indicates time units)
- Moved `heartbeat_interval_secs` config option to `MessageBusConfig` (the message bus handles external stream processing)
- Changed `WebSocketClient.send_text(...)` to take `data` as `bytes` rather than `str`
- Changed `CryptoPerpetual` Arrow schema to include `multiplier` field
- Changed `CryptoFuture` Arrow schema to include `multiplier` field

Fixes
- Fixed `OrderBook` memory deallocation in Python finalizer (memory was not being freed on object destruction), thanks for reporting zeyuhuan
- Fixed `Order` tags serialization (was not concatenating to a single string), thanks for reporting DevRoss
- Fixed `types_filter` serialization in `MessageBusConfig` during kernel setup
- Fixed `InstrumentProvider` handling of `load_ids_on_start` when elements are already `InstrumentId`s
- Fixed `InstrumentProviderConfig` hashing for `filters` field

---

Released on 19th August 2024 (UTC).

Enhancements
- Added `LiveExecEngineConfig.generate_missing_orders` reconciliation config option to align internal and external position states
- Added `LogLevel::TRACE` (only available in Rust for debug/development builds)
- Added `Actor.subscribe_signal(...)` method and `Data.is_signal(...)` class method (1853), thanks faysou
- Added Binance Futures support for `HEDGE` mode (1846), thanks DevRoss
- Overhauled and refined error modeling and handling in Rust (1849, 1858), thanks twitu
- Improved `BinanceExecutionClient` position report requests (can now filter by instrument and includes reporting for flat positions)
- Improved `BybitExecutionClient` position report requests (can now filter by instrument and includes reporting for flat positions)
- Improved `LiveExecutionEngine` reconciliation robustness and recovery when internal positions do not match external positions
- Improved `customdataclass` decorator constructor to allow more positional arguments (1850), thanks faysou
- Improved `customdataclass` documentation (1854), thanks faysou
- Upgraded `datafusion` crate to v41.0.0
- Upgraded `tokio` crate to v1.39.3
- Upgraded `uvloop` to v0.20.0 (upgrades libuv to v1.48.0)

Breaking Changes
- Changed `VolumeWeightedAveragePrice` calculation formula to use each bars "typical" price (1842), thanks evgenii-prusov
- Changed `OptionContract` constructor parameter ordering and Arrow schema (consistently group option kind and strike price)
- Renamed `snapshot_positions_interval` to `snapshot_positions_interval_secs` (more explicitly indicates time units)
- Moved `snapshot_orders` config option to `ExecEngineConfig` (can now be used for all environment contexts)
- Moved `snapshot_positions` config option to `ExecEngineConfig` (can now be used for all environment contexts)
- Moved `snapshot_positions_interval_secs` config option to `ExecEngineConfig` (can now be used for all environment contexts)

Fixes
- Fixed `Position` exception type on duplicate fill (should be `KeyError` to align with the same error for `Order`)
- Fixed Bybit position report parsing when position is flat (`BybitPositionSide` now correctly handles the empty string)

---