Nautilus-trader

Released on 10th January 2025 (UTC).

Enhancements
- Added `PerContractFeeModel`, thanks stefansimik
- Added `DYDXInternalError` and `DYDXOraclaPrice` data types for dYdX (2155), thanks davidsblom
- Added proper `OrderBookDeltas` flags parsing for Betfair
- Added Binance TradeLite message support (2156), thanks DeevsDeevs
- Added `DataEngineConfig.time_bars_skip_first_non_full_bar` config option (2160), thanks faysou
- Added `execution.fast` support for Bybit (2165), thanks sunlei
- Added catalog helper functions to export data (2135), thanks twitu
- Added additional timestamp properties for `NautilusKernel`
- Added `event_logging` config option for `StrategyConfig` (2183), thanks sunlei
- Added `bar_adaptive_high_low_ordering` to `BacktestVenueConfig` (2188), thanks faysou and stefansimik

Breaking Changes
- Removed optional `value` param from `UUID4` (use `UUID4.from_str(...)` instead), aligns with Nautilus PyO3 API
- Changed `unix_nanos_to_iso8601` to output an ISO 8601 (RFC 3339) format string with nanosecond precision
- Changed `format_iso8601` to output ISO 8601 (RFC 3339) format string with nanosecond precision
- Changed `format_iso8601` `dt` parameter to enforce `pd.Timestamp` (which has nanosecond precision)
- Changed `TradingNode.is_built` from a property to a method `.is_built()`
- Changed `TradingNode.is_running` from a property to a method `.is_running()`
- Changed `OrderInitialized` Arrow schema (`linked_order_ids` and `tags` data types changed from `string` to `binary`)
- Changed order dictionary representation field types for `avg_px` and `slippage` from `str` to `float` (as out of alignment with position events)
- Changed `aggregation_source` filter parameter for `Cache.bar_types(...)` to optional with default of `None`

Internal Improvements
- Improved market order handling when no size available in book (now explicitly rejects)
- Improved validation for `TradeTick` by ensuring `size` is always positive
- Improved validation for `OrderBookDelta` by ensuring `order.size` is positive when `action` is either `ADD` or `UPDATE`
- Improved validation for `BarSpecification` by ensuring `step` is always positive
- Standardized ISO 8601 timestamps to RFC 3339 spec with nanosecond precision
- Standardized flags for `OrderBookDeltas` parsing across adapters
- Refined parsing candles for dYdX (2148), thanks davidsblom
- Refined imports for type hints in Bybit (2149), thanks sunlei
- Refined private WebSocket message processing for Bybit (2170), thanks sunlei
- Refined WebSocket client re-subscribe log for Bybit (2179), thanks sunlei
- Refined margin balance report for dYdX (2154), thanks davidsblom
- Enhanced `lotSizeFilter` field for Bybit (2166), thanks sunlei
- Renamed WebSocket private client for Bybit (2180), thanks sunlei
- Added unit tests for custom dYdX types (2163), thanks davidsblom
- Allow bar aggregators to persist after `request_aggregated_bars` (2144), thanks faysou
- Handle directory and live streams to catalog (2153), thanks limx0
- Use timeout when initializing account for dYdX (2169), thanks davidsblom
- Use retry manager when sending websocket messages for dYdX (2196), thanks davidsblom
- Refined error logs when sending pong for dYdX (2184), thanks davidsblom
- Optimized message bus topic `is_matching` (2151), thanks ryantam626
- Added tests for `bar_adaptive_high_low_ordering` (2197), thanks faysou
- Ported `OrderManager` to Rust (2161), thanks Pushkarm029
- Ported trailing stop logic to Rust (2174), thanks DeevsDeevs
- Ported `FeeModel` to Rust (2191), thanks filipmacek
- Implemented IDs generator for `OrderMatchingEngine` in Rust (2193), thanks filipmacek
- Upgraded Cython to v3.1.0a1
- Upgraded `tokio` crate to v1.43.0
- Upgraded `datafusion` crate to v44.0.0

Fixes
- Fixed type check for `DataClient` on requests to support clients other than `MarketDataClient`
- Fixed processing trade ticks from bars in `OrderMatchingEngine` - that could result in zero-size trades, thanks for reporting stefansimik
- Fixed `instrument is None` check flows for `DataEngine` and `PolymarketExecutionClient`
- Fixed instrument updates in `BetfairDataClient` (2152), thanks limx0
- Fixed processing of time events on backtest completion when they occur after the final data timestamp
- Fixed missing enum member `CANCELED_MARKET_RESOLVED` for `PolymarketOrderStatus`
- Fixed missing `init_id` field from some order `.to_dict()` representations
- Fixed writing `DYDXOraclePrice` to catalog (2158), thanks davidsblom
- Fixed account balance for dYdX (2167), thanks davidsblom
- Fixed markets schema for dYdX (2190), thanks davidsblom
- Fixed missing `OrderEmulated` and `OrderReleased` Arrow schemas
- Fixed websocket public channel reconnect for Bybit (2176), thanks sunlei
- Fixed execution report parsing for Binance Spot (client order ID empty string now becomes a UUID4 string)
- Fixed docs typo for `fill_order` function in `OrderMatchingEngine` (2189), thanks filipmacek

Documentation Updates
- Added docs for `Cache`, slippage and spread handling in backtesting (2162), thanks stefansimik
- Added docs for `FillModel` and bar based execution (2187), thanks stefansimik
- Added docs for choosing data (cost vs. accuracy) and bars OHLC processing (2195), thanks stefansimik
- Added docs for bar processing in backtests (2198), thanks stefansimik
- Added docs for timestamp and UUID specs

---

Released on 25th December 2024 (UTC).

Enhancements
- Added WebSocket API trading support for Bybit (2129), thanks sunlei
- Added `BybitOrderBookDeltaDataLoader` with tutorial for Bybit backtesting (2131), thanks DeevsDeevs
- Added margin and commission docs (2128), thanks stefansimik
- Added optional `depth` parameter for some `OrderBook` methods
- Added trade execution support where trades are processed by the matching engine (can be useful backtesting with throttled book and trades data)
- Refactored to use `exchange` MIC code as `venue` for instrument IDs with Databento GLBX dataset (2108, 2121, 2124, 2126), thanks faysou
- Refactored to use `self.config` attributes consistently (2120), thanks stefansimik

Internal Improvements
- Optimized `UUID4::new()` avoiding unnecessary string allocation, achieving a ~2.8x performance improvement (added benches)
- Upgraded v4-proto for dYdX (2136), thanks davidsblom
- Upgraded `databento` crate to v0.17.0

Breaking Changes
- Moved `BinanceOrderBookDeltaDataLoader` from `nautilus_trader.persistence.loaders` to `nautilus_trader.adapters.binance.loaders`

Fixes
- Fixed multi-threaded monotonicity for `AtomicTime` in real-time mode
- Fixed timeout error code for Bybit (2130), thanks sunlei
- Fixed instruments info retrieval for Bybit (2134), thanks sunlei
- Fixed `request_aggregated_bars` metadata handling (2137), thanks faysou
- Fixed demo notebook `backtest_high_level.ipynb` (2142), thanks stefansimik

---

Released on 15th December 2024 (UTC).

Enhancements
- Added specific `params` for data subscriptions and requests which supports Databento `bbo-1s` and `bbo-1m` quotes (2083, 2094), thanks faysou
- Added support for `STOP_LIMIT` entry order type for `OrderFactory.bracket(...)`
- Added `.group_bids(...)` and `.group_asks(...)` for `OrderBook`
- Added `.bids_to_dict()` and `.asks_to_dict()` for `OrderBook`
- Added `ShutdownSystem` command and `shutdown_system(...)` method for components (system-wide shutdown for backtest, sandbox, or live environments)
- Added `max_ws_reconnection_tries` to `BybitDataClientConfig` (2100), thanks sunlei
- Added additional API functionality for Bybit (2102), thanks sunlei
- Added position and execution.fast subscriptions for Bybit (2104), thanks sunlei
- Added `max_ws_reconnection_tries` to `BybitExecClientConfig` (2109), thanks sunlei
- Added `margin_init`, `margin_maint`, `maker_fee`, `taker_fee` params and attributes for `FuturesContract`
- Added `margin_init`, `margin_maint`, `maker_fee`, `taker_fee` params and attributes for `FuturesSpread`
- Added `margin_init`, `margin_maint`, `maker_fee`, `taker_fee` params and attributes for `OptionContract`
- Added `margin_init`, `margin_maint`, `maker_fee`, `taker_fee` params and attributes for `OptionSpread`
- Improved Databento symbology support for Interactive Brokers (2113), thanks rsmb7z
- Improved support of `STOP_MARKET` and `STOP_LIMIT` orders for dYdX (2069), thanks Saransh-Bhandari
- Improved timer validation for `interval_ns` (avoids panicking from Rust)

Internal Improvements
- Added `.bids_as_map()` and `.asks_as_map()` for `OrderBook` in Rust
- Added type stubs for `core` subpackage
- Added type stubs for `common` and `model` enums
- Added type stubs for `common.messages`
- Added re-exports and module declarations to enhance code ergonomics and improve import discoverability
- Added subscriptions for block height websocket messages for dYdX (2085), thanks davidsblom
- Added sccache in CI (2093), thanks sunlei
- Refined `BybitWebSocketClient` private channel authentication (2101), thanks sunlei
- Refined `BybitWebSocketClient` subscribe and unsubscribe (2105), thanks sunlei
- Refined place order class definitions for Bybit (2106), thanks sunlei
- Refined `BybitEnumParser` (2107), thanks sunlei
- Refined batch cancel orders for Bybit (2111), thanks sunlei
- Upgraded `tokio` crate to v1.42.0

Breaking Changes
- Renamed `Level` to `BookLevel` (standardizes order book type naming conventions)
- Renamed `Ladder` to `BookLadder` (standardizes order book type naming conventions)
- Changed `FuturesContract` Arrow schema (added `margin_init`, `margin_maint`, `maker_fee`, `taker_fee`)
- Changed `FuturesSpread` Arrow schema (added `margin_init`, `margin_maint`, `maker_fee`, `taker_fee`)
- Changed `OptionContract` Arrow schema (added `margin_init`, `margin_maint`, `maker_fee`, `taker_fee`)
- Changed `OptionSpread` Arrow schema (added `margin_init`, `margin_maint`, `maker_fee`, `taker_fee`)

Fixes
- Fixed data requests when specifying `end` with no catalog registered (comparison between `pd.Timestamp` and `NoneType`)
- Fixed `BEST_EFFORT_CANCELED` order status report for dYdX (2082), thanks davidsblom
- Fixed order handling for `BEST_EFFORT_CANCELED` messages of dYdX (2095), thanks davidsblom
- Fixed specifying price for market orders on dYdX (2088), thanks davidsblom
- Fixed interest rate curve custom data and interpolation (2090), thanks gcheshkov
- Fixed `BybitHttpClient` error handling when not a JSON string (2096), thanks sunlei
- Fixed `BybitWebSocketClient` private channel reconnect (2097), thanks sunlei
- Fixed incorrect order side use in `BybitExecutionClient` (2098), thanks sunlei
- Fixed default `http_base_url` for Bybit (2110), thanks sunlei

---

Released on 29th November 2024 (UTC).

Enhancements
- Implemented mixed catalog data requests with catalog update (2043), thanks faysou
- Added Databento symbology support for Interactive Brokers (2073), thanks rsmb7z
- Added `metadata` parameter for data requests (2043), thanks faysou
- Added `STOP_MARKET` and `STOP_LIMIT` order support for dYdX (2066), thanks davidsblom
- Added `max_reconnection_tries` to data client config for dYdX (2066), thanks davidsblom
- Added wallet subscription for Bybit (2076), thanks sunlei
- Added docs clarity on loading historical bars (2078), thanks dodofarm
- Added `price_precision` optional parameter for `DatabentoDataLoader` methods
- Improved `Cache` behavior when adding more recent quotes, trades, or bars (now adds to cache)

Internal Improvements
- Ported `Portfolio` and `AccountManager` to Rust (2058), thanks Pushkarm029
- Implemented `AsRef<str>` for `Price`, `Money`, and `Currency`
- Improved expired timer cleanup in clocks (2064), thanks twitu
- Improved live engines error logging (will now log all exceptions rather than just `RuntimeError`)
- Improved symbol normalization for Tardis
- Improved historical bar request performance for Tardis
- Improved `TradeId` Debug implementation to display value as proper UTF-8 string
- Refined `HttpClient` for use directly from Rust
- Refined Databento decoder (removed currency hard coding and use of `unsafe`)
- Upgraded `datafusion` crate to v43.0.0 (2056), thanks twitu

Breaking Changes
- Renamed `TriggerType.LAST_TRADE` to `LAST_PRICE` (more conventional terminology)

Fixes
- Fixed missing venue -> exchange mappings for Tardis integration
- Fixed account balance and order status parsing for dYdX (2067), thanks davidsblom
- Fixed parsing best effort opened order status for dYdX (2068), thanks davidsblom
- Fixed occasionally incorrect `price_precision`, `multiplier` and `lot_size` decoding for Databento instruments
- Fixed missing Arrow schemas for instrument deserialization
- Reconcile order book for dYdX when inconsistent (2077), thanks davidsblom

---

Released on 17th November 2024 (UTC).

Enhancements
- Added `TardisDataClient` providing live data streams from a Tardis Machine WebSocket server
- Added `TardisInstrumentProvider` providing instrument definitions from Tardis through the HTTP instrument metadata API
- Added `Portfolio.realized_pnl(...)` method for per instrument realized PnL (based on positions)
- Added `Portfolio.realized_pnls(...)` method for per venue realized PnL (based on positions)
- Added configuration warning for `InstrumentProvider` (to warn when node starts with no instrument loading)
- Implemented Tardis optional [symbol normalization](https://nautilustrader.io/docs/nightly/integrations/tardis/#symbology-and-normalization)
- Implemented `WebSocketClient` reconnection retries (2044), thanks davidsblom
- Implemented `OrderCancelRejected` event generation for Binance and Bybit
- Implemented `OrderModifyRejected` event generation for Binance and Bybit
- Improved `OrderRejected` handling of `reason` string (`None` is now allowed which will become the string `'None'`)
- Improved `OrderCancelRejected` handling of `reason` string (`None` is now allowed which will become the string `'None'`)
- Improved `OrderModifyRejected` handling of `reason` string (`None` is now allowed which will become the string `'None'`)

Internal Improvements
- Ported `RiskEngine` to Rust (2035), thanks Pushkarm029 and twitu
- Ported `ExecutionEngine` to Rust (2048), thanks twitu
- Added globally shared data channels to send events from engines to Runner in Rust (2042), thanks twitu
- Added LRU caching for dYdX HTTP client (2049), thanks davidsblom
- Improved identifier constructors to take `AsRef<str>` for a cleaner more flexible API
- Refined identifiers `From` trait impls
- Refined `InstrumentProvider` initialization behavior and logging
- Refined `LiveTimer` cancel and performance testing
- Simplified `LiveTimer` cancellation model (2046), thanks twitu
- Refined Bybit HMAC authentication signatures (now using Rust implemented function)
- Refined Tardis instrument ID parsing
- Removed Bybit `msgspec` redundant import alias (2050), thanks sunlei
- Upgraded `databento` crate to v0.16.0

Breaking Changes
None

Fixes
- Fixed loading specific instrument IDs for `InstrumentProviderConfig`
- Fixed PyO3 instrument conversions for `raw_symbol` (was incorrectly using the normalized symbol)
- Fixed reconcile open orders and account websocket message for dYdX (2039), thanks davidsblom
- Fixed market order `avg_px` for Polymarket trade reports
- Fixed Betfair clients keepalive (2040), thanks limx0
- Fixed Betfair reconciliation (2041), thanks limx0
- Fixed Betfair customer order ref limit to 32 chars
- Fixed Bybit handling of `PARTIALLY_FILLED_CANCELED` status orders
- Fixed Polymarket size precision for `BinaryOption` instruments (precision 6 to match USDC.e)
- Fixed adapter instrument reloading (providers were not reloading instruments at the configured interval due to internal state flags)
- Fixed static time logging for `BacktestEngine` when running with `use_pyo3` logging config
- Fixed in-flight orders check and improve error handling (2053), thanks davidsblom
- Fixed dYdX handling for liquidated fills (2052), thanks davidsblom
- Fixed `BybitResponse.time` field as optional `int` (2051), thanks sunlei
- Fixed single instrument requests for `DatabentoDataClient` (was incorrectly calling `_handle_instruments` instead of `_handle_instrument`), thanks for reporting Emsu
- Fixed `fsspec` recursive globbing behavior to ensure only file paths are included, and bumped dependency to version 2024.10.0
- Fixed jupyterlab url typo (2057), thanks Alsheh

---

Released on 3rd November 2024 (UTC).

Enhancements
- Added Tardis Machine and HTTP API integration in Python and Rust
- Added `LiveExecEngineConfig.open_check_interval_secs` config option to actively reconcile open orders with the venue
- Added aggregation of bars from historical data (2002), thanks faysou
- Added monthly and weekly bar aggregations (2025), thanks faysou
- Added `raise_exception` optional parameter to `TradingNode.run` (2021), thanks faysou
- Added `OrderBook.get_avg_px_qty_for_exposure` in Rust (1893), thanks elementace
- Added timeouts to Interactive Brokers adapter configurations (2026), thanks rsmb7z
- Added optional time origins for time bar aggregation (2028), thanks faysou
- Added Polymarket position status reports and order status report generation based on fill reports
- Added USDC.e (PoS) currency (used by Polymarket) to internal currency map
- Upgraded Polymarket WebSocket API to new version

Internal Improvements
- Ported analysis subpackage to Rust (2016), thanks Pushkarm029
- Improved Postgres testing (2018), thanks filipmacek
- Improved Redis version parsing to support truncated versions (improves compatibility with Redis-compliant databases)
- Refined Arrow serialization (record batch functions now also available in Rust)
- Refined core `Bar` API to remove unnecessary unwraps
- Standardized network client logging
- Fixed all PyO3 deprecations for API breaking changes
- Fixed all clippy warning lints for PyO3 changes (2030), thanks Pushkarm029
- PyO3 upgrade refactor and repair catalog tests (2032), thanks twitu
- Upgraded `pyo3` crate to v0.22.5
- Upgraded `pyo3-async-runtimes` crate to v0.22.0
- Upgraded `tokio` crate to v1.41.0

Breaking Changes
- Removed PyO3 `DataTransformer` (was being used for namespacing, so refactored to separate functions)
- Moved `TEST_DATA_DIR` constant from `tests` to `nautilus_trader` package (2020), thanks faysou

Fixes
- Fixed use of Redis `KEYS` command which, is unsupported in cluster environments (replaced with `SCAN` for compatibility)
- Fixed decoding fill HTTP messages for dYdX (2022), thanks davidsblom
- Fixed account balance report for dYdX (2024), thanks davidsblom
- Fixed Interactive Brokers market data client subscription log message (2012), thanks marcodambros
- Fixed Polymarket execution reconciliation (was not able to reconcile from closed orders)
- Fixed catalog query mem leak test (2031), thanks Pushkarm029
- Fixed `OrderInitialized.to_dict()` `tags` value type to `list[str]` (was a concatenated `str`)
- Fixed `OrderInitialized.to_dict()` `linked_order_ids` value type to `list[str]` (was a concatenated `str`)
- Fixed Betfair clients shutdown (2037), thanks limx0

---