Python-can

=============

Breaking Changes
----------------
* The ``can.BitTiming`` class was replaced with the new
``can.BitTiming`` and `can.BitTimingFd` classes (1468, 1515).
Early adopters of ``can.BitTiming`` will need to update their code. Check the
[documentation](https://python-can.readthedocs.io/en/develop/bit_timing.html)
for more information. Currently, the following interfaces support the new classes:
* canalystii (1468)
* cantact (1468)
* nixnet (1520)
* pcan (1514)
* vector (1470, 1516)

There are open pull requests for kvaser (1510), slcan (1512) and usb2can (1511). Testing
and reviewing of these open PRs would be most appreciated.

Features
--------

IO
* Add support for MF4 files (1289).
* Add support for version 2 TRC files and other TRC file enhancements (1530).

Type Annotations
* Export symbols to satisfy type checkers (1547, 1551, 1558, 1568).

Interface Improvements
* Add ``__del__`` method to ``can.BusABC`` to automatically release resources (1489, 1564).
* pcan: Update PCAN Basic to 4.6.2.753 (1481).
* pcan: Use select instead of polling on Linux (1410).
* socketcan: Use ip link JSON output in ``find_available_interfaces`` (1478).
* socketcan: Enable SocketCAN interface tests in GitHub CI (1484).
* slcan: improve receiving performance (1490).
* usb2can: Stop using root logger (1483).
* usb2can: Faster channel detection on Windows (1480).
* vector: Only check sample point instead of tseg & sjw (1486).
* vector: add VN5611 hwtype (1501).

Documentation
-------------
* Add new section about related tools to documentation. Add a list of
plugin interface packages (1457).

Bug Fixes
---------
* Automatic type conversion for config values (1498, 1499).
* pcan: Fix ``Bus.__new__`` for CAN-FD interfaces (1458, 1460).
* pcan: Fix Detection of Library on Windows on ARM (1463).
* socketcand: extended ID bug fixes (1504, 1508).
* vector: improve robustness against unknown HardwareType values (1500, 1502).

Deprecations
------------
* The ``bustype`` parameter of ``can.Bus`` is deprecated and will be
removed in version 5.0, use ``interface`` instead. (1462).
* The ``context`` parameter of ``can.Bus`` is deprecated and will be
removed in version 5.0, use ``config_context`` instead. (1474).
* The ``bit_timing`` parameter of ``CantactBus`` is deprecated and will be
removed in version 5.0, use ``timing`` instead. (1468).
* The ``bit_timing`` parameter of ``CANalystIIBus`` is deprecated and will be
removed in version 5.0, use ``timing`` instead. (1468).
* The ``brs`` and ``log_errors`` parameters of `` NiXNETcanBus`` are deprecated
and will be removed in version 5.0. (1520).

Miscellaneous
-------------
* Use high resolution timer on Windows to improve
timing precision for BroadcastManager (1449).
* Improve ThreadBasedCyclicSendTask timing (1539).
* Make code examples executable on Linux (1452).
* Fix CanFilter type annotation (1456).
* Fix ``The entry_points().get`` deprecation warning and improve
type annotation of ``can.interfaces.BACKENDS`` (1465).
* Add ``ignore_config`` parameter to ``can.Bus`` (1474).
* Add deprecation period to utility function ``deprecated_args_alias`` (1477).
* Add `ruff` to the CI system (1551)

=============

Breaking Changes
----------------

* ``windows-curses`` was moved to optional dependencies (1395).
Use ``pip install python-can[viewer]`` if you are using the ``can.viewer``
script on Windows.
* The attributes of ``can.interfaces.vector.VectorChannelConfig`` were renamed
from camelCase to snake_case (1422).


Features
--------

IO

* The canutils logger preserves message direction (1244)
and uses common interface names (e.g. can0) instead of just
channel numbers (1271).
* The ``can.logger`` script accepts the ``-a, --append`` option
to add new data to an existing log file (1326, 1327, 1361).
Currently only the blf-, canutils- and csv-formats are supported.
* All CLI ``extra_args`` are passed to the bus, logger
and player initialisation (1366).
* Initial support for TRC files (1217)

Type Annotations
* python-can now includes the ``py.typed`` marker to support type checking
according to PEP 561 (1344).

Interface Improvements
* The gs_usb interface can be selected by device index instead
of USB bus/address. Loopback frames are now correctly marked
with the ``is_rx`` flag (1270).
* The PCAN interface can be selected by its device ID instead
of just the channel name (1346).
* The PCAN Bus implementation supports auto bus-off reset (1345).
* SocketCAN: Make ``find_available_interfaces()`` find slcanX interfaces (1369).
* Vector: Add xlGetReceiveQueueLevel, xlGenerateSyncPulse and
xlFlushReceiveQueue to xldriver (1387).
* Vector: Raise a CanInitializationError, if the CAN settings can not
be applied according to the arguments of ``VectorBus.__init__`` (1426).
* Ixxat bus now implements BusState api and detects errors (1141)

Bug Fixes
---------

* Improve robustness of USB2CAN serial number detection (1129).
* Fix channel2int conversion (1268, 1269).
* Fix BLF timestamp conversion (1266, 1273).
* Fix timestamp handling in udp_multicast on macOS (1275, 1278).
* Fix failure to initiate the Neousys DLL (1281).
* Fix AttributeError in IscanError (1292, 1293).
* Add missing vector devices (1296).
* Fix error for DLC > 8 in ASCReader (1299, 1301).
* Set default mode for FileIOMessageWriter to wt instead of rt (1303).
* Fix conversion for port number from config file (1309).
* Fix fileno error on Windows (1312, 1313, 1333).
* Remove redundant ``writer.stop()`` call that throws error (1316, 1317).
* Detect and cast types of CLI ``extra_args`` (1280, 1328).
* Fix ASC/CANoe incompatibility due to timestamp format (1315, 1362).
* Fix MessageSync timings (1372, 1374).
* Fix file name for compressed files in SizedRotatingLogger (1382, 1683).
* Fix memory leak in neoVI bus where message_receipts grows with no limit (1427).
* Raise ValueError if gzip is used with incompatible log formats (1429).
* Allow restarting of transmission tasks for socketcan (1440)

Miscellaneous
-------------

* Allow ICSApiError to be pickled and un-pickled (1341)
* Sort interface names in CLI API to make documentation reproducible (1342)
* Exclude repository-configuration from git-archive (1343)
* Improve documentation (1397, 1401, 1405, 1420, 1421, 1434)
* Officially support Python 3.11 (1423)
* Migrate code coverage reporting from Codecov to Coveralls (1430)
* Migrate building docs and publishing releases to PyPi from Travis-CI to GitHub Actions (1433)

====

TL;DR: This release includes a ton of improvements from 2.5 years of development! 🎉 Test thoroughly after switching.

For more than two years, there was no major release of *python-can*.
However, development was very much active over most of this time, and many parts were switched out and improved.
Over this time, over 530 issues and PRs have been resolved or merged, and discussions took place in even more.
Statistics of the final diff: About 200 files changed due to ~22k additions and ~7k deletions from more than thirty contributors.

This changelog diligently lists the major changes but does not promise to be the complete list of changes.
Therefore, users are strongly advised to thoroughly test their programs against this new version.
Re-reading the documentation for your interfaces might be helpful too as limitations and capabilities might have changed or are more explicit.
While we did try to avoid breaking changes, in some cases it was not feasible and in particular, many implementation details have changed.

Major features
--------------

* Type hints for the core library and some interfaces (652 and many others)
* Support for Python 3.7-3.10+ only (dropped support for Python 2.* and 3.5-3.6) (528 and many others)
* [Granular and unified exceptions](https://python-can.readthedocs.io/en/develop/api.html#errors) (356, 562, 1025; overview in 1046)
* [Support for automatic configuration detection](https://python-can.readthedocs.io/en/develop/api.html#can.detect_available_configs) in most interfaces (303, 640, 641, 811, 1077, 1085)
* Better alignment of interfaces and IO to common conventions and semantics

New interfaces
--------------

* udp_multicast (644)
* robotell (731)
* cantact (853)
* gs_usb (905)
* nixnet (968, 1154)
* neousys (980, 1076)
* socketcand (1140)
* etas (1144)

Improved interfaces
-------------------

* socketcan
* Support for multiple Cyclic Messages in Tasks (610)
* Socketcan crash when attempting to stop CyclicSendTask with same arbitration ID (605, 638, 720)
* Relax restriction of arbitration ID uniqueness for CyclicSendTask (721, 785, 930)
* Add nanosecond resolution time stamping to socketcan (938, 1015)
* Add support for changing the loopback flag (960)
* Socketcan timestamps are missing sub-second precision (1021, 1029)
* Add parameter to ignore CAN error frames (1128)
* socketcan_ctypes
* Removed and replaced by socketcan after deprecation period
* socketcan_native
* Removed and replaced by socketcan after deprecation period
* vector
* Add chip state API (635)
* Add methods to handle non message events (708)
* Implement XLbusParams (718)
* Add support for VN8900 xlGetChannelTime function (732, 733)
* Add vector hardware config popup (774)
* Fix Vector CANlib treatment of empty app name (796, 814)
* Make VectorError pickleable (848)
* Add methods get_application_config(), set_application_config() and set_timer_rate() to VectorBus (849)
* Interface arguments are now lowercase (858)
* Fix errors using multiple Vector devices (898, 971, 977)
* Add more interface information to channel config (917)
* Improve timestamp accuracy on Windows (934, 936)
* Fix error with VN8900 (1184)
* Add static typing (1229)
* PCAN
* Do not incorrectly reset CANMsg.MSGTYPE on remote frame (659, 681)
* Add support for error frames (711)
* Added keycheck for windows platform for better error message (724)
* Added status_string method to return simple status strings (725)
* Fix timestamp timezone offset (777, 778)
* Add [Cygwin](https://www.cygwin.com/) support (#840)
* Update PCAN basic Python file to February 7, 2020 (929)
* Fix compatibility with the latest macOS SDK (947, 948, 957, 976)
* Allow numerical channel specifier (981, 982)
* macOS: Try to find libPCBUSB.dylib before loading it (983, 984)
* Disable command PCAN_ALLOW_ERROR_FRAMES on macOS (985)
* Force english error messages (986, 993, 994)
* Add set/get device number (987)
* Timestamps are silently incorrect on Windows without uptime installed (1053, 1093)
* Implement check for minimum version of pcan library (1065, 1188)
* Handle case where uptime is imported successfully but returns None (1102, 1103)
* slcan
* Fix bitrate setting (691)
* Fix fileno crash on Windows (924)
* ics_neovi
* Filter out Tx error messages (854)
* Adding support for send timeout (855)
* Raising more precise API error when set bitrate fails (865)
* Avoid flooding the logger with many errors when they are the same (1125)
* Omit the transmit exception cause for brevity (1086)
* Raise ValueError if message data is over max frame length (1177, 1181)
* Setting is_error_frame message property (1189)
* ixxat
* Raise exception on busoff in recv() (856)
* Add support for 666 kbit/s bitrate (911)
* Add function to list hwids of available devices (926)
* Add CAN FD support (1119)
* seeed
* Fix fileno crash on Windows (902)
* kvaser
* Improve timestamp accuracy on Windows (934, 936)
* usb2can
* Fix "Error 8" on Windows and provide better error messages (989)
* Fix crash on initialization (1248, 1249)
* Pass flags instead of flags_t type upon initialization (1252)
* serial
* Fix "TypeError: cannot unpack non-iterable NoneType" and more robust error handling (1000, 1010)
* canalystii
* Fix is_extended_id (1006)
* Fix transmitting onto a busy bus (1114)
* Replace binary library with python driver (726, 1127)

Other API changes and improvements
----------------------------------

* CAN FD frame support is pretty complete (963)
* ASCWriter (604) and ASCReader (741)
* Canutils reader and writer (1042)
* Logger, viewer and player tools can handle CAN FD (632)
* Many bugfixes and more testing coverage
* IO
* [Log rotation](https://python-can.readthedocs.io/en/develop/listeners.html#can.SizedRotatingLogger) (648, 874, 881, 1147)
* Transparent (de)compression of [gzip](https://docs.python.org/3/library/gzip.html) files for all formats (#1221)
* Add [plugin support to can.io Reader/Writer](https://python-can.readthedocs.io/en/develop/listeners.html#listener) (783)
* ASCReader/Writer enhancements like increased robustness (820, 1223, 1256, 1257)
* Adding absolute timestamps to ASC reader (761)
* Support other base number (radix) at ASCReader (764)
* Add [logconvert script](https://python-can.readthedocs.io/en/develop/scripts.html#can-logconvert) (1072, 1194)
* Adding support for gzipped ASC logging file (.asc.gz) (1138)
* Improve [IO class hierarchy](https://python-can.readthedocs.io/en/develop/internal-api.html#module-can.io.generic) (1147)
* An [overview over various "virtual" interfaces](https://python-can.readthedocs.io/en/develop/interfaces/virtual.html#other-virtual-interfaces) (644)
* Make ThreadBasedCyclicSendTask event based & improve timing accuracy (656)
* Ignore error frames in can.player by default, add --error-frames option (690)
* Add an error callback to ThreadBasedCyclicSendTask (743, 781)
* Add direction to CAN messages (773, 779, 780, 852, 966)
* Notifier no longer raises handled exceptions in rx_thread (775, 789) but does so if no listener handles them (1039, 1040)
* Changes to serial device number decoding (869)
* Add a default fileno function to the BusABC (877)
* Disallow Messages to simultaneously be "FD" and "remote" (1049)
* Speed up interface plugin imports by avoiding pkg_resources (1110)
* Allowing for extra config arguments in can.logger (1142, 1170)
* Add changed byte highlighting to viewer.py (1159)
* Change DLC to DL in Message.\_\_str\_\_() (1212)

Other Bugfixes
--------------

* BLF PDU padding (459)
* stop_all_periodic_tasks skipping every other task (634, 637, 645)
* Preserve capitalization when reading config files (702, 1062)
* ASCReader: Skip J1939Tp messages (701)
* Fix crash in Canutils Log Reader when parsing RTR frames (713)
* Various problems with the installation of the library
* ASCWriter: Fix date format to show correct day of month (754)
* Fixes that some BLF files can't be read ( 763, 765)
* Seek for start of object instead of calculating it (786, 803, 806)
* Only import winreg when on Windows (800, 802)
* Find the correct Reader/Writer independently of the file extension case (895)
* RecursionError when unpickling message object (804, 885, 904)
* Move "filelock" to neovi dependencies (943)
* Bus() with "fd" parameter as type bool always resolved to fd-enabled configuration (954, 956)
* Asyncio code hits error due to deprecated loop parameter (1005, 1013)
* Catch time before 1970 in ASCReader (1034)
* Fix a bug where error handlers were not called correctly (1116)
* Improved user interface of viewer script (1118)
* Correct app_name argument in logger (1151)
* Calling stop_all_periodic_tasks() in BusABC.shutdown() and all interfaces call it on shutdown (1174)
* Timing configurations do not allow int (1175)
* Some smaller bugfixes are not listed here since the problems were never part of a proper release
* ASCReader & ASCWriter using DLC as data length (1245, 1246)

Behind the scenes & Quality assurance
-------------------------------------

* We publish both source distributions (`sdist`) and binary wheels (`bdist_wheel`) (1059, 1071)
* Many interfaces were partly rewritten to modernize the code or to better handle errors
* Performance improvements
* Dependencies have changed
* Derive type information in Sphinx docs directly from type hints (654)
* Better documentation in many, many places; This includes the examples, README and python-can developer resources
* Add issue templates (1008, 1017, 1018, 1178)
* Many continuous integration (CI) discussions & improvements (for example: 951, 940, 1032)
* Use the [mypy](https://github.com/python/mypy) static type checker (#598, 651)
* Use [tox](https://tox.wiki/en/latest/) for testing (#582, 833, 870)
* Use [Mergify](https://mergify.com/) (#821, 835, 937)
* Switch between various CI providers, abandoned [AppVeyor](https://www.appveyor.com/) (#1009) and partly [Travis CI](https://travis-ci.org/), ended up with mostly [GitHub Actions](https://docs.github.com/en/actions) (#827, 1224)
* Use the [black](https://black.readthedocs.io/en/stable/) auto-formatter (#950)
* [Good test coverage](https://app.codecov.io/gh/hardbyte/python-can/branch/develop) for all but the interfaces
* Testing: Many of the new features directly added tests, and coverage of existing code was improved too (for example: 1031, 581, 585, 586, 942, 1196, 1198)

====

Last call for Python2 support.

* 850 Fix socket.error is a deprecated alias of OSError used on Python versions lower than 3.3.

====

Backported fixes from 4.x development branch which targets Python 3.

* 798 Backport caching msg.data value in neovi interface.
* 796 Fix Vector CANlib treatment of empty app name.
* 771 Handle empty CSV file.
* 741 ASCII reader can now handle FD frames.
* 740 Exclude test packages from distribution.
* 713 RTR crash fix in canutils log reader parsing RTR frames.
* 701 Skip J1939 messages in ASC Reader.
* 690 Exposes a configuration option to allow the CAN message player to send error frames
(and sets the default to not send error frames).
* 638 Fixes the semantics provided by periodic tasks in SocketCAN interface.
* 628 Avoid padding CAN_FD_MESSAGE_64 objects to 4 bytes.
* 617 Fixes the broken CANalyst-II interface.
* 605 Socketcan BCM status fix.

====

Minor bug fix release addressing issue in PCAN RTR.