Pyocd

New features

- pyOCD now supports managed CMSIS-Packs. A new `pack` subcommand has been added to the `pyocd` tool that lets you install packs to add target support with a single command. See 610 and the new [target support documentation](https://github.com/mbedmicro/pyOCD/blob/master/docs/target_support.md) for details.
- Target auto-detection is supported for STLinkV2-1 probes (those with a USB MSD volume containing an "mbed.htm" file), enabling the same ease of use provided by the DAPLink firmware.
- SVD support is now built-in. Most built-in targets have SVD files integrated into pyOCD, and all pack targets should support SVD. The main use right now is through the commander, where you can `show peripherals` and read or write peripheral registers by name.
- The gdbserver supports the range step command from gdb. This may improve step performance slightly.

Boards and targets

- Fixed a number of built-in targets where the flash algo was accidentally dropped from the flash region definition in the 0.19.0 release: lpc11xx_32, lpc11u24, lpc1768, lpc54114, lpc54608, lpc824, kl28z, stm32f429xx, stm32f439xx.
- Adjusted Musca targets' memory map to work around chip errata.
- Fixed page size and enabled double buffering for some built-in STM32 targets: stm32f412xx, stm32f429xx, stm32f439xx, stm32l031xx.

Changes

- Logging more information about the DP and AP types and versions.
- More changes to reduce the amount of libusb related log messages. Many debug messages were removed, and warnings about possible permissions issues are only logged once per device per process.
- Added `debug.log_flm_info` user option to log debug messages with details of .FLM flash algorithms.
- Commander `show map` command prints both sector and page sizes.
- A universal wheel of pyOCD is now released to PyPI in addition to the source distribution.

Fixes

- Flash programming will not attempt to read data from flash regions that are marked as non-readable.
- Fixed the `get_probe_with_id()` APIs on debug probes.
- Correctly decoding the USB serial number for certain versions of STLinkV2 where the serial number is incorrectly encoded.
- Catching additional errors that could be thrown by USB devices when checking to see if they provide a CMSIS-DAPv2 interface.
- Commander `savemem` command inits the flash algo when accessing flash regions with `is_powered_on_boot` flash set to false.
- High level flash erase operations, e.g. via `pyocd erase`, correctly deal with sector and page sizes being different.
- Invalid CMSIS-Pack paths are handled gracefully with a warning log message, but pyOCD continues executing (unless the selected target type is in the missing pack, in which case you'll get an unknown target type error).
- Removed websocket-client and future packages from requirements.

Python API

- The most recently created `Session` instance is accessible via `Session.get_current()`.

Testing

- Fixed `basic_test.py` for flash regions with small number of sectors.
- `flash_loader_test.py` fixes.

New features

- Support for different flash erase sector and program page sizes (452).
- Initial v8-M support, with known issues.
- New commander commands to run the gdbserver from within commander, to view and modify user options, to change the MEM-AP used for memory transfers, and to view and modify the current HPROT/HNONSEC for the selected AP.

Boards and targets

Note: there are still issues with most of the new v8-M targets.

- NXP LPC55S69
- Arm Musca-A1
- Arm Musca-B1

Changes

- pyOCD checks that at least one CPU was found and if not, reports a more understandable error.
- The core number is included in the log message indicating a core was discovered.
- `pyocd` was added to the commander Python expression evaluation namespace, so that all of pyOCD's Python API can be accessed.
- Major cleanup of log messages related to libusb and debug probes.
- The warning about no libusb is only logged once per process.
- pyOCD clears all sticky DP error flags when a protocol error occurs.
- If pyOCD sees a timeout error on transfers, it will issue a DP abort to stop the erroneous transaction.
- Flash programming raises exceptions when the flash algorithm returns errors.
- Added `smart_flash` and `keep_unwritten` user options to better control the flash programming logic.
- Builtin targets that used a custom `Flash` subclass to handle different erase/program sizes were updated to use the new support for that feature.
- Transfer errors that occur while reading ROM table entries are trapped, and the associated AP is ignored.
- The commander uses a selectable MEM-AP for all memory transfers.
- Configuring the TPIU for SWO is delayed until the `TPIU.set_swo_clock()` method is called. This is required due to a change in the v8-M TPIU that requires the trace clock to be active before accessing certain registers.

Fixes

- The NXP LPC-LinkII debug probe is recognized properly.
- Worked around apparent libusb issue in Windows where STLink probes would intermittently not be visible (575).
- Corrected handling of error responses from STLink probes (581). This also fixes problems with the latest STLink firmware version (V2J33M25).
- Fixed a problem introduced recently where probes running the old "mbed CMSIS-DAP" firmware was not recognized (588).
- Fixed CMSIS-Pack targets not working with `pyocd-gdbserver` (570).
- Fixed `--no-init` option for commander.
- Fixed the default value for the gdbserver's `vector_catch` option. If the option was not set, an exception was raised.
- Corrected `GDBServer.stop()` so it no longer uninits the board.
- Commander `disasm` command works while the target is running.
- The `--help-options` argument no longer fails for user options that have multiple allowed types, such as `pack`.
- Fixed `LOG_DAP` functionality in `pyocd.coresight.ap`.
- Updated pyyaml requirement to >= 5.1 (582). The entire 4.1 release series that pyOCD was pinned to turned out to be buggy (though it didn't seem to affect pyOCD's usage).
- Fixed `FileProgrammer` trying to raise the undefined `ArgumentError` exception for errors. It now raises the standard `ValueError`.
- Fixed a number of issues with how `FileProgrammer` checked for file errors, including file not found (590).
- Fixed default names for memory regions.
- Fixed a problem where `FileProgrammer` used programmatically would not default to sector erase.
- Access permissions are set on memory regions created for Pack targets.
- Fixed the `selected_core` property setter on `CoreSightTarget`.

Python API

- Added optional `init_board` parameter to `Session.open()` to control whether the board (and therefore target) are initialized. Used to implement the commander `--no-init` option.
- Removed `STLinkException` from the STLink probe code; now using standard pyOCD exceptions.
- Added page and phrase sizes and program and erase weight attributes to flash memory regions.
- Added `smart_flash` and `keep_unwritten` parameters to `FileProgrammer` and `FlashLoader`.
- `AccessPort` methods are automatically serialized via a mutex.
- Changing the HPROT and HNONSEC values used for MEM-AP transfers is supported.
- `MEM_AP` attempts to detect the supported HPROT and HNONSEC bits in CSW.
- `AHB_AP` detects whether the MSTRTYP bit is supported in CSW. (Only the M3/M4 AHB-AP supports this bit.)
- Added context manager methods to `MEM_AP` to temporarily change HPROT or HNONSEC. The AP is locked while the context manager is live.
- Moved the CoreSight components list to a separate source file.
- New `CortexM_v8M` subclass for v8-M cores.

Documentation

- Updated [contributing guidelines](https://github.com/mbedmicro/pyOCD/blob/master/CONTRIBUTING.md) and removed mention of the contributor license agreement that no longer needs to be signed.
- Added readme to [udev](https://github.com/mbedmicro/pyOCD/tree/master/udev) folder with detailed information.
- The udev section in the main readme was simplified.
- Link to GNU Arm Embedded toolchain in a couple places.
- Updated the architecture diagram.
- Note about `gdb_test.py` requiring 32-bit Python 2.7 on Windows.

Testing

- Testing both 8- and 32-bit transfers in `speed_test.py`.
- `speed_test.py` limits the size it will read or write during testing to 1 MB.
- Rewrote the simple flash erase/program test in `basic_test.py` to support nonequal erase/program sizes.

Known issues

- The LPC-LinkII probe may not work correctly under Windows using pywinusb.
- All new v8-M targets have certain issues with flash programming or other operations.

Boards and targets

- Added MAX32620 target and MAX32620FTHR board.
- Added MAX32630 target and MAX32630FTHR board.

Changes

- Sector erase is now the default mode for all flash programming since it is a safer default, especially with so many IoT projects now storing data on internal flash.
- Made the warning for an unknown board ID more helpful and less scary. It now says, ""Board ID XXXX is not recognized; you will be able to use pyOCD but not program flash."
- The SWV reader supports 16- and 32-bit ITM channel 0 events that send multiple characters.
- pyOCD only attempts to read the board ID from the probe's USB serial number if it knows the probe is running DAPLink, by comparing the USB VID and PID. This prevents unknown board ID warnings for other CMSIS-DAP compatible probes.
- pyOCD is now aware that a board ID of "0000" means the DAPLink is generic, so it won't try to look up the matching board.
- Removed websocket support.

Fixes

- Fixed a failure to start SWO transfers from the probe by sending the command to stop SWO before starting it. The failure could happen if the probe was left with SWO running when pyOCD exited, and then pyOCD was started again.
- Updated Argon RTOS support to match the latest version.
- Fixed an issue that prevented SWO from working with CMSIS-DAPv1.
- The SWV reader checks that the probe supports SWO before attempting to enable it.
- The `trace_start()` delegate and user script method will be called by the SWV reader if the target is reset.
- Fixed memory cache bugs that caused duplication of cache contents. This most often appeared as extra data being sent by the gdbserver in response to a gdb read memory command, though it was not visible to users.
- Fixed the `FlashReaderContext` class' usage of a removed intervaltree API. This problem would be triggered if you passed `pyocd gdbserver` an ELF file.
- Any enabled watchpoints are removed on disconnect.
- The list of valid APs is saved only after the full AP scan completes, to prevent a partial list being saved.
- Rewrote gdbserver memory read command handler to use a much simpler and more efficient hex encoding method.
- Catching `NotImplementedError` exceptions raised on Windows by the CMSIS-DAPv2 interface code.

Python API

- Memory map API now allows empty memory ranges, but not empty memory regions.
- Removed `has_fpu` attribute from the `Target` class. This attribute is only supported on `CortexM` now.
- `is_float_register()` returns true for both single and double precision float registers.
- Added a setter for the `CoreSightTarget.selected_core` property. The `select_core()` method is now deprecated.
- Changed the name of the `format` parameter of `FileProgrammer.program()` to `file_format` so it doesn't shadow the builtin function.
- Refactored `GDBSocket` into `pyocd.utility.socket.ListenerSocket`.
- Renamed `pyocd.utility.py3_helpers` module to `pyocd.utility.compatibility`.
- Removed unused `Flash_cortex_m` class.
- A few remaining timeouts were changed to use the `Timeout` class.
- Got rid of a pointless mutex in `GDBServer`.
- Removed reading of flash related user options from `GDBServer` since these are now handled elsewhere.

Documentation

- Renamed files in the `docs/` folder to all lower case.
- Started process of converting doxygen comments to doxygen-enabled doc comments.
- Added some missing user options to docs.
- Fixed some errors and typos in `user_scripts.md` and delegate method doc comments.
- Converted license headers to use single-line comments, mostly so they don't show up in Doxygen output.

Testing

- Set up tox for running functional tests under Python 2.7 and 3.7. This will soon be used in our CI system to test on both Python versions.
- Moved unit tests from the pyocd Python package to the `test/` folder in the repo root.
- Added tests to cover memory cache bugs mentioned above.
- `flash_loader_test.py` tests hex file programming.
- `speed_test.py` tests cached memory accesses and records the ROM read speed.
- `cortex_test.py` reports test metrics.
- Removed use of `test_boards.yaml` file in favour of the now-standard `pyocd.yaml` config file.
- Fixed `blank_test.py`.
- New `debug_context_test.py`.

New features

- pyOCD now supports user scripts, written in Python, that can extend, modify, and override default behaviour at multiple key points in the connection lifetime. This feature is documented in [docs/user_scripts.md](https://github.com/mbedmicro/pyOCD/blob/master/docs/user_scripts.md).
- Providing the foundation for user scripts, the Python API supports attaching a delegate object to the session and other objects such as the target and cores. The same hook functions are used for delegates and user scripts.
- SWO and SWV are now supported for both CMSIS-DAP and STLink. See the description of 543 for details.
- CMSIS-DAPv2 support. v2 uses bulk endpoints instead of HID for improved latency. Tested with the Keil ULINKplus.
- Sessions have the concept of a project directory. This directory defaults to the working directory where pyOCD was started. It can be changed with the `-j/--dir` command line argument or `project_dir` user option. Any files used by pyOCD, such as config files, user scripts, or .FLM flash algos, will be searched for in the project directory if they are specified as a relative path.

Boards and targets

- Added STM32F429xI.

Changes

- The objects in the pyOCD object graph are now linked to form a generic object graph, in addition to the explicit references that already exist. This is intended to make it easier to find objects as the graph gets much more complex with support for SWO and other CoreSight components.
- The path to an .FLM style flash algorithm can be set on a `FlashRegion` via the `flm` property. pyOCD will then load the flash algorithm for you. Very useful for user scripts.
- Renamed the `flash_algo` property of `FlashRegion` to just `algo` and made it and `flash_class` settable. This is all to make it easy to set or modify the flash algo in user scripts and delegates.
- Renamed `SESSION_OPTIONS.md` documentation file to `CONFIGURATION.md` since it covers all configuration topics and not just a list of options.
- Added ULINKplus to example udev rules.

Fixes

- Fixed a typo in the code to build a flash algo from an .FLM file, which caused STM32 CMSIS-Pack targets to not be able to program flash.
- Corrected a performance issue for Python 3 on Windows and Linux. The pywinusb and pyusb backends for CMSIS-DAP were not yielding to other threads while waiting for data. (Thanks Cypress!)
- Improved detection of post-reset state for RTX5.
- Reduced warning messages related to USB device permissions issues on Linux through improved filtering for CMSIS-DAP devices.
- A few strings were not being properly formatted with '%'. (Thanks mbolivar!)
- Added catch for unknown directory when parsing ELF symbols.

Fixes

- Addressed a case where the USB device product name could be returned as bytes by hidapi instead of a string.
- Fixed `to_str_safe()` on Python 2 with non-ASCII characters.
- Fixed Python 2 incompatibility in CMSIS DFP family matching.

Changes

- A little refactoring of some code in the `CmsisPack` class.
- Renamed `pyocd.target.pack.pack` to `pyocd.target.pack.cmsis_pack`.

New features

- Initial enablement of target support through CMSIS Device Family Packs. Pass the `--pack` argument to the `pyocd` tool, or set the `pack` session option in a config file. Any devices defined in the specified pack(s) become available as targets.
- "erase" gdbserver monitor command. It accepts the same arguments as the `pyocd erase` subcommand.

Boards and targets

- Cypress PSoC6 CY8C6xx7 and CY8C6xxA targets and related boards.
- Maxim MAX32625 target and related board.
- Added STM32L031x6 target. (Thanks michieldwitte!)
- Fixed the memory map for STM32F412xG. (Thanks anthrax-0!)

Changes

- New `Target.ResetType` enumeration.
- Much improved reset configuration. `CortexM` has `default_reset_type` and `default_software_reset_type` properties. Software reset is used by default, and the default software reset depends on whether the core is the primary or a secondary core.
- New emulated software reset type for v6-M and v8-M devices that don't support `VECTRESET`.
- Presenting a standard VFP view to GDB.
- Nonstandard erased flash value can be set on a `FlashRegion` object with the `erased_byte_value` attribute.
- Builtin targets were moved to `pyocd.target.builtin`.

Fixes

- Fixed an issue with STLink communication by forcing the probe out of DFU mode on connection. (Thanks michieldwitte!)
- Addressed a potential divide-by-zero error in `FlashLoader` if no data was written to flash.
- Invalid addresses in hex files will be ignored rather than raise an exception. Some vendors put metadata in hex files as invalid addresses.