Nbconvert

Major features

Tag Based Element Filtering

For removing individual elements from notebooks, we need a way to signal
to nbconvert that the elements should be removed. With this release, we
introduce the use of tags for that purpose.

Tags are user-defined strings attached to cells or outputs. They are
stored in cell or output metadata. For more on tags see the [nbformat
docs on cell
metadata](https://nbformat.readthedocs.io/en/latest/format_description.html#cell-metadata).

**Usage**:

1. Apply tags to the elements that you want to remove.

For removing an entire cell, the cell input, or all cell outputs apply
the tag to the cell.

For removing individual outputs, put the tag in the output metadata
using a call like
`display(your_output_element, metadata={tags=[<your_tags_here>]})`.

_NB_: Use different tags depending on whether you want to remove the
entire cell, the input, all outputs, or individual outputs.

2. Add the tags for removing the different kinds of elements to the
following traitlets. Which kind of element you want to remove
determines which traitlet you add the tags to.

The following traitlets remove elements of different kinds:

- `remove_cell_tags`: removes cells
- `remove_input_tags`: removes inputs
- `remove_all_outputs_tag`: removes all outputs
- `remove_single_output_tag`: removes individual outputs

Comprehensive notes

- new: configurable `browser` in ServePostProcessor
[618](https://github.com/jupyter/nbconvert/pull/618)
- new: `--clear-output` command line flag to clear output in-place
[619](https://github.com/jupyter/nbconvert/pull/619)
- new: remove elements based on tags with `TagRemovePreprocessor`.
[640](https://github.com/jupyter/nbconvert/pull/640), [#643](https://github.com/jupyter/nbconvert/pull/643)
- new: CellExecutionError can now be imported from
`nbconvert.preprocessors` [656](https://github.com/jupyter/nbconvert/pull/656)
- new: slides now can enable scrolling and custom transitions
[600](https://github.com/jupyter/nbconvert/pull/600)
- docs: Release instructions for nbviewer-deploy
- docs: improved instructions for handling errors using the
`ExecutePreprocessor` [656](https://github.com/jupyter/nbconvert/pull/656)
- tests: better height/width metadata testing for images in rst & html
[601](https://github.com/jupyter/nbconvert/pull/601) [#602](https://github.com/jupyter/nbconvert/pull/602)
- tests: normalise base64 output data to avoid false positives
[650](https://github.com/jupyter/nbconvert/pull/650)
- tests: normalise ipython traceback messages to handle old and new
style [631](https://github.com/jupyter/nbconvert/pull/631)
- bug: mathjax obeys `\\(\\)` & `\\[\\]` (both nbconvert & pandoc)
[609](https://github.com/jupyter/nbconvert/pull/609) [#617](https://github.com/jupyter/nbconvert/pull/617)
- bug: specify default templates using extensions
[639](https://github.com/jupyter/nbconvert/pull/639)
- bug: fix pandoc version number [638](https://github.com/jupyter/nbconvert/pull/638)
- bug: require recent mistune version [630](https://github.com/jupyter/nbconvert/pull/630)
- bug: catch errors from IPython `execute_reply` and `error` messages
[642](https://github.com/jupyter/nbconvert/pull/642)
- nose completely removed & dependency dropped [595](https://github.com/jupyter/nbconvert/pull/595) [#660](https://github.com/jupyter/nbconvert/pull/660)
- mathjax processing in mistune now only uses inline grammar
[611](https://github.com/jupyter/nbconvert/pull/611)
- removeRegex now enabled by default on all TemplateExporters, does
not remove cells with outputs [616](https://github.com/jupyter/nbconvert/pull/616)
- validate notebook after applying each preprocessor (allowing
additional attributes) [645](https://github.com/jupyter/nbconvert/pull/645)
- changed COPYING.md to LICENSE for more standard licensing that
GitHub knows how to read [654](https://github.com/jupyter/nbconvert/pull/654)

Major features

In this release (along with the usual bugfixes and documentation
improvements, which are legion) we have a few new major features that
have been requested for a long time:

Global Content Filtering

You now have the ability to remove input or output from code cells,
markdown cells and the input and output prompts. The easiest way to
access all of these is by using traitlets like
TemplateExporter.exclude_input = True (or, for example
HTMLExporter.exclude_markdown = True if you wanted to make it specific
to HTML output). On the command line if you just want to not have input
or output prompts just use --no-prompt.

Execute notebooks from a function

You can now use the executenb function to execute notebooks as though
you ran the execute preprocessor on the notebooks. It returns the
standard notebook and resources options.

Remove cells based on regex pattern

This removes cells based on their matching a regex pattern (by default,
empty cells). This is the RegexRemovePreprocessor.

Script exporter entrypoints for nonpython scripts

Now there is an entrypoint for having an exporter specific to the type
of script that is being exported. While designed for use with the
IRkernel in particular (with a script exporter focused on exporting R
scripts) other non-python kernels that wish to have a language specific
exporter can now surface that directly.

Comprehensive notes

- new: configurable ExecutePreprocessor.startup_timeout configurable
[583](https://github.com/jupyter/nbconvert/pull/583)
- new: RemoveCell preprocessor based on cell content (defaults to
empty cell) [575](https://github.com/jupyter/nbconvert/pull/575)
- new: function for executing notebooks: `executenb`
[573](https://github.com/jupyter/nbconvert/pull/573)
- new: global filtering to remove inputs, outputs, markdown cells
(&c.), this works on all templates [554](https://github.com/jupyter/nbconvert/pull/554)
- new: script exporter entrypoint [531](https://github.com/jupyter/nbconvert/pull/531)
- new: configurable anchor link text (previously ¶)
`HTMLExporter.anchor_link_text` [522](https://github.com/jupyter/nbconvert/pull/522)
- new: configurable values for slides exporter [542](https://github.com/jupyter/nbconvert/pull/542) [#558](https://github.com/jupyter/nbconvert/pull/558)
- improved releases (how-to documentation, version-number generation
and checking) [593](https://github.com/jupyter/nbconvert/pull/593)
- doc improvements [593](https://github.com/jupyter/nbconvert/pull/593)
[580](https://github.com/jupyter/nbconvert/pull/580) [#565](https://github.com/jupyter/nbconvert/pull/565) [#554](https://github.com/jupyter/nbconvert/pull/554)
- language information from cell magics (for highlighting) is now
included in more formats [586](https://github.com/jupyter/nbconvert/pull/586)
- mathjax upgrades and cdn fixes [584](https://github.com/jupyter/nbconvert/pull/584) [#567](https://github.com/jupyter/nbconvert/pull/567)
- better CI [571](https://github.com/jupyter/nbconvert/pull/571)
[540](https://github.com/jupyter/nbconvert/pull/540)
- better traceback behaviour when execution errs
[521](https://github.com/jupyter/nbconvert/pull/521)
- deprecated nose test features removed [519](https://github.com/jupyter/nbconvert/pull/519)
- bug fixed: we now respect width and height metadata on jpeg and png
mimetype outputs [588](https://github.com/jupyter/nbconvert/pull/588)
- bug fixed: now we respect the `resolve_references` filter in
`report.tplx` [577](https://github.com/jupyter/nbconvert/pull/577)
- bug fixed: output metadata now is removed by ClearOutputPreprocessor
[569](https://github.com/jupyter/nbconvert/pull/569)
- bug fixed: display id respected in execute preproessor
[563](https://github.com/jupyter/nbconvert/pull/563)
- bug fixed: dynamic defaults for optional jupyter_client import
[559](https://github.com/jupyter/nbconvert/pull/559)
- bug fixed: don't self-close non-void HTML tags
[548](https://github.com/jupyter/nbconvert/pull/548)
- buf fixed: upgrade jupyter_client dependency to 4.2
[539](https://github.com/jupyter/nbconvert/pull/539)
- bug fixed: LaTeX output through md→LaTeX conversion shouldn't be
touched [535](https://github.com/jupyter/nbconvert/pull/535)
- bug fixed: now we escape `<` inside math formulas when converting to
html [514](https://github.com/jupyter/nbconvert/pull/514)

Credits

This release has been larger than previous releases. In it 33 authors
contributed a total of 546 commits.

Many thanks to the following individuals who contributed to this release
(in alphabetical order):

- Adam Chainz
- Andreas Mueller
- Bartosz T
- Benjamin Ragan-Kelley
- Carol Willing
- Damián Avila
- Elliot Marsden
- Gao, Xiang
- Jaeho Shin
- Jan Schulz
- Jeremy Kun
- Jessica B. Hamrick
- John B Nelson
- juhasch
- Livia Barazzetti
- M Pacer
- Matej Urbas
- Matthias Bussonnier
- Matthias Geier
- Maximilian Albert
- Michael Scott Cuthbert
- Nicholas Bollweg
- Paul Gowder
- Paulo Villegas
- Peter Parente
- Philipp A
- Scott Sanderson
- Srinivas Reddy Thatiparthy
- Sylvain Corlay
- Thomas Kluyver
- Till Hoffmann
- Xiang Gao
- YuviPanda

- fix version numbering because of incomplete previous version number

- improved CSS (specifically tables, in line with notebook)
[498](https://github.com/jupyter/nbconvert/pull/498)
- improve in-memory templates handling [491](https://github.com/jupyter/nbconvert/pull/491)
- test improvements [516](https://github.com/jupyter/nbconvert/pull/516)
[509](https://github.com/jupyter/nbconvert/pull/509) [#505](https://github.com/jupyter/nbconvert/pull/505)
- new configuration option: IOPub timeout [513](https://github.com/jupyter/nbconvert/pull/513)
- doc improvements [489](https://github.com/jupyter/nbconvert/pull/489)
[500](https://github.com/jupyter/nbconvert/pull/500) [#493](https://github.com/jupyter/nbconvert/pull/493) [#506](https://github.com/jupyter/nbconvert/pull/506)
- newly customizable: output prompt [500](https://github.com/jupyter/nbconvert/pull/500)
- more python2/3 compatible unicode handling [502](https://github.com/jupyter/nbconvert/pull/502)

- Use `xelatex` by default for latex
export, improving unicode and font support.
- Use entrypoints internally to access Exporters, allowing for
packages to declare custom exporters more easily.
- New ASCIIDoc Exporter.
- New preprocessor for sanitised html output.
- New general `convert_pandoc` filter to reduce the need to hard-code
lists of filters in templates.
- Use pytest, nose dependency to be removed.
- Refactored Exporter code to avoid ambiguity and cyclic dependencies.
- Update to traitlets 4.2 API.
- Fixes for Unicode errors when showing execution errors on Python 2.
- Default math font matches default Palatino body text font.
- General documentation improvements. For example, testing,
installation, custom exporters.
- Improved link handling for LaTeX output
- Refactored the automatic id generation.
- New kernel_manager_class configuration option for allowing systems
to be set up to resolve kernels in different ways.
- Kernel errors now will be logged for debugging purposes when
executing notebooks.