Music21

The beta/release-candidate for v3 of music21 -- a toolkit for computer-aided musicology, has been released! We are very close to a v3 final, with many great improvements.

For details, see:
https://music21-mit.blogspot.com/2016/08/music21-v-3-beta-306-released.html

The first release of the `music21` v.3 system. Major changes include the creation of "StreamIterator" classes that greatly reduce the number of Streams created. Full release notes are available on the music21 list. Not backwards compatible.


v.2.2.1
This maintenance release of music21 fixes several errors in MusicXML input and output and marks some functions or properties (such as Music21Object.isGrace) as deprecated for 3.0. It mainly fixes under-the-hood errors. It also adds compatibility with matplotlib 1.5.

Major changes are going in the music21 3.0 track.


v.2.1.2
Fixes some small errors throughout that were uncaught in v.2.1.0 release due to one of the errors breaking the test system, and one larger error in stream.replace() which is main reason for the quick update.

Adds some features that were originally slated for 2.3, but which were ready and thus are easier to add now than to back out and put out later, including:

1) improvements to multi-measure rest handling on import/export musicxml and representing in the note.Rest() object.
2) rest positions off center line are now imported and exported to musicxml (note: musicxml records rest position as a pitch irrespective of clef. Music21 records offset from center line)
3) open File objects and file-like objects (BytesIO and StringIO) can be passed in place of file names in many cases.
4) stream.chordify(addPartIdAsGroup=True, consolidatePitches=True) now works properly if a single pitch appears in multiple parts.
5) improvements to layout.py
6) like lists, tuples, etc. -- bool(Stream) now returns True if there's at least one object in it, and False if it is empty. Similarly for StreamIterator objects
7) speed improvements throughout many stream functions by replacing`if len(stream.getX()) > 0:` with `if stream.iter.getX()` which will return True as soon as the first item is found.
8) alpha version of having music21 automatically load certain %magic functions under IPython/Jupyter Notebook. Will announce this further when I'm sure it works.
9) Added documentation warnings about using `.elements` -- there's almost no need to do so; will work on potentially deprecating it soon.

The long-awaited (at least by me) version 2 of `music21` is released! This is the first version of the v.2 release to be out of beta and stable enough for general use by everyone.

The first non-beta release of `music21` since v. 1.9.3 (June 2014) gives a ton of new features and lots of new speed. But being a major release change number, it also has some changes that every programmer using the system needs to be aware of. The release notes on GitHub gives all the details, but here are the highlights since 1.9:

Changed and Added features
- Duration and Offset now use Fractions when necessary for exact representation of tuplets. Many, many errors from rounding are gone. For now, you can use Duration.quarterLengthFloat and offsetFloat to get the old behavior, but float(Duration.quarterLength) and float(offset) are better.
- Converters support easy to install custom sub converters. MEI is now supported (thanks to McGill university)
- Python 2.6 is not supported. Python 3.4 is **highly recommended**; 2.7, 3.3, and 3.5 also work.
- Loading cached streams is extremely fast. All streams are automatically cached when loaded from disk.
- Sorting is much more consistent and faster
- MusicXML parsing and showing have been rewritten to use cElementTree and many new features.
- Stream's internal mechanisms have been hugely rearranged. Now offsets are stored inside Streams instead of inside Notes, etc., making lots of things faster and more reliable.
- Streams support filters on iteration using the `.iter` property and the `recurse()` method. These are big changes for speed and reliability.
- Namedtuples replace anonymous tuples in many places
- Music21 is available under the BSD license.
- Musedata files are no longer available in the corpus. However, new files in MusicXML format have replaced several of them.
- Complete rewrite of TinyNotation making it much easier to subclass for your needs.
- If you have MuseScore 2, try sc.show('musicxml.png') to get a beautifully rendered musicxml file. Or use .pdf to get something ready to print. Thanks Nicholas, Thomas, and Walter!
- Builds are automatically tested for errors and documentation coverage.
- Experimental modules moved to the `alpha` sub package. `demos` reorganized.
- Lots of documentation changes!
- Obscure and almost never used (or actually never used) methods and attributes have been removed.
- Did I mention how much better the documentation is getting?

In case anyone is keeping track, since v.1.0 (June 2012), here are the:

Ten days since the last release, so time for a new one. Again, speed, stability, and new features. The biggest change is the entirely new MusicXML output system to match the entirely new input system introduced last release.

The second biggest is in the (re)introduction of StreamIterators and RecursiveIterator. I'll need to get some demos up of this soon, but this will be a game changer for some tasks.

**Bigger changes**
1. **MusicXML now uses the faster, more reliable ElementTree output generator.** Please report any bugs on import or export, especially if they are regressions from `format='oldmusicxml'`. `oldmusicxml` will disappear soon.
2. Better docs (see below), especially for the long under documented _recurse_ function. Everything that was in _overview_ is now in the _User's Guide_.
3. **Streams now support filters on iteration** -- if you have been using: `for e in s.getElementsByClass('X')`, try: `for e in s.iter.getElementsByClass('X')` for a major speedup, especially if you just want the first one or something of that sort. Recurse() supports the same, so `for e in s.recurse().notes.getElementsByGroup('tuba')` will be WAY faster than before. You might not notice the difference on your own work, but internally things are getting a lot faster. (obscure non-filter routines will be deprecated and disappear soon).
4. Corpus docs/indexes, etc. are updated with more recent corpus changes (nothing new, but easier to find).
5. Use of deprecated functions now generates a warning. This should help people plan for migration in case you're not reading the documentation religiously.

**Smaller changes**
1. Documentation is improved and updated working with Jupyter/IPython 4 (note: a bug in nbconvert + pandoc requires pandoc v. 1.33 or older to make; they're working on a patch). Docs build in parallel, so it's very fast -- you'll see updates more often.
2. Documentation is now separated into "source/" and "autogenerated/" folders -- everything in source is user editable. Nothing in autogenerated is.
3. A number of obscure, long deprecated functions are gone, the biggest being `n.removeLocationBySite()` use `n.sites.remove()`
4. normalization in features has been fixed (Thanks Frank Zalkow)
5. Parsing of cappella MusicXML files has been improved.
6. Improved parsing of RomanText files; bugs in several encodings of rntxt and abc files have been fixed.
7. `common.nearestCommonFraction` has been renamed `addFloatPrecision` to better reflect what it does. This has always confused me.

It's been a busy week in music21's world.

This post announces the v.2.0.10 beta release of music21, which is moving quickly to the official v.2 release, v.2.1. Some of the changes have already been announced on the music21list Google Groups mailing list. The major changes include:
- New parsing engine for MusicXML (see below)
- DurationTuples replace DurationUnits
- Percussion clefs and No Clefs now are supported properly in musicxml output
- Improvements to the RomanText and clercqTemperly formats (thanks DT!)
- Some obscure modules removed from the main namespace:
*\* intervalNetwork becomes scale.intervalNetwork and BoundIntervalNetwork becomes simply IntervalNetwork.
*\* scala becomes scale.scala
*\* chord becomes a package and chordTables becomes chord.tables
*\* In the next version, expect languageExcerpts to become text.languageDetection and the "xmlnode" module to disappear.
*\* Environment and CapellaXML, which depended on XMLNode now don't. CapellaXML processing is 10x faster.
*\* jsonpickling is upgraded and safer.
*\* Building documentation now works on IPython 4/Jupyter 4.0
*\* MusicXML output with Unicode now works on Py3 (thanks Sarig!)
*\* Spanners on Rests now export properly in MusicXML
*\* VexFlow only supports the music21j based output now. More bug fixes there to come (or will be moved to alpha support). The older version wasn't working with recent VexFlow.
*\* Everything overall is about 30% faster than a month ago.

The biggest change in this version is how MusicXML is processed. When Christopher Ariza joined the music21 team in 2008, music21 had a tiny limitation: it didn't work with MusicXML, at all. Whoops! It was just too big a task to tackle for me when I was still figuring out how Streams, Sites, Durations, etc. would work. Thankfully Chris took it on and extremely quickly produced a great parser for MusicXML. The problem back then was that few people were on the latest, greatest version of Python 2.5, and music21 aimed to support at least back to Python 2.1, and only the newest Python 2.5 had the brand new "ElementTree" Python processing module (and there were still substantial bugs in that module before Python 2.6). We were determined not to make MusicXML parsing require an external library such as "lxml", so that left two choices, xml.minidom and xml.sax.

Anyone who knows anything about the structure of MusicXML and the differences in philosophy between DOM and SAX will know that DOM is the logical choice for MusicXML parsing -- it allows nodes to look at their neighbors, parents, children, and make logical decisions (am I a note, rest, or chord?) based on the context. SAX on the other hand is built on calling functions whenever a particular tag start is encountered, whenever data is encountered, and whenever a stop tag is encountered. Great for certain types of text formatting, insanely difficult for a format like MusicXML (or MEI or just about any music format besides perhaps MIDI). So, if memory serves, Chris wrote a quick DOM processor for MusicXML and it was getting notes, durations, measures, beautifully.

But Chris Ariza is also probably the best programmer I've ever met and before going further he profiled the system and extrapolated what it would be like to work with a large corpus of MusicXML files using it. Slow as slime. The minidom was implemented entirely in Python, not highly optimized, and was not going to make anyone want to use MusicXML in the toolkit.

So, he basically did the impossible: implemented a blazingly fast SAX processor for MusicXML that built a close-to-the-original representation of the file (musicxml.mxObjects) and then processed that in a much more friendly format. Bam! Speed went up by an order of magnitude, and everything that music21 could do with MusicXML was born. In the dozens of releases since he moved on from the project, I've barely had to touch the internals at all even as the rest of the system has expanded and changed dramatically. And there was a system for caching the mxObjects representation for a speedup in the next parse.

Fast forward 7 years. Python has changed. Version 2.7 is now the minimum requirement (it's over five years old already; we just found a check for Python > 2.2 somewhere in the system! removed it) V.3.3 and 3.4 are supported (3.5 should be out this week and of course will be supported). And everyone has access to xml.etree.ElementTree now. And the final representation of all parsed formats is now cached, so there is no need for the mxObjects cache. So in the interest of simplifying parsing (and getting a 40% speedup over SAX + mxObjects), it made sense to rewrite the MusicXML parsing engine.

The new version is called musicxml.xmlToM21. There are a few miscellaneous files in a new musicxml.xmlObjects file, but basically all the parsing takes place in the xmlToM21 file. Every tag in musicxml is now written directly into the file to make it easier to see exactly which tag is causing any particular problem. (Line number properties may be possible to add soon). Because the format of the parser is now much closer to the format of the MusicXML document, a TODO: has been added for every missing tag, or attribute. Expect music21 to support every tag and attribute in MusicXML 3.0 sometime soon. If you've ever wanted to hack additional support into Music21's MusicXML parsing but it seemed too daunting, give another look at the code now.

This is a major change on the most used format for music21. Thankfully, Ariza wrote so many tests into the system that I am relatively confident that everything now works exactly like before. The exceptions are: non-printed notes are no longer skipped (this was to prevent the next bug), notes with incorrect divisions are now corrected rather than skipped, and spanners preceding rests are now attached to the rest rather than the next adjacent note. (My intention was to be 100% compatible with before, but it would've been very hard to replicate this incorrect behavior). The one negative side-effect you will see is that parsing some of the Beethoven files is now slower (rather than 40% faster) because some of those files used a large number of incorrectly notated, non-printing notes to represent playback of trills. For certain files (such as the Große Fuge) the number of notes in the score will almost double with the new system.

Because this change is major, for now you can still use the old parsing system via converter.parse('filename.xml', format='oldmusicxml'). I suggest also adding "forceSource=True" to make sure that you are reading the file from disk and not from Cache.

I'm extremely excited by this change -- we will get the writing of music21 files to use the new system by the next release (a much easier task).

As always, music21 has been supported by the Seaver Institute, the NEH Digging into Data grant, and MIT Music and Theater Arts/SHASS.

The newest release of the music21 v.2 (beta) developments improves the stability and performance of the system.

The biggest change in this version is the movement of all experimental modules into a new "alpha" sub-package; in the future, all releases that add something useful to music21 but which are not well tested or mostly documented will begin in this folder. They may graduate into the main music21 mainspace at some later point, remain in "alpha", or be removed. Among the modules that are moved include: `webapps` (system for running music21 as a WSGI service-oriented architecture), `trecento` (fourteenth-century musical analysis), `theoryAnalysis` (common-practice error detection), `counterpoint/species` (first-species counterpoint generator), `medren` (miscellaneous pre-1600 applications; this will return soon after refactoring), `contour` (contour analysis), `chant` (Gregorian chant generation), and `analysis.search` (find scales inside Streams).

The "demos" directory has also been reorganized. The next release will focus on making this directory easier to use.

Bug fixes and improvements:
- `search.lyrics` -- modules for searching within lyrics while retaining position information about matches.
- Interval objects have been improved to have additional properties.
- .priority changes will automatically re-sort Streams. This change will make `.priority` more useful.
- `Stream.elementsChanged()` is a new method that can trigger a cache clear for Streams.
- `Stream.remove()` gets a `recurse` function.
- Lilypond color works again (thanks Ringw)
- `Accidental` becomes a SlottedObject -- pro: much faster. con: arbitrary attributes cannot be added to Accidentals.
- MuseScore 2 is now discovered automatically in `python3 configure.py`.
- Tremolo support (including in MusicXML)
- Unlikely bugs in Chord fixed.
- MIDI support for > 16 channels output.
- Fixes for PIL/Pillow support of more versions
- More places taking advantage of exact fractions in music21 offsets and lengths which used to be kludges