Relstorage

==================

- PostgreSQL: Fixed another minor compatibility issue with PostgreSQL 9.0.
Packing raised an error when the client used old an version of libpq.

- Delete empty transactions in batches of 1000 rows instead of all in one
go, to prevent holding the transaction lock for longer than absolutely
necessary.

- Oracle: Fix object reference downloading performance for large RelStorage
databases during the garbage collection phase of a pack.

- Oracle, PostgreSQL: Switch to storing ZODB blob in chunks up to 4GB
(the maximum supported by cx_Oracle) or 2GB (PostgreSQL maximum blob size)
to maximize blob reading and writing performance.

The PostgreSQL blob_chunk schema changed to support this, see
notes/migrate-to-1.5.txt to update existing databases.

- zodbconvert: When copying a database containing blobs, ensure the source
blob file exists long enough to copy it.

====================

- Better packing based on experience with large databases. Thanks
to Martijn Pieters!

- Added more feedback to the packing process. It'll now report
during batch commit how much of the total work has been
completed, but at most every .1% of the total number of
transactions or objects to process.

- Renamed the --dry-run option to --prepack and added a
--use-prepack-state to zodbpack. With these 2 options the
pre-pack and pack phases can be run separately, allowing re-use
of the pre-pack analysis data or even delegating the pre-pack
phase off to a separate server.

- Replaced the packing duty cycle with a nowait locking strategy.
The pack operation will now request the commit lock but pauses if
it is already taken. It releases the lock after every batch
(defaulting to 1 second processing). This makes the packing
process faster while at the same time yielding to regular ZODB
commits when busy.

- Do not hold the commit lock during pack cleanup while deleting
rows from the object reference tables; these tables are
pack-specific and regular ZODB commits never touch these.

- Added an option to control schema creation / updating on startup.
Setting the ``create-schema`` option to false will let you use
RelStorage without a schema update.

- Fixed compatibility with PostgreSQL 9.0, which is capable of
returning a new 'hex' type to the client. Some builds of psycopg2
return garbage or raise an error when they see the new type. The fix
was to encode more SQL query responses using base 64.

- With the new shared-blob-dir option set to false, it was possible
for a thread to read a partially downloaded blob. Fixed. Thanks for
the report from Maurits van Rees.

- Support for "shared-blob-dir false" now requires ZODB 3.9 or better.
The code in the ZODB 3.8 version of ZODB.blob is not compatible with
BlobCacheLayout, leading to blob filename collisions.

====================

- Added a state_size column to object_state, making it possible
to query the size of objects without loading the state. The new
column is intended for gathering statistics. A schema migration
is required.

- Added more logging during zodbconvert to show that something is
happening and give an indication of how far along the process is.

- Fixed a missing import in the blob cache cleanup code.

- Added a --dry-run option to zodbpack.

- Replaced the graph traversal portion of the pack code with
a more efficient implementation using Python sets (instead of SQL).
The new code is much faster for packing databases with deeply
nested objects.

====================

- Added an option to store ZODB blobs in the database. The option is
called "shared-blob-dir" and it behaves very much like the ZEO
option of the same name. Blobs stored in the database are broken
into chunks to reduce the impact on RAM.

- Require setuptools or distribute. Plain distutils is not sufficient.

==================

- Fixed compatibility with ZODB 3.10. As reported by JĂźrgen Herrmann,
there was a problem with conflict errors. The RelStorage patch of the
sync() method now works with ZODB 3.10.

- Fixed a bug in packing history-free databases. If changes were
made to the database during the pack, the pack code could delete
too many objects. Thanks to Chris Withers for writing test code
that revealed the bug. A schema migration is required for history-free
databases; see notes/migration-to-1.4.txt.

- Enabled logging to stderr in zodbpack.

==================

- Oracle: always connect in threaded mode. Without threaded mode,
clients of Oracle 11g sometimes segfault.