Sqlalchemy

:released: March 30, 2021

.. change::
:tags: bug, schema
:tickets: 6152

Fixed bug first introduced in as some combination of :ticket:`2892`,
:ticket:`2919` nnd :ticket:`3832` where the attachment events for a
:class:`_types.TypeDecorator` would be doubled up against the "impl" class,
if the "impl" were also a :class:`_types.SchemaType`. The real-world case
is any :class:`_types.TypeDecorator` against :class:`_types.Enum` or
:class:`_types.Boolean` would get a doubled
:class:`_schema.CheckConstraint` when the ``create_constraint=True`` flag
is set.


.. change::
:tags: bug, schema, sqlite
:tickets: 6007
:versions: 1.4.0

Fixed issue where the CHECK constraint generated by :class:`_types.Boolean`
or :class:`_types.Enum` would fail to render the naming convention
correctly after the first compilation, due to an unintended change of state
within the name given to the constraint. This issue was first introduced in
0.9 in the fix for issue 3067, and the fix revises the approach taken at
that time which appears to have been more involved than what was needed.

.. change::
:tags: bug, orm
:tickets: 5983
:versions: 1.4.0

Removed very old warning that states that passive_deletes is not intended
for many-to-one relationships. While it is likely that in many cases
placing this parameter on a many-to-one relationship is not what was
intended, there are use cases where delete cascade may want to be
disallowed following from such a relationship.



.. change::
:tags: bug, postgresql
:tickets: 5989
:versions: 1.4.0

Fixed issue where using :class:`_postgresql.aggregate_order_by` would
return ARRAY(NullType) under certain conditions, interfering with
the ability of the result object to return data correctly.

.. change::
:tags: bug, schema
:tickets: 5919
:versions: 1.4.0

Repaired / implemented support for primary key constraint naming
conventions that use column names/keys/etc as part of the convention. In
particular, this includes that the :class:`.PrimaryKeyConstraint` object
that's automatically associated with a :class:`.schema.Table` will update
its name as new primary key :class:`_schema.Column` objects are added to
the table and then to the constraint. Internal failure modes related to
this constraint construction process including no columns present, no name
present or blank name present are now accommodated.

.. change::
:tags: bug, schema
:tickets: 6071
:versions: 1.4.3

Adjusted the logic that emits DROP statements for :class:`_schema.Sequence`
objects among the dropping of multiple tables, such that all
:class:`_schema.Sequence` objects are dropped after all tables, even if the
given :class:`_schema.Sequence` is related only to a :class:`_schema.Table`
object and not directly to the overall :class:`_schema.MetaData` object.
The use case supports the same :class:`_schema.Sequence` being associated
with more than one :class:`_schema.Table` at a time.

.. change::
:tags: bug, orm
:tickets: 5952
:versions: 1.4.0

Fixed issue where the process of joining two tables could fail if one of
the tables had an unrelated, unresolvable foreign key constraint which
would raise :class:`_exc.NoReferenceError` within the join process, which
nonetheless could be bypassed to allow the join to complete. The logic
which tested the exception for significance within the process would make
assumptions about the construct which would fail.


.. change::
:tags: bug, postgresql, reflection
:tickets: 6161
:versions: 1.4.4

Fixed issue in PostgreSQL reflection where a column expressing "NOT NULL"
will supersede the nullability of a corresponding domain.

.. change::
:tags: bug, engine
:tickets: 5929
:versions: 1.4.0

Fixed bug where the "schema_translate_map" feature failed to be taken into
account for the use case of direct execution of
:class:`_schema.DefaultGenerator` objects such as sequences, which included
the case where they were "pre-executed" in order to generate primary key
values when implicit_returning was disabled.

.. change::
:tags: bug, orm
:tickets: 6001
:versions: 1.4.0

Fixed issue where the :class:`_mutable.MutableComposite` construct could be
placed into an invalid state when the parent object was already loaded, and
then covered by a subsequent query, due to the composite properties'
refresh handler replacing the object with a new one not handled by the
mutable extension.


.. change::
:tags: bug, types, postgresql
:tickets: 6023
:versions: 1.4.3

Adjusted the psycopg2 dialect to emit an explicit PostgreSQL-style cast for
bound parameters that contain ARRAY elements. This allows the full range of
datatypes to function correctly within arrays. The asyncpg dialect already
generated these internal casts in the final statement. This also includes
support for array slice updates as well as the PostgreSQL-specific
:meth:`_postgresql.ARRAY.contains` method.

.. change::
:tags: bug, mssql, reflection
:tickets: 5921

Fixed issue regarding SQL Server reflection for older SQL Server 2005
version, a call to sp_columns would not proceed correctly without being
prefixed with the EXEC keyword. This method is not used in current 1.4
series.


.. changelog::

:released: February 1, 2021

.. change::
:tags: bug, ext
:tickets: 5836

Fixed issue where the stringification that is sometimes called when
attempting to generate the "key" for the ``.c`` collection on a selectable
would fail if the column were an unlabeled custom SQL construct using the
``sqlalchemy.ext.compiler`` extension, and did not provide a default
compilation form; while this seems like an unusual case, it can get invoked
for some ORM scenarios such as when the expression is used in an "order by"
in combination with joined eager loading. The issue is that the lack of a
default compiler function was raising :class:`.CompileError` and not
:class:`.UnsupportedCompilationError`.

.. change::
:tags: bug, postgresql
:tickets: 5645

For SQLAlchemy 1.3 only, setup.py pins pg8000 to a version lower than
1.16.6. Version 1.16.6 and above is supported by SQLAlchemy 1.4. Pull
request courtesy Giuseppe Lumia.

.. change::
:tags: bug, postgresql
:tickets: 5850

Fixed issue where using :meth:`_schema.Table.to_metadata` (called
:meth:`_schema.Table.tometadata` in 1.3) in conjunction with a PostgreSQL
:class:`_postgresql.ExcludeConstraint` that made use of ad-hoc column
expressions would fail to copy correctly.

.. change::
:tags: bug, sql
:tickets: 5816

Fixed bug where making use of the :meth:`.TypeEngine.with_variant` method
on a :class:`.TypeDecorator` type would fail to take into account the
dialect-specific mappings in use, due to a rule in :class:`.TypeDecorator`
that was instead attempting to check for chains of :class:`.TypeDecorator`
instances.


.. change::
:tags: bug, mysql, reflection
:tickets: 5860

Fixed bug where MySQL server default reflection would fail for numeric
values with a negation symbol present.


.. change::
:tags: bug, oracle
:tickets: 5813

Fixed regression in Oracle dialect introduced by :ticket:`4894` in
SQLAlchemy 1.3.11 where use of a SQL expression in RETURNING for an UPDATE
would fail to compile, due to a check for "server_default" when an
arbitrary SQL expression is not a column.


.. change::
:tags: usecase, mysql
:tickets: 5808

Casting to ``FLOAT`` is now supported in MySQL >= (8, 0, 17) and
MariaDb >= (10, 4, 5).

.. change::
:tags: bug, mysql
:tickets: 5898

Fixed long-lived bug in MySQL dialect where the maximum identifier length
of 255 was too long for names of all types of constraints, not just
indexes, all of which have a size limit of 64. As metadata naming
conventions can create too-long names in this area, apply the limit to the
identifier generator within the DDL compiler.

.. change::
:tags: bug, oracle
:tickets: 5812

Fixed bug in Oracle dialect where retrieving a CLOB/BLOB column via
:meth:`_dml.Insert.returning` would fail as the LOB value would need to be
read when returned; additionally, repaired support for retrieval of Unicode
values via RETURNING under Python 2.

.. change::
:tags: bug, mysql
:tickets: 5821

Fixed deprecation warnings that arose as a result of the release of PyMySQL
1.0, including deprecation warnings for the "db" and "passwd" parameters
now replaced with "database" and "password".


.. change::
:tags: bug, mysql
:tickets: 5800

Fixed regression from SQLAlchemy 1.3.20 caused by the fix for
:ticket:`5462` which adds double-parenthesis for MySQL functional
expressions in indexes, as is required by the backend, this inadvertently
extended to include arbitrary :func:`_sql.text` expressions as well as
Alembic's internal textual component, which are required by Alembic for
arbitrary index expressions which don't imply double parenthesis. The
check has been narrowed to include only binary/ unary/functional
expressions directly.

.. changelog::

:released: December 18, 2020

.. change::
:tags: bug, oracle
:tickets: 5784
:versions: 1.4.0b2

Fixed regression which occurred due to :ticket:`5755` which implemented
isolation level support for Oracle. It has been reported that many Oracle
accounts don't actually have permission to query the ``v$transaction``
view so this feature has been altered to gracefully fallback when it fails
upon database connect, where the dialect will assume "READ COMMITTED" is
the default isolation level as was the case prior to SQLAlchemy 1.3.21.
However, explicit use of the :meth:`_engine.Connection.get_isolation_level`
method must now necessarily raise an exception, as Oracle databases with
this restriction explicitly disallow the user from reading the current
isolation level.

.. changelog::

:released: December 17, 2020

.. change::
:tags: bug, orm
:tickets: 5774
:versions: 1.4.0b2

Added a comprehensive check and an informative error message for the case
where a mapped class, or a string mapped class name, is passed to
:paramref:`_orm.relationship.secondary`. This is an extremely common error
which warrants a clear message.

Additionally, added a new rule to the class registry resolution such that
with regards to the :paramref:`_orm.relationship.secondary` parameter, if a
mapped class and its table are of the identical string name, the
:class:`.Table` will be favored when resolving this parameter. In all
other cases, the class continues to be favored if a class and table
share the identical name.

.. change::
:tags: sqlite, usecase
:tickets: 5685

Added ``sqlite_with_rowid=False`` dialect keyword to enable creating
tables as ``CREATE TABLE … WITHOUT ROWID``. Patch courtesy Sean Anderson.

.. change::
:tags: bug, sql
:tickets: 5691

A warning is emitted if a returning() method such as
:meth:`_sql.Insert.returning` is called multiple times, as this does not
yet support additive operation. Version 1.4 will support additive
operation for this. Additionally, any combination of the
:meth:`_sql.Insert.returning` and :meth:`_sql.ValuesBase.return_defaults`
methods now raises an error as these methods are mutually exclusive;
previously the operation would fail silently.


.. change::
:tags: bug, mssql
:tickets: 5751

Fixed bug where a CREATE INDEX statement was rendered incorrectly when
both ``mssql-include`` and ``mssql_where`` were specified. Pull request
courtesy Adiorz.

.. change::
:tags: bug, postgresql, mysql
:tickets: 5729
:versions: 1.4.0b2

Fixed regression introduced in 1.3.2 for the PostgreSQL dialect, also
copied out to the MySQL dialect's feature in 1.3.18, where usage of a non
:class:`_schema.Table` construct such as :func:`_sql.text` as the argument
to :paramref:`_sql.Select.with_for_update.of` would fail to be accommodated
correctly within the PostgreSQL or MySQL compilers.


.. change::
:tags: bug, mssql
:tickets: 5646

Added SQL Server code "01000" to the list of disconnect codes.


.. change::
:tags: usecase, postgresql
:tickets: 5604
:versions: 1.4.0b2

Added new parameter :paramref:`_postgresql.ExcludeConstraint.ops` to the
:class:`_postgresql.ExcludeConstraint` object, to support operator class
specification with this constraint. Pull request courtesy Alon Menczer.

.. change::
:tags: bug, mysql, reflection
:tickets: 5744
:versions: 1.4.0b2

Fixed issue where reflecting a server default on MariaDB only that
contained a decimal point in the value would fail to be reflected
correctly, leading towards a reflected table that lacked any server
default.


.. change::
:tags: bug, orm
:tickets: 5664

Fixed bug in :meth:`_query.Query.update` where objects in the
:class:`_ormsession.Session` that were already expired would be
unnecessarily SELECTed individually when they were refreshed by the
"evaluate"synchronize strategy.

.. change::
:tags: usecase, oracle
:tickets: 5755

Implemented support for the SERIALIZABLE isolation level for Oracle
databases, as well as a real implementation for
:meth:`_engine.Connection.get_isolation_level`.

.. seealso::

:ref:`oracle_isolation_level`

.. change::
:tags: mysql, sql
:tickets: 5696

Added missing keywords to the ``RESERVED_WORDS`` list for the MySQL
dialect: ``action``, ``level``, ``mode``, ``status``, ``text``, ``time``.
Pull request courtesy Oscar Batori.

.. change::
:tags: bug, orm
:tickets: 5737
:versions: 1.4.0b2

Fixed bug involving the ``restore_load_context`` option of ORM events such
as :meth:`_ormevent.InstanceEvents.load` such that the flag would not be
carried along to subclasses which were mapped after the event handler were
first established.



.. change::
:tags: bug, sql
:tickets: 5656

Fixed structural compiler issue where some constructs such as MySQL /
PostgreSQL "on conflict / on duplicate key" would rely upon the state of
the :class:`_sql.Compiler` object being fixed against their statement as
the top level statement, which would fail in cases where those statements
are branched from a different context, such as a DDL construct linked to a
SQL statement.


.. change::
:tags: mssql, sqlite, reflection
:tickets: 5661

Fixed issue with composite primary key columns not being reported
in the correct order. Patch courtesy fulpm.

.. changelog::

:released: October 12, 2020

.. change::
:tags: bug, orm
:tickets: 4428

An :class:`.ArgumentError` with more detail is now raised if the target
parameter for :meth:`_query.Query.join` is set to an unmapped object.
Prior to this change a less detailed ``AttributeError`` was raised.
Pull request courtesy Ramon Williams.

.. change::
:tags: bug, mysql
:tickets: 5568

The "skip_locked" keyword used with ``with_for_update()`` will emit a
warning when used on MariaDB backends, and will then be ignored. This is
a deprecated behavior that will raise in SQLAlchemy 1.4, as an application
that requests "skip locked" is looking for a non-blocking operation which
is not available on those backends.



.. change::
:tags: bug, engine
:tickets: 5599

Fixed issue where a non-string object sent to
:class:`_exc.SQLAlchemyError` or a subclass, as occurs with some third
party dialects, would fail to stringify correctly. Pull request
courtesy Andrzej Bartosiński.

.. change::
:tags: bug, sql
:tickets: 5644

Fixed issue where the ``pickle.dumps()`` operation against
:class:`_expression.Over` construct would produce a recursion overflow.

.. change::
:tags: postgresql, usecase
:tickets: 4392

The psycopg2 dialect now support PostgreSQL multiple host connections, by
passing host/port combinations to the query string. Pull request courtesy
Ramon Williams.

.. seealso::

:ref:`psycopg2_multi_host`

.. change::
:tags: bug, mysql
:tickets: 5617

Fixed bug where an UPDATE statement against a JOIN using MySQL multi-table
format would fail to include the table prefix for the target table if the
statement had no WHERE clause, as only the WHERE clause were scanned to
detect a "multi table update" at that particular point. The target
is now also scanned if it's a JOIN to get the leftmost table as the
primary table and the additional entries as additional FROM entries.


.. change::
:tags: bug, postgresql
:tickets: 5518

Adjusted the :meth:`_types.ARRAY.Comparator.any` and
:meth:`_types.ARRAY.Comparator.all` methods to implement a straight "NOT"
operation for negation, rather than negating the comparison operator.

.. change::
:tags: bug, pool
:tickets: 5582

Fixed issue where the following pool parameters were not being propagated
to the new pool created when :meth:`_engine.Engine.dispose` were called:
``pre_ping``, ``use_lifo``. Additionally the ``recycle`` and
``reset_on_return`` parameter is now propagated for the
:class:`_engine.AssertionPool` class.

.. change::
:tags: bug, ext, associationproxy
:tickets: 5541, 5542

An informative error is now raised when attempting to use an association
proxy element as a plain column expression to be SELECTed from or used in a
SQL function; this use case is not currently supported.


.. change::
:tags: bug, sql
:tickets: 5618

Fixed bug where an error was not raised in the case where a
:func:`_sql.column` were added to more than one :func:`_sql.table` at a
time. This raised correctly for the :class:`_schema.Column` and
:class:`_schema.Table` objects. An :class:`_exc.ArgumentError` is now
raised when this occurs.

.. change::
:tags: bug, orm
:tickets: 4589

Fixed issue where using a loader option against a string attribute name
that is not actually a mapped attribute, such as a plain Python descriptor,
would raise an uninformative AttributeError; a descriptive error is now
raised.



.. change::
:tags: mysql, usecase
:tickets: 5462

Adjusted the MySQL dialect to correctly parenthesize functional index
expressions as accepted by MySQL 8. Pull request courtesy Ramon Williams.

.. change::
:tags: bug, engine
:tickets: 5632

Repaired a function-level import that was not using SQLAlchemy's standard
late-import system within the sqlalchemy.exc module.


.. change::
:tags: change, mysql
:tickets: 5539

Add new MySQL reserved words: ``cube``, ``lateral`` added in MySQL 8.0.1
and 8.0.14, respectively; this indicates that these terms will be quoted if
used as table or column identifier names.

.. change::
:tags: bug, mssql
:tickets: 5592

Fixed issue where a SQLAlchemy connection URI for Azure DW with
``authentication=ActiveDirectoryIntegrated`` (and no username+password)
was not constructing the ODBC connection string in a way that was
acceptable to the Azure DW instance.

.. change::
:tags: bug, postgresql
:tickets: 5520

Fixed issue where the :class:`_postgresql.ENUM` type would not consult the
schema translate map when emitting a CREATE TYPE or DROP TYPE during the
test to see if the type exists or not. Additionally, repaired an issue
where if the same enum were encountered multiple times in a single DDL
sequence, the "check" query would run repeatedly rather than relying upon a
cached value.


.. change::
:tags: bug, tests
:tickets: 5635

Fixed incompatibilities in the test suite when running against Pytest 6.x.


.. changelog::

:released: August 17, 2020

.. change::
:tags: usecase, py3k
:tickets: 5357

Added a ``**kw`` argument to the :meth:`.DeclarativeMeta.__init__` method.
This allows a class to support the :pep:`487` metaclass hook
``__init_subclass__``. Pull request courtesy Ewen Gillies.


.. change::
:tags: bug, sql
:tickets: 5470

Repaired an issue where the "ORDER BY" clause rendering a label name rather
than a complete expression, which is particularly important for SQL Server,
would fail to occur if the expression were enclosed in a parenthesized
grouping in some cases. This case has been added to test support. The
change additionally adjusts the "automatically add ORDER BY columns when
DISTINCT is present" behavior of ORM query, deprecated in 1.4, to more
accurately detect column expressions that are already present.

.. change::
:tags: usecase, mysql
:tickets: 5481

The MySQL dialect will render FROM DUAL for a SELECT statement that has no
FROM clause but has a WHERE clause. This allows things like "SELECT 1 WHERE
EXISTS (subquery)" kinds of queries to be used as well as other use cases.


.. change::
:tags: bug, mssql, sql
:tickets: 5467

Fixed bug where the mssql dialect incorrectly escaped object names that
contained ']' character(s).

.. change::
:tags: bug, reflection, sqlite, mssql
:tickets: 5456

Applied a sweep through all included dialects to ensure names that contain
single or double quotes are properly escaped when querying system tables,
for all :class:`.Inspector` methods that accept object names as an argument
(e.g. table names, view names, etc). SQLite and MSSQL contained two
quoting issues that were repaired.

.. change::
:tags: bug, mysql
:tickets: 5411

Fixed an issue where CREATE TABLE statements were not specifying the
COLLATE keyword correctly.

.. change::
:tags: bug, datatypes, sql
:tickets: 4733

The ``LookupError`` message will now provide the user with up to four
possible values that a column is constrained to via the :class:`.Enum`.
Values longer than 11 characters will be truncated and replaced with
ellipses. Pull request courtesy Ramon Williams.

.. change::
:tags: bug, postgresql
:tickets: 5476

Fixed issue where the return type for the various RANGE comparison
operators would itself be the same RANGE type rather than BOOLEAN, which
would cause an undesirable result in the case that a
:class:`.TypeDecorator` that defined result-processing behavior were in
use. Pull request courtesy Jim Bosch.



.. change::
:tags: bug, mysql
:tickets: 5493

Added MariaDB code 1927 to the list of "disconnect" codes, as recent
MariaDB versions apparently use this code when the database server was
stopped.

.. change::
:tags: usecase, declarative, orm
:tickets: 5513

The name of the virtual column used when using the
:class:`_declarative.AbstractConcreteBase` and
:class:`_declarative.ConcreteBase` classes can now be customized, to allow
for models that have a column that is actually named ``type``. Pull
request courtesy Jesse-Bakker.

.. change::
:tags: usecase, orm
:tickets: 5494

Adjusted the workings of the :meth:`_orm.Mapper.all_orm_descriptors`
accessor to represent the attributes in the order that they are located in
a deterministic way, assuming the use of Python 3.6 or higher which
maintains the sorting order of class attributes based on how they were
declared. This sorting is not guaranteed to match the declared order of
attributes in all cases however; see the method documentation for the exact
scheme.



.. change::
:tags: bug, sql
:tickets: 5500

Fixed issue where the
:paramref:`_engine.Connection.execution_options.schema_translate_map`
feature would not take effect when the :meth:`_schema.Sequence.next_value`
function function for a :class:`_schema.Sequence` were used in the
:paramref:`_schema.Column.server_default` parameter and the create table
DDL were emitted.

.. changelog::