Aiootp

--------------------------


Major Changes
^^^^^^^^^^^^^

- All ``Database`` & ``AsyncDatabase`` filenames have been converted to
base36 to aid in making the manifest files & the databases as a whole
more space efficient. These changes are not backwards compatible.
- More work was done to clean up the databases & make them more
efficient, as well as equalize the sizes of the database files to
mitigate leaking metadata about what they might contain.
- Added new ``X25519`` & ``Ed25519`` classes that greatly simplify the
usage of the cryptography module's 25519 based tools. They also help
organize the codebase better -- where ``Ropake`` was holding onto
all of the asymmetric tooling even though those tools were not part
of the Ropake protocol.
- New base & helper ``Asymmetric25519`` & ``BaseEllipticCurve`` classes
were added as well to facilitate the reorganization.
- Many methods in ``Ropake`` were turned private to simplify & clean up
the interface so its intended use as a protocol is more clear for users.
- Added the time-to-live functionality to ``Ropake`` decryption functions.
The ``TIMEOUT`` attribute on the class can also be changed to import
a global time-to-live for all ``Ropake`` ciphertexts.
- Removed all ``nc_`` hash functions from the package/generics.py module.
- The ``Namespace`` class now has a ``keys`` method so that namespaces
can be unpacked using star-star syntax.
- Because of the ongoing failures of gnupg, we are moving away from
signing our packages with gnupg. Our new Ed25519 keys will be from
the cryptography package, & we'll sign those with our gnupg key as a
secondary form of attestation. Our package signing will be automated
in the setup.py file & the methods we use will be transparent in the
code. The new signatures for each package version will be placed in
the file ``SIGNATURES.txt``.


Minor Changes
^^^^^^^^^^^^^

- Many fixes & additions to docstrings & tutorials.
- Massive refactorings, cleanups & typo fixes across the library,
especially in the database classes, ``Ropake`` & the ``ciphers`` module.
- Added comprehensive functional tests for the Ropake class.
- Added ``BASE_36_TABLE`` to the ``commons`` module.
- Fixed metadata issues in setup.py that caused upload issues to pypi.
- The ``generate_profile``, ``load_profile``, ``agenerate_profile`` &
``aload_profile`` database methods now accept arbitrary keyword arguments
that get passed into the database's __init__ constructor.
- ``username`` & ``password`` are now required keyword-only arguments
to the ``agenerate_profile_tokens`` & ``generate_profile_tokens``
classmethods.
- The ``aload`` & ``load`` database methods now take a ``manifest`` kwarg
that when toggled ``True`` will also refresh the manifest file from
disk.
- Now when a database object is ordered to delete itself, the entirety
of the instance's caches & attribute values are cleared & deleted.
- Filled out the references to strong key generators & protocols in the
``keygens`` module.

--------------------------


Major Changes
^^^^^^^^^^^^^

- Security Patch: The previous update left the default salt stored by
the ``Ropake`` class on the user filesystem as an empty string for
new files that were created since the ``asalt`` & ``salt`` functions
were switched to producing 256-bit values instead of 512-bits. This
bug has now been fixed.
- An 8 byte timestamp is now prepended to each plaintext during the
padding step. The decryption functions now take a ``ttl`` kwarg which
will measure & enforce a time-to-live for ciphertexts under threat of
``TimeoutError``.
- Added new profile feature to the database classes. This standardizes
& simplifies the process for users to open databases using only
low-entropy "profile" information such as ``username``, ``password``,
``*credentials`` & an optional ``salt`` a user may have access to.
The new ``agenerate_profile_tokens``, ``generate_profile_tokens``,
``agenerate_profile``, ``generate_profile``, ``aprofile_exists``,
``profile_exists``, ``aload_profile``, ``load_profile``, ``adelete_profile``
& ``delete_profile`` functions are the public part of this new feature.
- Some more database class attributes have been turned private to clean
up the api.
- Fixed typo in ``__exit__`` method of ``Database`` class which referenced
a method which had its name refactored, leading to a crash.
- Shifted the values in the ``primes`` dictionary such that the key for
each element in the dictionary is the exclusive maximum of each prime
in that element. Ex: primes[512][-1].to_bytes(64, "big") is now valid.
Whereas before, primes[512] was filled with primes that were 64 bytes
and 1 bit long, making them 65 byte primes. This changes some of the
values of constants in the package & therefore some values derived
from those constants.
- Slimmed down the number of elements in the ``primes`` & ``bits``
dictionaries, reducing the size of the package a great deal. ``primes``
now contains two primes in each element, the first is the minimum
prime of that bit length, the latter the maximum.
- Added ``URLSAFE_TABLE`` to the package.
- Made ``salt`` & ``pid`` & ``ttl`` keyword only arguments in key
generators & encryption / decryption functions, further tighening up
the api.


Minor Changes
^^^^^^^^^^^^^

- Added ``this_second`` function to ``asynchs`` module for integer time.
- Added ``apadding_key``, ``padding_key``, ``aplaintext_stream`` &
``plaintext_stream`` functions to the ``ciphers`` module.
- Added ``apadding_key``, ``padding_key`` to the ``keygens`` module &
``AsyncKeys`` & ``Keys`` classes.
- Added ``axi_mix``, ``xi_mix``, ``acheck_timestamp``, ``check_timestamp``,
to the ``generics`` module.
- Added ``acsprbg``, ``csprbg``, ``asalt``, ``salt``, ``apadding_key``,
``padding_key``, ``aplaintext_stream`` & ``plaintext_stream`` functions
to OneTimePad class as ``staticmethod`` & instance methods.
- Added ``acheck_timestamp`` & ``check_timestamp`` functions to the
``BytesIO`` class.
- Added ``adeniable_filename`` & ``deniable_filename`` to the ``paths``
module.
- Removed check for falsey data in encryption functions. Empty data is
& should be treated as valid plaintext.
- Various refactorings, docstring fixes & efficiency improvements.
- Added some new tests for database profiles.

--------------------------


Major Changes
^^^^^^^^^^^^^

- Security patch: The ``apad_bytes``, ``pad_bytes``, ``adepad_bytes`` &
``depad_bytes`` functions were changed internally to execute in a
more constant time. The variations were small for 256-byte buffers
(the default), but can grow very wide with larger buffers. The salt
in the package's encryption utilities is now used to derive the
plaintext's padding, making each padding unique.
- Unified the types of encodings the library's encryption functions
utilize for producing ciphertext. This includes databases. They now
all use the ``LIST_ENCODING``. This greatly increases the efficiency
of the databases' encryption/decryption, save/load times. And this
encoding is more space efficient. This change is backwards
incompatible.
- The ``LIST_ENCODING`` specification was also changed to produce
smaller ciphertexts. The salt is no longer encrypted & included as
the first 256 byte chunk of ciphertext. It is now packaged along with
ciphertext in the clear & is restricted to being a 256-bit hex
string.
- The interfaces for the ``Database`` & ``AsyncDatabase`` were cleaned
up. Many attributes & functions that were not intended as the public
interface of the classes were made "private". Also, the no longer
used utilities for encrypting & decrypting under the MAP_ENCODING
were removed.
- Updated the ``abytes_xor``, ``bytes_xor``, ``axor`` & ``xor`` generators
to shrink the size of the ``seed`` that's fed into the ``keystream``. This
allows the one-time-pad cipher to be more cpu efficient.


Minor Changes
^^^^^^^^^^^^^

- Fixed various typos, docstrings & tutorials that have no kept up
with the pace of changes.
- Various refactorings throughout.
- The ``akeypair`` & ``keypair`` functions now produce a ``Namespace``
populated with a 512-bit hex key & a 256-bit hex salt to be more
consistent with their intended use-case with the one-time-pad cipher.
- Removed ``aencode_salt``, ``encode_salt``, ``adecode_salt`` &
``decode_salt`` functions since they are no longer used in conjunction
with LIST_ENCODING ciphertexts.
- Updated tests to recognize these changes.
- Gave the ``OneTimePad`` class access to a ``BytesIO`` object under a
new ``io`` attribute.

--------------------------


Major Changes
^^^^^^^^^^^^^

- Security Patch: ``xor`` & ``axor`` functions that define the
one-time-pad cipher had a vulnerability fixed that can leak <1-bit of
plaintext. The issue was in the way keys were built, where the
multiplicative products of two key segments were xor'd together. This
lead to keys being slightly more likely to be positive integers,
meaning the final bit had a greater than 1/2 probability of being a
``0``. The fix is accompanied with an overhaul of the one-time-pad
cipher which is more efficient, faster, & designed with a better
understanding of the way bytes are processed & represented. The key
chunks now do not, & must not, surpass 256 bytes & neither should
any chunk of plaintext output. Making each chunk deterministically
256 bytes allows for reversibly formatting ciphertext to & from
bytes-like strings. These changes are backwards incompatible with
prior versions of this package & are strongly recommended.
- Added ``bytes_xor`` & ``abytes_xor`` functions which take in key
generators which produce key segments of type bytes instead of hex
strings.
- ``AsyncDatabase`` & ``Database`` now save files in bytes format,
making them much more efficient on disk space. They use the new
``BytesIO`` class in the ``generics`` module to transparently convert
to & from json & bytes. This change is also not backwards compatible.
- Removed ``acipher``, ``cipher``, ``adecipher``, ``decipher``,
``aorganize_encryption_streams``, ``organize_encryption_streams``,
``aorganize_decryption_streams``, ``organize_decryption_streams``,
``aencrypt``, ``encrypt``, ``adecrypt``, ``decrypt``, ``asubkeys`` &
``subkeys`` generators from the ``ciphers`` module & package to slim
down the code, remove repetition & focus on the cipher tools that
include hmac authentication.
- Removed deprecated diffie-hellman methods in ``Ropake`` class.
- Removed the static ``power10`` dictionary from the package.
- The default secret salt for the ``Ropake`` class is now derived from the
contents of a file that's in the databases directory which is chmod'd to
0o000 unless needed.
- Made ``aclient_message_key``, ``client_message_key``, ``aserver_message_key``,
& ``server_message_key`` ``Ropake`` class methods to help distinguish
client-to-server & server-to-client message keys which prevents replay
attacks on the one-message ROPAKE protocol.
- Added protocol coroutines to the ``Ropake`` class which allow for easily
engaging in 2DH & 3DH elliptic curve exchanges for servers & clients.
- Efficiency improvements to the ``aseeder`` & ``seeder`` generator functions
in the ``randoms`` module. This affects the ``acsprng`` & ``csprng`` objects
& all the areas in the library that utilize those objects.
- Changed the repr behavior of ``Comprende`` instances to redact all args &
kwargs by default to protect cryptographic material from unintentionally
being displayed on user systems. The repr can display full contents by
calling the ``enable_debugging`` method of the ``DebugControl`` class.
- All generator functions decorated with ``comprehension`` are now given
a ``root`` attribute. This allows direct access to the function without
needing to instantiate or run it as a ``Comprende`` object. This saves
a good deal of cpu & time in the overhead that would otherwise be
incurred by the class. This is specifically more helpful in tight &/or
lower-level looping.


Minor Changes
^^^^^^^^^^^^^

- Various refactorings across the library.
- Fixed various typos, bugs & inaccurate docstrings throughout the library.
- Add ``chown`` & ``chmod`` functions to the ``asynchs.aos`` module.
- Now makes new ``multiprocessing.Manager`` objects in the ``asynchs.Processes``
& ``asynchs.Threads`` classes to avoid errors that occur when using a stale
object whose socket connections are closed.
- Changed ``Ropake`` class' ``adb_login`` & ``db_login`` methods to
``adatabase_login_key`` & ``database_login_key``. Also, fix a crash bug in
those methods.
- Changed ``Ropake`` class' ``aec25519_pub``, ``ec25519_pub``, ``aec25519_priv``
& ``ec25519_priv`` methods to ``aec25519_public_bytes``, ``ec25519_public_bytes``,
``aec25519_private_bytes`` & ``ec25519_private_bytes``.
- Added low-level private methods to ``Ropake`` class which do derivation
& querying of the default class key & salt.
- Behavior changes to the ``ainverse_int`` & ``inverse_int`` functions in the
``generics`` module to allow handling bases represented in ``str`` or ``bytes``
type strings.
- Behavior & name changes to the ``abinary_tree`` & ``binary_tree`` functions in the
``generics`` module to ``abuild_tree`` & ``build_tree``. They now allow making
uniform trees of any width & depth, limited only by the memory in a
user's machine.
- Provided new ``acsprbg`` & ``csprbg`` objects to the library that return 512-bits
of cryptographically secure pseudo-random ``bytes`` type strings. They are
made by the new ``abytes_seeder`` & ``bytes_seeder`` generators.
- The ``csprng``, ``acsprng``, ``csprbg`` & ``acsprbg`` objects were
wrapped in functions that automatically restart the generators if they're
stalled / interrupted during a call. This keeps the package from melting
down if it can no longer call the CSPRNGs for new entropy.
- Cleaned up & simplified ``table_key`` functions in the ``keygens`` module.
- Changed the output of ``asafe_symm_keypair`` & ``safe_symm_keypair`` functions
to contain bytes values not their hex-only representation. Also removed
these functions from the main imports of the package since they are slow
& their main contribution is calling ``arandom_number_generator`` &
``random_number_generator`` to utilize a large entropy pool when starting
CSPRNGs.
- Added new values to the ``bits`` dictionary.
- Added ``apad_bytes``, ``pad_bytes``, ``adepad_bytes`` & ``depad_bytes``
functions which use ``shake_256`` to pad/depad plaintext bytes to & from
multiples of 256 bytes. They take in a key to create the padding.
This method is intended to also aid in protecting against padding
oracle attacks.

--------------------------


Major Changes
^^^^^^^^^^^^^

- The OPAKE protocol was renamed to ROPAKE, an acronym for Ratcheting
Opaque Password Authenticated Key Exchange. This change was necessary
since OPAKE is already a name for an existing PAKE protocol. This change
also means the ``Opake`` class name was changed to ``Ropake``.
- The ``Ropake`` class' registration algorithm was slightly modified to
use the generated Curve25519 ``shared_key`` an extra time in the key
derivation process. This shouldn't break any currently authenticated
sessions.
- The ``asyncio_contextmanager`` package is no longer a listed dependency
in ``setup.py``. The main file from that package was copied over into the
``/aiootp`` directory in order to remove the piece of code that caused
warnings to crop up when return values were retrieved from async
generators. This change will put an end to this whack-a-mole process of
trying to stop the warnings with try blocks scattered about the codebase.
- Added ``asave_tag``, ``save_tag``, ``asave_file`` & ``save_file`` methods
to the database classes so that specific entries can be saved to disk
without having to save the entire database which is much more costly. The
manifest file isn't saved to disk when these methods are used, so if a
tag file isn't already saved in the database, then the saved files will
not be present in the manifest or in the cache upon subsequent loads of
the database. The saved file will still however be saved on the
filesystem, though unbeknownst to the database instance.
- The ``Namespace`` class now redacts all obvious key material in instance
repr's, which is any 64+ hex character string, or any number with 64+
decimal digits.
- Removed the experimental recursive value retrieval within ``Comprende``'s
``__aexamine_sent_exceptions`` & ``__examine_sent_exceptions`` methods.
This change leads to more reliable & faster code, in exchange for an
unnecessary feature being removed.
- Bug fix of the ``auuids`` & ``uuids`` methods by editing the code in
the ``asyncio_contextmanager`` dependency & using the patched package
instead of the ``comprehension`` decorator for the ``arelay`` & ``relay``
methods of ``Comprende``. Their internal algorithms was also updated to
be simpler, but are incompatible with the outputs of past versions of
these methods.


Minor Changes
^^^^^^^^^^^^^

- Various refactorings & documentation additions / modifications throughout
the library.
- Various small bug fixes.
- The shared keys derived from the ``Ropake`` protocol are now returned in
a ``Namespace`` object instead of a raw dictionary, which allows the
values to be retrieved by dotted &/or bracketed lookup.
- The ``atest_hmac`` & ``test_hmac`` algorithms / methods were made more
efficient & were refactored. Now they call ``atime_safe_equality`` &
``time_safe_equality`` internally, which are new methods that can apply
the non-constant time but randomized timing comparisons on any pairs of
values.

--------------------------


Major Changes
^^^^^^^^^^^^^

- The Opake protocol was made greatly more efficient. This was done by
replacing the diffie-hellman verifiers with a hash & xor commit & reveal
system. Most hashing was made more efficient my using quicker & smaller
``sha_512`` function instead of ``nc_512``, & streamlining the protocol.
- The ``Opake.client`` & ``Opake.client_registration`` methods now take
an instantiated client database instead of client credentials which
improves security, efficiency & usability. This change reduces the amount
of exposure received by user passwords & other credentials. It also
simplifies usage of the protocol by only needing to carry around a
database instead of a slew of credentials, which is also faster, since
the credentials are passed through the cpu & memory hard ``passcrypt``
function everytime to open the database.


Minor Changes
^^^^^^^^^^^^^

- Heavy refactorings & documentation additions / modifications of the
``Opake`` class. Removed the ``Opake.ainit_database`` & ``Opake.init_database``
methods, & made the ``salt`` default argument parameter in
``Opake.aclient_database``, ``Opake.client_database``, ``Opake.adb_login`` &
``Opake.db_login`` into a keyword only argument so any extra user defined
``credentials`` are able to be passed without specifying a salt.
- The decorators for the ``Comprende.arelay`` & ``Comprende.relay`` methods
were changed from ``asyncio_contextmanager.async_contextmanager`` to
``comprehension()`` to stop that package from raising exceptions when
we retrieve return values from async generators.