Databricks-labs-ucx

* Added `databricks labs ucx alias` command to create a view of tables from one schema/catalog in another schema/catalog ([837](https://github.com/databrickslabs/ucx/pull/837)).
* Added `databricks labs ucx save-aws-iam-profiles` command to scan instance profiles identify AWS S3 access and save a CSV with permissions ([817](https://github.com/databrickslabs/ucx/pull/817)).
* Added total view counts in the assessment dashboard ([834](https://github.com/databrickslabs/ucx/pull/834)).
* Cleaned up `assess_jobs` and `assess_clusters` tasks in the `assessment` workflow to improve testing and reduce redundancy.([825](https://github.com/databrickslabs/ucx/pull/825)).
* Added documentation for the assessment report ([806](https://github.com/databrickslabs/ucx/pull/806)).
* Fixed escaping for SQL object names ([836](https://github.com/databrickslabs/ucx/pull/836)).

Dependency updates:

* Updated databricks-sdk requirement from ~=0.17.0 to ~=0.18.0 ([832](https://github.com/databrickslabs/ucx/pull/832)).

* Added `databricks labs ucx validate-groups-membership` command to validate groups to see if they have same membership across acount and workspace level ([772](https://github.com/databrickslabs/ucx/pull/772)).
* Added baseline for getting Azure Resource Role Assignments ([764](https://github.com/databrickslabs/ucx/pull/764)).
* Added issue and pull request templates ([791](https://github.com/databrickslabs/ucx/pull/791)).
* Added linked issues to PR template ([793](https://github.com/databrickslabs/ucx/pull/793)).
* Added optional `debug_truncate_bytes` parameter to the config and extend the default log truncation limit ([782](https://github.com/databrickslabs/ucx/pull/782)).
* Added support for crawling grants and applying Hive Metastore UDF ACLs ([812](https://github.com/databrickslabs/ucx/pull/812)).
* Changed Python requirement from 3.10.6 to 3.10 ([805](https://github.com/databrickslabs/ucx/pull/805)).
* Extend error handling of delta issues in crawlers and hive metastore ([795](https://github.com/databrickslabs/ucx/pull/795)).
* Fixed `databricks labs ucx repair-run` command to execute correctly ([801](https://github.com/databrickslabs/ucx/pull/801)).
* Fixed handling of `DELTASHARING` table format ([802](https://github.com/databrickslabs/ucx/pull/802)).
* Fixed listing of workflows via CLI ([811](https://github.com/databrickslabs/ucx/pull/811)).
* Fixed logger import path for DEBUG notebook ([792](https://github.com/databrickslabs/ucx/pull/792)).
* Fixed move table command to delete table/view regardless if permissions are present, skipping corrupted tables when crawling table size and making existing tests more stable ([777](https://github.com/databrickslabs/ucx/pull/777)).
* Fixed the issue of `databricks labs ucx installations` and `databricks labs ucx manual-workspace-info` ([814](https://github.com/databrickslabs/ucx/pull/814)).
* Increase the unit test coverage for cli.py ([800](https://github.com/databrickslabs/ucx/pull/800)).
* Mount Point crawler lists /Volume with four variations which is confusing ([779](https://github.com/databrickslabs/ucx/pull/779)).
* Updated README.md to remove mention of deprecated install.sh ([781](https://github.com/databrickslabs/ucx/pull/781)).
* Updated `bug` issue template ([797](https://github.com/databrickslabs/ucx/pull/797)).
* Fixed writing log readme in multiprocess safe way ([794](https://github.com/databrickslabs/ucx/pull/794)).

* Added assessment step to estimate the size of DBFS root tables ([741](https://github.com/databrickslabs/ucx/pull/741)).
* Added `TableMapping` functionality to table migrate ([752](https://github.com/databrickslabs/ucx/pull/752)).
* Added `databricks labs ucx move` command to move tables and schemas between catalogs ([756](https://github.com/databrickslabs/ucx/pull/756)).
* Added functionality to determine migration method based on DBFS Root ([759](https://github.com/databrickslabs/ucx/pull/759)).
* Added `get_tables_to_migrate` functionality in the mapping module ([755](https://github.com/databrickslabs/ucx/pull/755)).
* Added retry and rate limit to rename workspace group operation and corrected rate limit for reflecting account groups to workspace ([751](https://github.com/databrickslabs/ucx/pull/751)).
* Adopted `databricks-labs-blueprint` library for common utilities to be reused in the other projects ([758](https://github.com/databrickslabs/ucx/pull/758)).
* Converted `RuntimeBackend` query executions exceptions to SDK exceptions ([769](https://github.com/databrickslabs/ucx/pull/769)).
* Fixed issue with missing users and temp groups after workspace-local groups migration and skip table when crawling table size if it does not exist anymore ([770](https://github.com/databrickslabs/ucx/pull/770)).
* Improved error handling by not failing group rename step if a group was removed from account before reflecting it to workspace ([762](https://github.com/databrickslabs/ucx/pull/762)).
* Improved error message inference from failed workflow runs ([753](https://github.com/databrickslabs/ucx/pull/753)).
* Moved `TablesMigrate` to a separate module ([747](https://github.com/databrickslabs/ucx/pull/747)).
* Reorganized assessment dashboard to increase readability ([738](https://github.com/databrickslabs/ucx/pull/738)).
* Updated databricks-sdk requirement from ~=0.16.0 to ~=0.17.0 ([773](https://github.com/databrickslabs/ucx/pull/773)).
* Verify metastore exists in current workspace ([735](https://github.com/databrickslabs/ucx/pull/735)).

* Added `databricks labs ucx repair-run --step ...` CLI command for repair run of any failed workflows, like `assessment`, `migrate-groups` etc. ([724](https://github.com/databrickslabs/ucx/pull/724)).
* Added `databricks labs ucx revert-migrated-table` command ([729](https://github.com/databrickslabs/ucx/pull/729)).
* Allow specifying a group list when group match options are used ([725](https://github.com/databrickslabs/ucx/pull/725)).
* Fixed installation issue when upgrading from an older version of the tool and improve logs ([740](https://github.com/databrickslabs/ucx/pull/740)).
* Renamed summary panel from Failure Summary to Assessment Summary ([733](https://github.com/databrickslabs/ucx/pull/733)).
* Retry internal error when getting permissions and update legacy table ACL documentation ([728](https://github.com/databrickslabs/ucx/pull/728)).
* Speedup installer execution ([727](https://github.com/databrickslabs/ucx/pull/727)).

* Added `databricks labs ucx create-table-mapping` and `databricks labs ucx manual-workspace-info` commands for CLI ([682](https://github.com/databrickslabs/ucx/pull/682)).
* Added `databricks labs ucx ensure-assessment-run` to CLI commands ([708](https://github.com/databrickslabs/ucx/pull/708)).
* Added `databricks labs ucx installations` command ([679](https://github.com/databrickslabs/ucx/pull/679)).
* Added `databricks labs ucx skip --schema ... --table ...` command to mark table/schema for skipping in the table migration process ([680](https://github.com/databrickslabs/ucx/pull/680)).
* Added `databricks labs ucx validate-external-locations` command for cli ([715](https://github.com/databrickslabs/ucx/pull/715)).
* Added capturing `ANY FILE` and `ANONYMOUS FUNCTION` grants ([653](https://github.com/databrickslabs/ucx/pull/653)).
* Added cluster override and handle case of write protected DBFS ([610](https://github.com/databrickslabs/ucx/pull/610)).
* Added cluster policy selector in the installer ([655](https://github.com/databrickslabs/ucx/pull/655)).
* Added detailed UCX pre-requisite information to README.md ([689](https://github.com/databrickslabs/ucx/pull/689)).
* Added interactive wizard for `databricks labs uninstall ucx` command ([657](https://github.com/databrickslabs/ucx/pull/657)).
* Added more granular error retry logic ([704](https://github.com/databrickslabs/ucx/pull/704)).
* Added parallel fetching of registered model identifiers to speed-up assessment workflow ([691](https://github.com/databrickslabs/ucx/pull/691)).
* Added retry on workspace listing ([659](https://github.com/databrickslabs/ucx/pull/659)).
* Added support for mapping workspace group to account group by prefix/suffix/regex/external id ([650](https://github.com/databrickslabs/ucx/pull/650)).
* Changed cluster security mode from NONE to LEGACY_SINGLE_USER, as `crawl_tables` was failing when run on non-UC Workspace in No Isolation mode with unable to access the config file ([661](https://github.com/databrickslabs/ucx/pull/661)).
* Changed the fields of the table "Tables" to lower case ([684](https://github.com/databrickslabs/ucx/pull/684)).
* Enabled integration tests for `EXTERNAL` table migrations ([677](https://github.com/databrickslabs/ucx/pull/677)).
* Enforced `mypy` validation ([713](https://github.com/databrickslabs/ucx/pull/713)).
* Filtering out inventory database from loading into tables and filtering out the same from grant detail view ([705](https://github.com/databrickslabs/ucx/pull/705)).
* Fixed documentation for `reflect_account_groups_on_workspace` task and updated `CONTRIBUTING.md` guide ([654](https://github.com/databrickslabs/ucx/pull/654)).
* Fixed secret scope apply task to raise ValueError ([683](https://github.com/databrickslabs/ucx/pull/683)).
* Fixed legacy table ACL ownership migration and other integration testing issues ([722](https://github.com/databrickslabs/ucx/pull/722)).
* Fixed some flaky integration tests ([700](https://github.com/databrickslabs/ucx/pull/700)).
* New CLI command for workspace mapping ([678](https://github.com/databrickslabs/ucx/pull/678)).
* Reduce server load for getting workspace groups and their members ([666](https://github.com/databrickslabs/ucx/pull/666)).
* Throwing ManyError on migrate-groups tasks ([710](https://github.com/databrickslabs/ucx/pull/710)).
* Updated installation documentation to use Databricks CLI ([686](https://github.com/databrickslabs/ucx/pull/686)).

Dependency updates:

* Updated databricks-sdk requirement from ~=0.13.0 to ~=0.14.0 ([651](https://github.com/databrickslabs/ucx/pull/651)).
* Updated databricks-sdk requirement from ~=0.14.0 to ~=0.15.0 ([687](https://github.com/databrickslabs/ucx/pull/687)).
* Updated databricks-sdk requirement from ~=0.15.0 to ~=0.16.0 ([712](https://github.com/databrickslabs/ucx/pull/712)).

* Added current version of UCX to task logs ([566](https://github.com/databrickslabs/ucx/pull/566)).
* Fixed `'str' object has no attribute 'value'` failure on apply backup group permissions task ([574](https://github.com/databrickslabs/ucx/pull/574)).
* Fixed `crawl_cluster` failure over custom runtimes ([602](https://github.com/databrickslabs/ucx/pull/602)).
* Fixed `databricks labs ucx workflows` command ([608](https://github.com/databrickslabs/ucx/pull/608)).
* Fixed problematic integration test fixture `make_ucx_group` ([613](https://github.com/databrickslabs/ucx/pull/613)).
* Fixed internal API request retry logic by relying on concrete exception types ([637](https://github.com/databrickslabs/ucx/pull/637)).
* Fixed `tables.scala` notebook to read inventory database from `~/.ucx/config.yml` file. ([614](https://github.com/databrickslabs/ucx/pull/614)).
* Introduced `StaticTablesCrawler` for integration tests ([632](https://github.com/databrickslabs/ucx/pull/632)).
* Reduced runtime of `test_set_owner_permission` from 15 minutes to 44 seconds ([636](https://github.com/databrickslabs/ucx/pull/636)).
* Updated `LICENSE` ([643](https://github.com/databrickslabs/ucx/pull/643)).
* Updated documentation ([611](https://github.com/databrickslabs/ucx/pull/611), [#646](https://github.com/databrickslabs/ucx/pull/646)).

**Breaking changes** (existing installations need to remove `ucx` database, reinstall UCX and re-run assessment jobs)
* Fixed external locations widget to return hostname for `jdbc:`-sourced tables ([621](https://github.com/databrickslabs/ucx/pull/621)).