Mlrun

Features / Enhancements
* **Model Monitoring**: Infer v3io kv schema through batch job, 4243, Eyal-Danieli
* **Notification**: Add notification push failure reason, 4233, liranbg
* **Runtimes**: Enrich function with image from task, 4241, alonmr
* **Docs**: Add running spark job on databricks, 4161, jillnogold
* **Pipelines**: Update items function response type, 4235, liranbg
* **Feature store**: Add `deploy_ingestion_service_v2`, 4232, gtopper
* **Requirements**: Move alembic to api, 4213, alonmr
* **CI**: Periodically rebuild base image as well, 4227, liranbg
* **LogCollector**: Retry upon transient failures, 4225, liranbg
* **UI**: [Features & enhancement](https://github.com/mlrun/ui/releases/tag/v1.5.0-rc11#features-and-enhancements)

Bug fixes
* **CI**: Fix backwards compatibility check, 4239, liranbg
* **Project**: Fix set tagged function, 4230, alonmr
* **Model Monitoring**: Fix delete model endpoints resources on ce, 4229, Eyal-Danieli
* **UI**: [Bug fixes](https://github.com/mlrun/ui/releases/tag/v1.5.0-rc11#bug-fixes)


Pull requests:
af4a00a1 [Model Monitoring] Infer V3IO KV schema through batch job (4243)
b020efbc [Notification] Add notification push failure reason (4233)
04fe390b [Runtimes] Enrich function with image from task (4241)
9f0eeab6 [Docs] Add running Spark job on Databricks (4161)
a9b4235d [CI] Fix backwards compatibility check (4239)
8e1470be [Pipelines] Update items function response type (4235)
514ea27d [Feature store] Add `deploy_ingestion_service_v2` (4232)
8c8f0749 [Project] Fix set tagged function (4230)
5906080f [Requirements] Move alembic to API (4213)
0527f5df [Model Monitoring] Fix delete model endpoints resources on CE (4229)
38e92d27 [CI] Periodically rebuild base image as well (4227)
e913b438 [LogCollector] Retry upon transient failures (4225)

Features / Enhancements
* **Builder**: Add retry mechanism to kaniko, 4223, liranbg
* **CI**: Merging lint and licensing check onto a single job, 4222, liranbg
* **Model Monitoring**: Use a shared class for the model endpoint tables, 4196, jond01
* **Tests**: Limit test_log_artifact_with_run_function to enterprise only., 4221, tomerm-iguazio
* **Requirements**: Add `sqlalchemy~=1.4` to extra requirements, 4212, Eyal-Danieli
* **Model Monitoring**: Check stream pod health before pushing metrics to prometheus registry, 4197, Eyal-Danieli
* **Requirements**: Bump semver to 3.0, 4217, alonmr
* **Model Monitoring**: Enable prometheus stack installation in ce system tests, 4215, Eyal-Danieli
* **Project**: Load project raise if not found, 4214, alonmr
* **CI**: Remove ml-models image from system tests action, 4211, alonmr
* **Docs**: Add data store profile usage, 4131, jillnogold
* **Requirements**: Bump nuclio-jupyter to 0.9.12, 4209, alonmr
* **Artifacts**: Replacing run project in artifact path, 4193, tomerm-iguazio
* **Model Monitoring**: Add retry on model endpoint metrics assertion in system tests, 4199, Eyal-Danieli
* **Runtimes**: Prevent running databricks runtime locally, 4203, tomerm-iguazio
* **Docs**: Remove use of mlrun/ml-models image in tutorial, 4182, aviaIguazio
* **Datastore**: Proper error handling of datastore profile reading, 4195, alxtkr77
* **API**: Reduce data version resolvement overhead, 4201, alonmr
* **HTTPDB**: Change `list_projects` default format to name only, 4198, alonmr
* **Images**: Remove ml-models image, 4071, alonmr
* **Serving**: Save parent function before children deployment, 4194, alonmr
* **CI**: Update tutorial folder name [release], 4191, daniels290813
* **UI**: [Features & enhancement](https://github.com/mlrun/ui/releases/tag/v1.5.0-rc10#features-and-enhancements)

Bug fixes
* **Tests**: Fix transient failure, 4224, liranbg
* **HTTPDB**: Fix list runs, 4220, liranbg
* **Images**: [jupyter] fix path to tutorial images, 4207, alonmr
* **Images**: Fix jupyter tutorials path, 4202, alonmr
* **Model Monitoring**: Fix different dataset types when reading a dataset as a dataframe, 4189, Eyal-Danieli
* **Runtimes**: Fix mount_v3io: use `dirstocreate` volume param to enforce creation of user dir with o750 permission, 4192, assaf758
* **Datastore**: Fix error when using `store://` url in data source, 4190, gtopper
* **UI**: [Bug fixes](https://github.com/mlrun/ui/releases/tag/v1.5.0-rc10#bug-fixes)


Pull requests:
8a341998 [Tests] Fix transient failure (4224)
1a53c3ba [HTTPDB] Fix list runs (4220)
157fc33a [Builder] Add retry mechanism to kaniko (4223)
04d57e0c [CI] Merging lint and licensing check onto a single job (4222)
8152dbd7 [Model Monitoring] Use a shared class for the model endpoint tables (4196)
5b0e8ed7 [Tests] Limit test_log_artifact_with_run_function to enterprise only. (4221)
ad59161e [Requirements] Add `sqlalchemy~=1.4` to extra requirements (4212)
b0263946 [Model Monitoring] Check stream pod health before pushing metrics to Prometheus registry (4197)
1d939998 [Requirements] Bump semver to 3.0 (4217)
bfb213ea [Model Monitoring] Enable prometheus stack installation in CE system tests (4215)
3c4ea626 [Project] Load project raise if not found (4214)
6fb0e991 [CI] Remove ml-models image from system tests action (4211)
8eca58a8 [Docs] Add data store profile usage (4131)
86c2dea3 [Requirements] Bump nuclio-jupyter to 0.9.12 (4209)
91b491e0 [Artifacts] Replacing run project in artifact path (4193)
e49cd6ce [Model Monitoring] Add retry on model endpoint metrics assertion in system tests (4199)
8821f3c1 [Images][Jupyter] Fix path to tutorial images (4207)
b5fe0c05 [Runtimes] Prevent running databricks runtime locally (4203)
dcbfd53c [Docs] Remove use of mlrun/ml-models image in tutorial (4182)
29494b17 [Datastore] Proper error handling of datastore profile reading (4195)
e37f4dcf [Images] Fix Jupyter tutorials path (4202)
1e40c252 [API] Reduce data version resolvement overhead (4201)
1f9fb79f [HTTPDB] Change `list_projects` default format to name only (4198)
70e24f3d [Images] Remove ml-models image (4071)
9589e7fd [Serving] Save parent function before children deployment (4194)
6912fd05 [CI] Update tutorial folder name [release] (4191)
609ceaed [Model Monitoring] Fix different dataset types when reading a dataset as a DataFrame (4189)
0b678def [Runtimes] Fix mount_v3io: use `dirsToCreate` volume param to enforce creation of user dir with o750 permission (4192)
da9bd875 [Datastore] Fix error when using `store://` URL in data source (4190)

Features / Enhancements
* **Datastore**: Introducing the api for creating client temporary datastore profiles, 4183, alxtkr77
* **Runtimes**: Support for aborting databricks jobs, 4165, tomerm-iguazio
* **Dockerfiles**: Remedy python os packages vulnerabilities, 4181, liranbg
* **BatchRuns**: Improve err and default strategy, 4167, liranbg
* **Docs**: Add docs for notifications secret_params, 4176, yaelgen
* **Dockerfiles**: Update conda initial requirements, 4180, liranbg
* **Automation**: Add ability to reset db after deploy, 4153, yanburman
* **Runtimes**: [dask] ensure that mlrun_dbpath is available for workers, 4172, laurybueno
* **Dockerfiles**: Mlrun/mlrun to use miniconda + opmi, 4166, liranbg
* **Notifications**: Add secret_params to db, 4178, yaelgen
* **Notifications**: Add optional to secret_params, 4177, yaelgen
* **Datastore**: Add list and delete apis for datastore profiles, 4148, alxtkr77
* **Dockerfiles**: Update ignoredockerfile, 4173, liranbg
* **Docs**: Clarify use of csv, json files for hyperparameters, 4164, jillnogold
* **Frameworks**: Raise an exception when no framework has been found, 4171, Eyal-Danieli
* **Model Monitoring**: Add model endpoint validations when calling `get_or_create_model_endpoint()`, 4160, Eyal-Danieli
* **UI**: [Features & enhancement](https://github.com/mlrun/ui/releases/tag/v1.5.0-rc9#features-and-enhancements)

Bug fixes
* **Docs**: Change folder name from tutorial to tutorials, fix link to function hub, 4185, jillnogold
* **Tests**: Fix expected mlrun executable location to conda binaries, 4187, alonmr
* **Tests**: Fix kfp pipeline param test, 4184, alonmr
* **Dockerfiles**: Fix mlrun gpu build, 4186, liranbg
* **Dockerfiles**: Fix building base image, 4179, liranbg
* **Docs**: Fix monitoring grafana dashboards links for old igz versions, 4174, Eyal-Danieli
* **Notifications**: Fix notifications spamming project secret - part 2, 4170, yaelgen
* **Model Monitoring**: Fix histogram padding when model object has no feature stats, 4169, jond01
* **Notifications**: Fix notifications spamming project secret - part 1, 4168, quaark
* **Runtimes**: Fix spark runtime to verify cpu request < limit, 4162, gtopper
* **Runs**: Fix list runs times out on too many runs, 4163, quaark
* **UI**: [Bug fixes](https://github.com/mlrun/ui/releases/tag/v1.5.0-rc9#bug-fixes)


Pull requests:
ce0f5672 [Docs] Change folder name from tutorial to tutorials, fix link to function hub (4185)
eb4cf70f [Datastore] Introducing the API for creating client temporary datastore profiles (4183)
24b6d817 [Tests] Fix expected mlrun executable location to conda binaries (4187)
a8aa52ee [Runtimes] Support for aborting Databricks jobs (4165)
6838266b [Tests] Fix KFP pipeline param test (4184)
b879b3db [Dockerfiles] Fix mlrun gpu build (4186)
e34350a4 [Dockerfiles] Remedy python os packages vulnerabilities (4181)
cca75db1 [BatchRuns] Improve err and default strategy (4167)
26fc24eb [Docs] Add docs for notifications secret_params (4176)
e187a710 [Dockerfiles] Update conda initial requirements (4180)
4da8b778 [Automation] Add ability to reset DB after deploy (4153)
163260a4 [Runtimes][Dask] Ensure that MLRUN_DBPATH is available for workers (4172)
78079fcc [Dockerfiles] Fix building base image (4179)
c0e188a6 [Dockerfiles] mlrun/mlrun to use miniconda + opmi (4166)
f398a50d [Notifications] Add secret_params to db (4178)
be0236fc [Notifications] Add optional to secret_params (4177)
ecc862c1 [Docs] Fix monitoring Grafana dashboards links for old igz versions (4174)
d4e0c6e2 [Notifications] Fix notifications spamming project secret - part 2 (4170)
65f9df66 [Model Monitoring] Fix histogram padding when model object has no feature stats (4169)
60a5b386 [Datastore] Add list and delete APIs for datastore profiles (4148)
a4b32c05 [Dockerfiles] Update ignoredockerfile (4173)
f20062c9 [Docs] Clarify use of CSV, JSON files for hyperparameters (4164)
16bce9de [Frameworks] Raise an exception when no framework has been found (4171)
5a497804 [Model Monitoring] add model endpoint validations when calling `get_or_create_model_endpoint()` (4160)
e7e7da8d [Notifications] Fix notifications spamming project secret - part 1 (4168)
eb5e9847 [Runtimes] Fix Spark Runtime to verify CPU request < limit (4162)
9e9f87f3 [Runs] Fix list runs times out on too many runs (4163)

Features / Enhancements
* **Dockerfiles**: Remove ml-models-gpu image, 4156, liranbg
* **Model Monitoring**: Remove drift analysis from `get_or_create_model_endpoint()`, 4135, Eyal-Danieli
* **Dask**: Increase timeouts to handle transient failures, 4154, liranbg
* **Images**: Introducing new gpu image, obsoleting models and models-gpu, 4151, liranbg
* **Secrets**: Abstract secrets provider, allow in-mem secrets for integration-tests, 4141, theSaarco
* **Docs**: V1.3.4 change log update, 4152, jillnogold
* **Model Monitoring**: Add edge bins for actual data outside of the expected distribution, 4124, jond01
* **CI**: System test updates, 4149, liranbg
* **Docs**: Function name best practice, 4142, jillnogold
* **Docs**: Update ce k8s installation instructions on apple silicon, 4144, jond01
* **Notifications**: Validate notification params size doesn't exceed 1mb, 4143, quaark
* **Unknown**: Disable explicit ack by default for v1.5.0, 4139, gtopper
* **Serving**: Limit explicit ack and respond features to applicable triggers, 4132, gtopper
* **Docs**: Update toc: move notifications under runs-workflows; ci-integration under ci-cd-a…, 4098, jillnogold
* **Requirements**: Bump storey to 1.5.5, 4133, gtopper
* **Client**: Support non-blocking project creation, 4116, liranbg
* **Requirements**: Bump storey to 1.5.4, 4129, gtopper
* **UI**: [Features & enhancement](https://github.com/mlrun/ui/releases/tag/v1.5.0-rc8#features-and-enhancements)

Bug fixes
* **Dockerfiles**: Fix mlrun-gpu security vulnerabilities, 4158, liranbg
* **Runtimes**: Fix default memory for spark applications, 4090, gtopper
* **Utils**: Fix line, 4157, yaelgen
* **Utils**: Fix http session retryable error handling, 4150, yaelgen
* **Datastore**: Fix clearing redis when the target is specified as a datastore profile, 4147, alxtkr77
* **Datastore**: Fix datastore profile name being prefixed to the redis key, 4146, alxtkr77
* **Kaniko**: Fix kaniko pod env not initialized in ecr flow, 4145, TomerShor
* **Runtimes**: Fix databricks runtime rerun, 4137, tomerm-iguazio
* **DB**: Fix migration scripts, 4140, yanburman
* **Spark**: Fix monitor runs ui url update, 4130, alonmr
* **UI**: [Bug fixes](https://github.com/mlrun/ui/releases/tag/v1.5.0-rc8#bug-fixes)


Pull requests:
e1cc243d [Dockerfiles] Fix mlrun-gpu security vulnerabilities (4158)
61933bf1 [Runtimes] Fix default memory for spark applications (4090)
6d7b6980 [Dockerfiles] Remove ml-models-gpu image (4156)
b9f8a901 [Utils] fix line (4157)
a08e48e0 [Utils] Fix HTTP session retryable error handling (4150)
377d5a1c [Model Monitoring] Remove drift analysis from `get_or_create_model_endpoint()` (4135)
56b3f6b1 [Dask] Increase timeouts to handle transient failures (4154)
a03ded24 [Images] Introducing new GPU image, obsoleting models and models-gpu (4151)
bb0e36d5 [Datastore] Fix clearing Redis when the target is specified as a datastore profile (4147)
fe86036c [Datastore] Fix datastore profile name being prefixed to the Redis key (4146)
acc968d2 [Secrets] Abstract secrets provider, allow in-mem secrets for integration-tests (4141)
f1566dde [Docs] v1.3.4 Change log update (4152)
a0b876fb [Model Monitoring] Add edge bins for actual data outside of the expected distribution (4124)
51453863 [CI] System test updates (4149)
75d4376e [Docs] function name best practice (4142)
6ce90c04 [Kaniko] Fix kaniko pod env not initialized in ECR flow (4145)
1f14030f [Docs] Update CE K8s installation instructions on Apple silicon (4144)
89b8aa68 [Notifications] Validate Notification Params Size doesn't exceed 1MB (4143)
b5e8128f [Runtimes] Fix databricks runtime rerun (4137)
75ba9c5a [DB] Fix migration scripts (4140)
c7988bb9 Disable explicit ack by default for v1.5.0 (4139)
8627f99f [Serving] Limit explicit ack and respond features to applicable triggers (4132)
37992674 [Docs] Update TOC: move notifications under runs-workflows; ci-integration under ci-cd-a… (4098)
e2e7b190 [Requirements] Bump storey to 1.5.5 (4133)
3f68331f [Client] Support non-blocking project creation (4116)
d444ea81 [Spark] Fix monitor runs ui url update (4130)
b510f56c [Requirements] Bump storey to 1.5.4 (4129)

Features / Enhancements
* **Hub**: Update default hub on data init, 4123, alonmr
* **Builder**: Improve line readability, 4117, yaelgen
* **API**: Redact unmasked run notifications param, 4114, alonmr
* **Serving**: Remove overriding of event path for non-http triggers, 4122, gtopper
* **API**: Remove redundant try clause in start function, 4119, alonmr
* **Model Monitoring**: Add abstraction and tests to the monitoring metrics, 4082, jond01
* **Runtimes**: Add databricks new cluster configurations, 4115, gtopper
* **Builder**: Remove env vars from dockerfile and change behavior for project secrets, 4107, yaelgen
* **Runtimes**: Deduct empty command in pass mode, 4110, alonmr
* **Unknown**: Revert "update dev_utilities.py (4109)", 4112, alonmr
* **Package**: Replace `mpi4py` with host name check, 4104, guy1992l
* **Unknown**: Update dev_utilities.py, 4109, eliyahu77
* **UI**: [Features & enhancement](https://github.com/mlrun/ui/releases/tag/v1.5.0-rc7#features-and-enhancements)

Bug fixes
* **Test**: Fix test store run masking, 4125, alonmr
* **Runtimes**: Fix - propagate databricks job errors, 4118, tomerm-iguazio
* **Datastore**: Fix errors introduced by last datastore-profile refactoring, 4113, alxtkr77
* **System test**: Fix install redisinsight command, 4108, eliyahu77
* **UI**: [Bug fixes](https://github.com/mlrun/ui/releases/tag/v1.5.0-rc7#bug-fixes)


Pull requests:
f0876d9c [Test] Fix test store run masking (4125)
459ca3ac [Runtimes] Fix - propagate databricks job errors (4118)
b9db5e71 [Hub] Update default hub on data init (4123)
21c86e30 [Builder] Improve line readability (4117)
ed2617bf [API] Redact unmasked run notifications param (4114)
96ea7cb4 [Serving] Remove overriding of event path for non-http triggers (4122)
3f3db0f1 [API] Remove redundant try clause in start function (4119)
965704af [Model Monitoring] Add abstraction and tests to the monitoring metrics (4082)
f257c982 [Runtimes] Add Databricks new cluster configurations (4115)
b303259f [Datastore] Fix errors introduced by last datastore-profile refactoring (4113)
8371355e [Builder] Remove env vars from Dockerfile and change behavior for project secrets (4107)
f7b6e3a7 [Runtimes] Deduct empty command in pass mode (4110)
7b9a03c6 Revert "Update dev_utilities.py (4109)" (4112)
2f8f9aa7 [Package] Replace `mpi4py` with host name check (4104)
8d81f5be Update dev_utilities.py (4109)
7f2be490 [System test] Fix install redisinsight command (4108)

Features / Enhancements
* **Runtimes**: Add databricks to runtime_with_handlers, 4103, tomerm-iguazio
* **API**: Migrate frontend spec schema to deployment mlrun requirement, 3787, alonmr
* **Model Monitoring**: Add `endpoint_store_connection` to client spec, 4092, Eyal-Danieli
* **Datastore**: A new entity for managing datastore credentials - addressing saar's comments, 4079, alxtkr77
* **Config**: Improve "redisnosql" default value, 4100, laurybueno
* **Hub**: Implement list hub sources in sqlrundb, 4095, alonmr
* **Docs**: Add docs on extra_args parameter, 4060, yaelgen
* **Requirements**: Move pymysql to api, 4096, alonmr
* **Requirements**: Bump ipython to 8.x, 4097, alonmr
* **Test**: Add databricks runtime to inheritance test, 4093, alonmr
* **Notifications**: Use bool instead of context in webhook ssl verify notifications, 4086, yaelgen
* **Tests**: Pass pytest flags on the arguments field of pycharm configuration, 4083, laurybueno
* **Requirements**: Blacklist apscheduler 3.10.2, 4085, alonmr
* **Notifications**: Add ssl verification to webhook notifications, 4076, yaelgen
* **UI**: [Features & enhancement](https://github.com/mlrun/ui/releases/tag/v1.5.0-rc6#features-and-enhancements)

Bug fixes
* **Datastore**: Fix dbfs preview, 4091, tomerm-iguazio
* **Feature Store**: Fix propagation of context object at feature set init, 4106, gtopper
* **Model Monitoring**: Fix parsing of default http target, 4094, Eyal-Danieli
* **CI**: Fix verifying is stable, 4099, liranbg
* **FeatureStore**: Fix format target path as artifact path, 4068, tomerm-iguazio
* **Build**: Fix mysql migration + add target for both migrations, 4027, yanburman
* **frontend-spec**: Fix redis' default path for the feature store, 4089, laurybueno
* **Build**: Fix auto mount overriding access key, 4087, alonmr
* **UI**: [Bug fixes](https://github.com/mlrun/ui/releases/tag/v1.5.0-rc6#bug-fixes)


Pull requests:
be0afaed [Datastore] Fix dbfs preview (4091)
be1b9570 [Runtimes] Add databricks to runtime_with_handlers (4103)
6d97bfba [Feature Store] Fix propagation of context object at feature set init (4106)
de3b22f3 [API] Migrate frontend spec schema to deployment mlrun requirement (3787)
48a7a2a5 [Model Monitoring] Add `endpoint_store_connection` to client spec (4092)
cee34ff5 [Datastore] A new entity for managing datastore credentials - addressing Saar's comments (4079)
d84bd20e [Config] Improve "redisnosql" default value (4100)
e4ca358f [Hub] Implement list hub sources in SQLRunDB (4095)
6623a6af [Model Monitoring] Fix parsing of default http target (4094)
65bb5e77 [CI] Fix verifying is stable (4099)
e43787b1 [FeatureStore] Fix format target path as artifact path (4068)
86c382e2 [Build] Fix mysql migration + add target for both migrations (4027)
e0d32513 [Docs] Add docs on extra_args parameter (4060)
08a93f03 [Requirements] Move pymysql to API (4096)
e763db5b [Requirements] Bump ipython to 8.x (4097)
65ec7f4e [Test] Add Databricks Runtime to inheritance test (4093)
18fa1ab9 [frontend-spec] Fix Redis' default path for the Feature Store (4089)
be3151f6 [Build] Fix auto mount overriding access key (4087)
6c9ba72f [Notifications] Use bool instead of context in webhook ssl verify notifications (4086)
6ceb24c9 [Tests] Pass Pytest flags on the Arguments field of Pycharm configuration (4083)
e5000113 [Requirements] Blacklist APScheduler 3.10.2 (4085)
dbe7a3eb [Notifications] Add ssl verification to webhook notifications (4076)