Hopsworks

[HWORKS-471](https://hopsworks.atlassian.net/browse/HWORKS-471) Remove usage of docker/dockerfile:experimental

[HWORKS-472](https://hopsworks.atlassian.net/browse/HWORKS-472) Workflows/load tests python dependencies should be defined in a requirements.txt

[HWORKS-474](https://hopsworks.atlassian.net/browse/HWORKS-474) Remove hdfs\_user foreign key from JupyterProject

[HWORKS-476](https://hopsworks.atlassian.net/browse/HWORKS-476) Remove hdfs\_user\_id foreign key from TensorBoard

[HWORKS-478](https://hopsworks.atlassian.net/browse/HWORKS-478) Add webhook configuration support for alertmanager

[HWORKS-480](https://hopsworks.atlassian.net/browse/HWORKS-480) Remove inode foreign key from storage connectors

[HWORKS-485](https://hopsworks.atlassian.net/browse/HWORKS-485) Do not create linux user/group in git container

[HWORKS-488](https://hopsworks.atlassian.net/browse/HWORKS-488) Add support for storing docker registry artifacts on S3

[HWORKS-497](https://hopsworks.atlassian.net/browse/HWORKS-497) Remove homebew dependency from java cookbook

[HWORKS-502](https://hopsworks.atlassian.net/browse/HWORKS-502) Add silence for CVE-2023-28462

[HWORKS-505](https://hopsworks.atlassian.net/browse/HWORKS-505) Upgrade GPU components to support Cgroups v2/Rhel8/Ubuntu22.04

[FSTORE-432](https://hopsworks.atlassian.net/browse/FSTORE-432) Fix event\_time in hopsworks-tutorial

[FSTORE-435](https://hopsworks.atlassian.net/browse/FSTORE-435) Fetching report which have results with deleted expectation throws null pointer exception.

[FSTORE-436](https://hopsworks.atlassian.net/browse/FSTORE-436) HSFS Java doesn't serialize JSONs correctly

[FSTORE-437](https://hopsworks.atlassian.net/browse/FSTORE-437) Key error 'type' in inode.from\_response\_json

[FSTORE-438](https://hopsworks.atlassian.net/browse/FSTORE-438) Methods get\_train\_validation\_test\_split and train\_validation\_test\_split not returning same column type

[FSTORE-443](https://hopsworks.atlassian.net/browse/FSTORE-443) Do not allow sending feature group feature offline type to be null

[FSTORE-444](https://hopsworks.atlassian.net/browse/FSTORE-444) Data validation minor bug fixes for python client

[FSTORE-445](https://hopsworks.atlassian.net/browse/FSTORE-445) Minor fix to tutorials

[FSTORE-453](https://hopsworks.atlassian.net/browse/FSTORE-453) Fix convert\_event\_time\_to\_timestamp for datetimes with millisecond precision \(hudi timestamps\)

[FSTORE-454](https://hopsworks.atlassian.net/browse/FSTORE-454) Show warning in output when data validation fails, right now it silently moves on

[FSTORE-455](https://hopsworks.atlassian.net/browse/FSTORE-455) Default great expectation configuration not working

[FSTORE-456](https://hopsworks.atlassian.net/browse/FSTORE-456) Validation time in the future

[FSTORE-457](https://hopsworks.atlassian.net/browse/FSTORE-457) Data Validation code example is broken

[FSTORE-459](https://hopsworks.atlassian.net/browse/FSTORE-459) Wrong FS name in API example for sharing FG

[FSTORE-461](https://hopsworks.atlassian.net/browse/FSTORE-461) Append feature default value not working for online feature store

[FSTORE-463](https://hopsworks.atlassian.net/browse/FSTORE-463) Import data to feature group job failure

[FSTORE-465](https://hopsworks.atlassian.net/browse/FSTORE-465) Can't assign filter to feature view query

[FSTORE-468](https://hopsworks.atlassian.net/browse/FSTORE-468) Cannot set timestamp type as partition\_key in feature group

[FSTORE-476](https://hopsworks.atlassian.net/browse/FSTORE-476) Schema not updated for hsfs java client after append

[FSTORE-480](https://hopsworks.atlassian.net/browse/FSTORE-480) Create new Dynamic Object for each row in transaction

[FSTORE-501](https://hopsworks.atlassian.net/browse/FSTORE-501) Can't insert streaming dataframe in streaming feature group using \`insert\_stream\` method

[FSTORE-507](https://hopsworks.atlassian.net/browse/FSTORE-507) figure out features and their transformation functions when attaching transformation functions with prefixes

[FSTORE-508](https://hopsworks.atlassian.net/browse/FSTORE-508) NPE when creating feature view with labels

[FSTORE-512](https://hopsworks.atlassian.net/browse/FSTORE-512) Typo in expectation\_type leads to failure to execute validate job

[FSTORE-518](https://hopsworks.atlassian.net/browse/FSTORE-518) Problem with Feature Groups Join

[FSTORE-520](https://hopsworks.atlassian.net/browse/FSTORE-520) Feature view - explicit provenance if parent feature group is deleted

[FSTORE-522](https://hopsworks.atlassian.net/browse/FSTORE-522) HSFS get redshift storage connector fails

[FSTORE-527](https://hopsworks.atlassian.net/browse/FSTORE-527) Fix deletion issue with expectation suite due to activity table constrain

[FSTORE-532](https://hopsworks.atlassian.net/browse/FSTORE-532) Redshift connector NPE when using Instance Role for authentication

[FSTORE-534](https://hopsworks.atlassian.net/browse/FSTORE-534) Redshift Connector IAM Role authentication jackson issue with expiration field

[FSTORE-538](https://hopsworks.atlassian.net/browse/FSTORE-538) Redshift connector read throws validation error for query argument inspite of table name specified on connector

[FSTORE-551](https://hopsworks.atlassian.net/browse/FSTORE-551) Fix loadtest incorrect assert

[FSTORE-552](https://hopsworks.atlassian.net/browse/FSTORE-552) Fix validate for pyspark when providing pandas dataframe

[FSTORE-560](https://hopsworks.atlassian.net/browse/FSTORE-560) In onlinefs configuration, Hopsworks domain name is not configured by chef

[FSTORE-639](https://hopsworks.atlassian.net/browse/FSTORE-639) Pin SQLalchemy version or upgrade to new 2.0.0 version

[FSTORE-640](https://hopsworks.atlassian.net/browse/FSTORE-640) Fix QuickStart and Credit\_Scores tutorials

[FSTORE-653](https://hopsworks.atlassian.net/browse/FSTORE-653) get\_feature\_vector is broken

[FSTORE-657](https://hopsworks.atlassian.net/browse/FSTORE-657) in dev branch tutorials imports are missing

[FSTORE-660](https://hopsworks.atlassian.net/browse/FSTORE-660) Docs: Syntax issue for creating external feature group

[FSTORE-661](https://hopsworks.atlassian.net/browse/FSTORE-661) Creating external feature group in Jupyter Notebook didn't work

[FSTORE-663](https://hopsworks.atlassian.net/browse/FSTORE-663) Model training tutorial:Error on predictions after model deployment

[FSTORE-682](https://hopsworks.atlassian.net/browse/FSTORE-682) Fix fg save\_expectation\_suite

[FSTORE-689](https://hopsworks.atlassian.net/browse/FSTORE-689) Shallow dataframe copy hides TZ bug

[FSTORE-691](https://hopsworks.atlassian.net/browse/FSTORE-691) Train test splits contain NaN column values

[FSTORE-693](https://hopsworks.atlassian.net/browse/FSTORE-693) Insert with validation suite throws date format exception

[FSTORE-730](https://hopsworks.atlassian.net/browse/FSTORE-730) Fix Multiple backfill job started in dataval loadtest

[HWORKS-749](https://hopsworks.atlassian.net/browse/HWORKS-749) Remove chef-ulimit dependency from hopsworks-chef Berksfile

[HWORKS-752](https://hopsworks.atlassian.net/browse/HWORKS-752) Flink workflow tests should get latest release candidate dynamically


Feature Store

Epic

[FSTORE-709](https://hopsworks.atlassian.net/browse/FSTORE-709) Rest API to retrieve data from the online feature store

Bug

[FSTORE-728](https://hopsworks.atlassian.net/browse/FSTORE-728) Improve error msgs when wrong API key provided for hopsworks.login\(\)

[FSTORE-824](https://hopsworks.atlassian.net/browse/FSTORE-824) Shows undefined in accept shared feature store prompt

[FSTORE-842](https://hopsworks.atlassian.net/browse/FSTORE-842) Setting fg.insert\(storage="online"\) adds data to offline as well

[FSTORE-857](https://hopsworks.atlassian.net/browse/FSTORE-857) Appending to an online feature group fails after upgrade

[FSTORE-879](https://hopsworks.atlassian.net/browse/FSTORE-879) Fix consecutive slashes \(//\) in file selector component

[FSTORE-906](https://hopsworks.atlassian.net/browse/FSTORE-906) Training data shouldn't be recreated when recomputing stats

[FSTORE-907](https://hopsworks.atlassian.net/browse/FSTORE-907) Recomputing statistics on feature groups without time-travel enabled fails

[FSTORE-921](https://hopsworks.atlassian.net/browse/FSTORE-921) Resolve the set of serving key in get\_feature\_vector

[FSTORE-928](https://hopsworks.atlassian.net/browse/FSTORE-928) When hitting limit of number of projects that one user can create, deleting a project doesn't work as expected

[FSTORE-942](https://hopsworks.atlassian.net/browse/FSTORE-942) ADLS connector path issues

[FSTORE-946](https://hopsworks.atlassian.net/browse/FSTORE-946) Orphan statistics are not removed from the DB

[FSTORE-965](https://hopsworks.atlassian.net/browse/FSTORE-965) fastavro fails to build when installing HSFS

[FSTORE-966](https://hopsworks.atlassian.net/browse/FSTORE-966) Removing spark options on update does not work on bigquery connector

[FSTORE-969](https://hopsworks.atlassian.net/browse/FSTORE-969) Scheduled jobs are not triggering when expected

[FSTORE-993](https://hopsworks.atlassian.net/browse/FSTORE-993) HSFS Java should use UTF-8 when creating the StringEntity

[FSTORE-999](https://hopsworks.atlassian.net/browse/FSTORE-999) Cannot get feature vector from a shared feature view

[FSTORE-1012](https://hopsworks.atlassian.net/browse/FSTORE-1012) Hopsworks-api got an unexpected keyword argument 'feature\_store\_topic'

[FSTORE-1016](https://hopsworks.atlassian.net/browse/FSTORE-1016) Onlinefs schema issue

Task

[FSTORE-50](https://hopsworks.atlassian.net/browse/FSTORE-50) Tutorial how to write custom transformation function for feature view

[FSTORE-449](https://hopsworks.atlassian.net/browse/FSTORE-449) Use Caffeine cache for schemas

[FSTORE-576](https://hopsworks.atlassian.net/browse/FSTORE-576) Tutorial for transformation functions in sklearn

[FSTORE-654](https://hopsworks.atlassian.net/browse/FSTORE-654) Should throw proper error if data format is not provided.

[FSTORE-672](https://hopsworks.atlassian.net/browse/FSTORE-672) Add shared feature store workflows tests

[FSTORE-742](https://hopsworks.atlassian.net/browse/FSTORE-742) Statistics API

[FSTORE-803](https://hopsworks.atlassian.net/browse/FSTORE-803) Beam support for HSFS java client

[FSTORE-839](https://hopsworks.atlassian.net/browse/FSTORE-839) Single Kafka topic per project

[FSTORE-840](https://hopsworks.atlassian.net/browse/FSTORE-840) Use existing Kafka cluster

[FSTORE-860](https://hopsworks.atlassian.net/browse/FSTORE-860) EJB timer to delete orphan statistics

[FSTORE-862](https://hopsworks.atlassian.net/browse/FSTORE-862) Allow naming of Kafka topics for data ingestion

[FSTORE-865](https://hopsworks.atlassian.net/browse/FSTORE-865) Update online feature store logs

[FSTORE-866](https://hopsworks.atlassian.net/browse/FSTORE-866) Add option to deprecate a feature group

[FSTORE-877](https://hopsworks.atlassian.net/browse/FSTORE-877) Create Training Datasets in Flyingduck

[FSTORE-884](https://hopsworks.atlassian.net/browse/FSTORE-884) Flying Duck - Write End-User Documentation

[FSTORE-887](https://hopsworks.atlassian.net/browse/FSTORE-887) Compute statistics on both untransformed and transformed data when creating a Training Dataset

[HWORKS-368](https://hopsworks.atlassian.net/browse/HWORKS-368) Fix serving tests

[HWORKS-372](https://hopsworks.atlassian.net/browse/HWORKS-372) explicit provenance missing community stereotype

[HWORKS-379](https://hopsworks.atlassian.net/browse/HWORKS-379) LDAP users cannot download certificates

[HWORKS-380](https://hopsworks.atlassian.net/browse/HWORKS-380) PKI.java class contains resource leaks

[HWORKS-381](https://hopsworks.atlassian.net/browse/HWORKS-381) hopsworks-api connection.close\(\) should clean up hsfs and hsml connections

[HWORKS-382](https://hopsworks.atlassian.net/browse/HWORKS-382) Provenance input links are missing for models trained on python kernels

[HWORKS-390](https://hopsworks.atlassian.net/browse/HWORKS-390) app\_id field in execution table is too short

[HWORKS-407](https://hopsworks.atlassian.net/browse/HWORKS-407) Setting the maximum number of Projects per user to 0 let the user create unlimited Projects

[HWORKS-410](https://hopsworks.atlassian.net/browse/HWORKS-410) Pin jupyter-console in Sparkmagic

[HWORKS-414](https://hopsworks.atlassian.net/browse/HWORKS-414) dockerfile:experimental should be pushed to our docker registry

[HWORKS-425](https://hopsworks.atlassian.net/browse/HWORKS-425) Update Grafana to address CVE-2022-39328, CVE-2022-39307 and CVE-2022-39306

[HWORKS-431](https://hopsworks.atlassian.net/browse/HWORKS-431) Git image version shoud not be hardcoded in Hopsworks settings

[HWORKS-444](https://hopsworks.atlassian.net/browse/HWORKS-444) Log aggregation reports incorrect failed status under load

[HWORKS-445](https://hopsworks.atlassian.net/browse/HWORKS-445) Increase polling time for log aggregation to finish in execution.wait\_until\_finished\(\)

[HWORKS-481](https://hopsworks.atlassian.net/browse/HWORKS-481) hopsworks-api create\_job fails on app\_path missing for flink jobs

[HWORKS-483](https://hopsworks.atlassian.net/browse/HWORKS-483) Add CREATING status for deployments

[HWORKS-484](https://hopsworks.atlassian.net/browse/HWORKS-484) Consecutive python uninstall commands uses incorrect tag

[HWORKS-507](https://hopsworks.atlassian.net/browse/HWORKS-507) kagent\_param for kube hopsmon certificate break the deployment

[HWORKS-524](https://hopsworks.atlassian.net/browse/HWORKS-524) HSML ignores knative\_domain parameter

Subtask

[HWORKS-464](https://hopsworks.atlassian.net/browse/HWORKS-464) Update Python job documentation to reflect the fact that a K8s cluster is needed

[HWORKS-467](https://hopsworks.atlassian.net/browse/HWORKS-467) Disable SSL/TLS Diffie-Hellman keys less that 2048 bits

Hopsworks

Bug

[HWORKS-224](https://hopsworks.atlassian.net/browse/HWORKS-224) hopsworks python SDK opensearch link broken

[HWORKS-853](https://hopsworks.atlassian.net/browse/HWORKS-853) Model export fails in integration tests

[HWORKS-862](https://hopsworks.atlassian.net/browse/HWORKS-862) Remote clients are not available on non das nodes

[HWORKS-864](https://hopsworks.atlassian.net/browse/HWORKS-864) Istio doesn't use the docker images from the registry

[HWORKS-865](https://hopsworks.atlassian.net/browse/HWORKS-865) NullPointerException when monitoring execution of a deleted job

[HWORKS-869](https://hopsworks.atlassian.net/browse/HWORKS-869) Email validation regex doesn't support capital letters

[HWORKS-872](https://hopsworks.atlassian.net/browse/HWORKS-872) Cannot disable oauth group mapping from cluster definition

[HWORKS-878](https://hopsworks.atlassian.net/browse/HWORKS-878) Grafana uses wrong prometheus name

[HWORKS-879](https://hopsworks.atlassian.net/browse/HWORKS-879) Upload will fail if the same file is uploaded again while the first upload is ongoing.

[HWORKS-881](https://hopsworks.atlassian.net/browse/HWORKS-881) Validate SparkJob and Python job application file exists before launching job

[HWORKS-882](https://hopsworks.atlassian.net/browse/HWORKS-882) Disallow exporting a model in the root path of a dataset

[HWORKS-890](https://hopsworks.atlassian.net/browse/HWORKS-890) Restoring RonDB backup breaks because restore-data does not allow restoring in a table with unique indexes

[HWORKS-898](https://hopsworks.atlassian.net/browse/HWORKS-898) PyTorch installation does not have GPU support

[HWORKS-899](https://hopsworks.atlassian.net/browse/HWORKS-899) Broken image links in gh-pages documentation

[HWORKS-901](https://hopsworks.atlassian.net/browse/HWORKS-901) Git installation support should not prepend git\+

[HWORKS-907](https://hopsworks.atlassian.net/browse/HWORKS-907) CloudManager should blacklist all instances from automatic removal

[HWORKS-908](https://hopsworks.atlassian.net/browse/HWORKS-908) NPE in ProjectQouataController for a project that failed to get created

[HWORKS-910](https://hopsworks.atlassian.net/browse/HWORKS-910) Endpoint identification alg missing from kafka get\_default\_config\(\)

[HWORKS-950](https://hopsworks.atlassian.net/browse/HWORKS-950) git pull fails if the user has multiple surnames

[HWORKS-954](https://hopsworks.atlassian.net/browse/HWORKS-954) api\_key for cert-operator is created empty

[HWORKS-961](https://hopsworks.atlassian.net/browse/HWORKS-961) Do not turn on MySQL binlog if global replication is not enabled

[HWORKS-974](https://hopsworks.atlassian.net/browse/HWORKS-974) Resource guard in ndb-chef for extracting RonDB does not work

[HWORKS-975](https://hopsworks.atlassian.net/browse/HWORKS-975) OpenSearchApi in hopsworks-api should not rely on ELASTIC\_ENDPOINT

[HWORKS-976](https://hopsworks.atlassian.net/browse/HWORKS-976) Replacement of CREATE USER with CREATE USER IF NOT EXISTS is wrong in RonDB backup restore script

[HWORKS-996](https://hopsworks.atlassian.net/browse/HWORKS-996) Removal of rotated one time JWT signing key should happen only on the Primary node

[HWORKS-1001](https://hopsworks.atlassian.net/browse/HWORKS-1001) Feature store sharing should also share data validation dataset

[HWORKS-1029](https://hopsworks.atlassian.net/browse/HWORKS-1029) Pin nbconvert in docker images

Task

[HWORKS-135](https://hopsworks.atlassian.net/browse/HWORKS-135) Models backend should store metadata in tables instead of opensearch

[HWORKS-302](https://hopsworks.atlassian.net/browse/HWORKS-302) Enhancement Request: Ability to disable access to Anaconda package repository and hide the functionality at a system defined level

[HWORKS-707](https://hopsworks.atlassian.net/browse/HWORKS-707) Store descriptive statistics in NDB instead of HopsFS

[HWORKS-802](https://hopsworks.atlassian.net/browse/HWORKS-802) Python 3.12 support for clients

[HWORKS-808](https://hopsworks.atlassian.net/browse/HWORKS-808) add database/url to parameter in kafka cookbook

[HWORKS-854](https://hopsworks.atlassian.net/browse/HWORKS-854) hops-system should use the minimal env definition without pydoop

[HWORKS-855](https://hopsworks.atlassian.net/browse/HWORKS-855) Remove Kagent ability to do service key rotation

[HWORKS-861](https://hopsworks.atlassian.net/browse/HWORKS-861) Disable Kafka alerting rules when bring your own kafka is enabled

[HWORKS-868](https://hopsworks.atlassian.net/browse/HWORKS-868) Increase yarn.resourcemanager.rmappsecurity.jwt.validity to 1 hour

[HWORKS-871](https://hopsworks.atlassian.net/browse/HWORKS-871) Hopsworks should be able to handle oauth claims in array format

[HWORKS-873](https://hopsworks.atlassian.net/browse/HWORKS-873) Add flag to delete RonDB backup with the same backup-id

[HWORKS-877](https://hopsworks.atlassian.net/browse/HWORKS-877) Helm chart for certs-operator

[HWORKS-880](https://hopsworks.atlassian.net/browse/HWORKS-880) Model registry tests should account for if model is stored on hdfs and also run in PySpark

[HWORKS-883](https://hopsworks.atlassian.net/browse/HWORKS-883) Fix Sonarqube issues in CommandsController

[HWORKS-886](https://hopsworks.atlassian.net/browse/HWORKS-886) OnlineFS should be able to read Kafka configuration from environment variables

[HWORKS-887](https://hopsworks.atlassian.net/browse/HWORKS-887) certs-operator deploy appropriate certificate for Strimzi operator

[HWORKS-889](https://hopsworks.atlassian.net/browse/HWORKS-889) Chef flag to disable unattended upgrades

[HWORKS-891](https://hopsworks.atlassian.net/browse/HWORKS-891) Add K8s GPUs monitoring to Grafana/Jupyter/Job UI

[HWORKS-892](https://hopsworks.atlassian.net/browse/HWORKS-892) Use a single YarnClientWrapper to monitor all jobs

[HWORKS-893](https://hopsworks.atlassian.net/browse/HWORKS-893) Nightly tests should run without installing the requirements.txt

[HWORKS-894](https://hopsworks.atlassian.net/browse/HWORKS-894) Add pytest to python base environment