Lithops

Added
- [Lithops] Added support for Python 3.12
- [CLI] Added "--include" parameter in "lithops image build" to allow users upload local files to the VM image

Changed
- [Standalone] Use redis in the master VM to store all the relevant data about jobs and workers
- [Standalone] Use redis to store the work queues
- [Standalone] Improved resiliency and worker granularity
- [CLI] Show the timestamp in the local timezone format on "lithops job list"
- [CLI] Show worker creation timestamp and time-to-dismantle on "lithops worker list"

Fixed
- [SSH Cli] Fixed minor error with the "err" variable
- [Cli] Fixed job status on "lithops job list" for standalone backends
- [Standalone] Fixed issue in the "lithops image build" that appears when the vpc is already created
- [Future] Fixed issue with missing 'worker_end_tstamp' variable

Added
- [Plots] Allow to set the figure size of the plots
- [Stats] Added new CPU, Memory and Network statistics in the function results
- [IBM VPC] Added a new parameter to enable/disable resource existence check in the platform

Changed
- [Config] Renamed 'customized_runtime' to 'runtime_include_function'
- [IBM VPC] Increased the total number of available IPs in the private network
- [Standalone] Do not stop the VM immediately after a job in the Consume mode

Fixed
- [Standalone] Fixed issue that appears when the invocation payload is too big
- [Invoker] Fixed "runtime_include_function" function/modules path
- [AWS EC2] Reset the public IP address of the master VM on stop

Added
- [k8s] Added a new way of invoking functions using a RabbitMQ work queue
- [IBM VPC] Added "zone" config parameter
- [IBM Code Engine] Get and print an error message in case of container execution failure

Changed
- [OpenWhisk] Updated default runtimes

Fixed
- [Standalone] Fixed issue with a wrong value of "chunksize"
- [IBM Code Engine] Fixed missing parameter on clean
- [Executor] Fixed potential deadlock in wait() and get_result() when an exception is produced in a function activation

Added
- [Cli] Added new 'lithops image delete' command for standalone backends
- [Cli] Added new 'lithops job list' command for standalone backends
- [Cli] Added new 'lithops worker list' command for standalone backends
- [AWS EC2] Added delete_image() method for deleting VM images through the cli
- [IBM VPC] Added delete_image() method for deleting VM images through the cli
- [localhost] New localhost backend v2 to maximize resource utilization when multiple maps are executed from the same FunctionExecutor
- [Standalone] Automatically retrieve the CPU_COUNT from the VM in case worker_processes is not set in config
- [Standalone] Keep track of the worker and job status
- [Storage] Include "Config" parameter to download_file() and upload_file() methods for boto3 related backends
- [Cli] Include 'worker name' in the 'lithops runtime list' cmd
- [AWS Lambda] Created 'namespace' config key to virtually separate worker deployments

Changed
- [Standalone] Changed default mode of execution from 'consume' to 'reuse'
- [Joblib] Updated the joblib backend to make it compatible with new versions of joblib
- [Joblib] Spawn only one function when 'prefer' is set to 'threads'
- [AWS EC2] Changed default image name from "lithops-worker-default" to "lithops-ubuntu-jammy-22.04-amd64-server"
- [IBM VPC] Changed default image name from "lithops-worker-default" to "lithops-ubuntu-22-04-3-minimal-amd64-1"
- [Serializer] Improve serializer performance when include_modules is set in config
- [SSH Client] Do not raise LithopsValidationError on Authentication failed
- [AWS Lambda] Renamed function name to "lithops-worker-xxxx"

Fixed
- [Job] Fixed max data size in the invocation payload
- [Multiprocessing] Fixed cpu_count
- [Standalone] Start new workers when the VM instance type changes (in reuse mode)
- [GCP Functions] Fixed issue with "function_url" variable
- [Standalone] Fixed multiple runtime usage at the same time in master VM
- [localhost] Get the correct docker/podman path for jobs that run in a container
- [k8s] Limit the size of the "user" label as the maximum allowed is 63 chars
- [Joblib] Fix shared objects utility when multiple maps run from the same executor
- [Azure VMs] Fix wrong exception when trying to connect to the master VM for the first time
- [Partitioner] Fix partitioner

New
- [OCI Functions] Added new 'Oracle Cloud Functions' serverless backend
- [OCI Object Storage] Added new 'Oracle Cloud Object storage' storage backend

Added
- [Kubernetes] Added Redis server in master pod for shared data between workers
- [Kubernetes] Allow to set "context" and "namespace" in lithops config

Changed
- [CodeEngine] Create the CE project only when necessary instead of creating it always
- [IBM CF] Create the CF namespace only when necessary instead of creating it always

Fixed
- [Executor] Fixed kwargs mapping in ServerlessExecutor and StandaloneExecutor
- [Serializer] Fixed issue in serializer when "include_modules" config var is set
- [CodeEngine] Fixed exception handling

New
- [Azure Virtual Machines] Added new 'Azure Virtual Machines' standalone backend

Added
- [Serverless] Added support for python 3.10 and 3.11 runtimes
- [Executor] Allow to set all the compute backend params programmatically in the FunctionExecutor()
- [AWS EC2] Allow to automatically create the VPC and all the necessary resources
- [IBM VPC & AWS EC2] General fixes and Improvements
- [Executor] Allow to pass the config file location in the FunctionExecutor()
- [Storage] Automatically create the storage bucket if not provided in the config
- [IBM] Allow to set "region" under "ibm" section
- [AWS] Allow to set "region" under "aws" section
- [Cli] Added new 'lithops image build' command for standalone backends
- [Cli] Added new 'lithops image list' command for standalone backends
- [IBM VPC] Added build_image() method for automatically building VM images
- [IBM VPC] Added list_image() method for listing the available VM images
- [AWS EC2] Added build_image() method for automatically building VM images
- [AWS EC2] Added list_image() method for listing the available VM images
- [Azure VMS] Added list_image() method for listing the available VM images
- [IBM CF] Automatically create a CF namespace if not provided in config
- [IBM VPC] Added Madrid (Spain) region
- [Code Engine] Automatically create a new project if not provided in config

Changed
- [Azure] Changed 'location' config parameter to 'region' for compatibility
- [Aliyun] Changed 'endpoint' config parameter to 'region' for compatibility
- [AWS EC2] Reduced number of mandatory parameters
- [AWS] Allow 'region' config parameter instead of 'region_name' for compatibility
- [IBM CF] Cloud-foundry namespaces have been deprecated in IBM Cloud. From now all the users must use an IAM-based namespace
- [IBM COS] Changed 'access_key' config parameter to 'access_key_id' for compatibility
- [IBM COS] Changed 'secret_key' config parameter to 'secret_access_key' for compatibility
- [IBM] Improved token manager
- [Core] Job creation now checks that each element in 'iterdata' is smaller than 8 KB
- [MapReduce] Make 'results' variable name not mandatory in the reduce function signature
- [CLI] Renamed 'lithops test' command to 'lithops hello'
- [CLI] Renamed 'lithops verify' command to 'lithops test'

Fixed
- [IBM VPC & AWS EC2] Make sure only VMs from the given VPC are removed
- [IBM VPC] Reuse floating IPs for public gateways
- [Serializer] Prevent analyzing the same module multiple times
- [SSH Cli] Fix issue with RSA keys
- [Tests] Fix broken links of testing files
- [Azure Container APPs] Make sure the lithops worker app is deployed
- [AWS Lambda] Fixed error with urllib3. Pin urllib3 to <2 as for now botocore does not support urllib3 2.0
- [Multiprocessing] Check redis connection before starting to submit jobs
- [Redis] Fixed redis backend exception regarding storage_bucket