Dirac

*Core

NEW: Basic support for the OAuth/OIDC tokens
NEW: (6299) Experimental support for Python 3.10
NEW: (6017) TornadoBaseRequestHandler define a class logger with an extra attribute "tornadoComponent"
NEW: (6017) DIRACDB can now use a parentLogger instead of direct gLogger
NEW: (6335) DIRAC.initialize now provides keyword arguments to customise initialisation
NEW: (6335) SecurityProperty enum for expressing security properties which can be assigned to groups/hosts
NEW: (6335) LogLevel enum for choosing the logging level
NEW: (6309) Introduce DReturnType/DOKReturnType/DErrorReturnType for DIRAC return structures
NEW: (6347) Introduce ``DIRAC_MYSQL_OPTIMIZER_TRACES_PATH`` environment variable for advance optimization of MySQL calls
NEW: (6094) option op_type in index API for Elasticsearch
NEW: (6299) Suppress InsecureRequestWarning when using dirac-configure with HTTPS servers
NEW: (6299) Support overriding ciphers and methods with environment variables when using HTTPS
NEW: (6291) adding general flag for disabling Monitoring for Agents (used by JobAgent)
CHANGE: (5872) Minimum Python version is now 3.9
CHANGE: (5883) Show callstack in service logs when returning S_ERROR
CHANGE: (5764) use Client instead of RPCClient
CHANGE: (5698) Replacing Elasticsearch python client with OpenSearch.
CHANGE: (5571) allow DIRAC AS return proxy by default
CHANGE: (6188) Propagate original proxy strength when generating new proxies
CHANGE: (6281) Removed Core.Utilities.MemStat module (use psutil module instead)
CHANGE: (6281) Removed possibility to Register a DISET service with a secondary name
CHANGE: (6347) calls to the MySQL class with keyword parameters MUST now be named
CHANGE: (6297) Use psutil.net_if_addrs for Network.discoverInterfaces
CHANGE: (6286) Default proxy length is 2048 bits
CHANGE: (6286) remove RFC argument for proxy generation
CHANGE: (6091) changed name and simplified repetitive methods of Utilities.TIme
CHANGE: (6017) services propagate their local loggers to the DB
CHANGE: (6017) tornado services use local logger
CHANGE: (5854) Treat X509 data as str instead of bytes
FIX: (5743) removed isPy3VersionNumber function
FIX: (5762) Replace DIRAC.Core.DISET.TransferClient with DIRAC.Core.Tornado.Client.ClientSelector
FIX: (6151) Replace deprecated threading.activeCount with active_count
FIX: (6149) use AnyThreadEventLoopPolicy asyncio policy
FIX: (6144) Removing time dependence of TimeUtilities Module
FIX: (5870) allow to use `deprecated` decorator with classes without predefined `__init__` function.


*DataManagementSystem

NEW: (5927) TornadoDataIntegrityHandler
CHANGE: (5391) The LcgFileCatalogClient has been removed

*FrameworkSystem

NEW: (6248) Add StdoutJsonBackend
NEW: (5901) move ProxyManager to HTTPs
NEW: (5899) dirac-login (AS) added VOMS extension
NEW: (5896) dirac-login try to connect to DIRAC AS if no local certificate
NEW: (5862) add password mask to dirac-login
CHANGE: (5965) removed gMonitor
CHANGE: (5865) remove unused 'rfc' argument
CHANGE: (5857) use DIRAC CAs location to request DIRAC AS

*MonitoringSystem

NEW: (5974) TornadoMonitoring handler
NEW: (5999) Added a PilotsHistory Monitoring
NEW: (5807) Move DataOperation to Monitoring
NEW: (5634) JSON file for WMS monitoring dashboard
CHANGE: (5788) moving pilot submission to monitoring
CHANGE: (6002) Replaced the flags for monitoring with a new unified flag system
CHANGE: (5944) Divided ComponentMonitoring into AgentMonitoring and ServiceMonitoring
FIX: (6286) ElasticSearchDB generates full index names based on UTC
FIX: (6286) DataOperationSender works on a copy of the documents
FIX: (6076) Removing multiplication in timestamp when committing to monitoring

*RequestManagementSystem

FIX: (5960) moved REA initialization inside the initialize() method

*Resources

NEW: (5996) Simple stomp interface
NEW: (6061) add SRM+HTTPs as a default TPC protocol in SRM
CHANGE: (6283) GFA2_SRM2 does not use anymore StorageElements/DefaultProtocols option in the CS (relies on specific OutputProtocols config)
CHANGE: (6283) explicit distinction between PluginName and ProtocolSection name in all the StorageElement components
CHANGE: (6283) API change for the StorageElement. See https://github.com/DIRACGrid/DIRAC/pull/6283 for details
CHANGE: (6283) StorageElement favors local plugins

*ResourceStatusSystem

NEW: (5830) Added tornado version for the three RSS services

*TransformationSystem

NEW: (5829) Added TornadoTransformationManager service
FIX: (5822) TransformationDB: removed old compatibility layer with mySAM tables

*WorkloadManagementSystem

NEW: (5925) added TornadoWMSAdministratorHandler
NEW: (6134) adding a test to show the modifications made to the JDL before being inserted in the JobDB
NEW: (5714) added WMSDashboard and diracLogsDashboard in NDJSON format.
CHANGE: (6336) Remove CPU(MHz) from CacheSize(kB) job parameters
FIX: (5714) removed old index patterns in WMSDashboard.json, updated documentation.

*docs

NEW: (5867) Added general service configuration options description including Authorization options
NEW: (5540) add documentation with information about new system APIs component
CHANGE: (5573) move the ConfReference DIRAC section part to dirac.cfg and add there OAuth 2 settings.
FIX: (5572) update nginx installation instruction

[integration]

*Core

NEW: (6347) Introduce ``DIRAC_MYSQL_OPTIMIZER_TRACES_PATH`` environment variable for advance optimization of MySQL calls
CHANGE: (6347) calls to the MySQL class with keyword parameters MUST now be named
FIX: (6345) Setting codeProperties["version"] in AgentModule

[integration]

*Core

FIX: (6341) createClient now creates subclasses of the original class
NEW: (6335) DIRAC.initialize now provides keyword arguments to customise initialisation
NEW: (6335) SecurityProperty enum for expressing security properties which can be assigned to groups/hosts
NEW: (6335) LogLevel enum for choosing the logging level
NEW: (6309) Introduce DReturnType/DOKReturnType/DErrorReturnType for DIRAC return structures
CHANGE: (6299) Set OpenSSL ciphers to "DEFAULT:SECLEVEL=1"
NEW: (6299) Experimental support for Python 3.10
NEW: (6299) Suppress InsecureRequestWarning when using dirac-configure with HTTPS servers
NEW: (6299) Support overriding ciphers and methods with environment variables when using HTTPS
CHANGE: (6298) Replace suds-jurko with suds
CHANGE: (6297) Use psutil.net_if_addrs for Network.discoverInterfaces
NEW: (6291) adding general flag for disabling Monitoring for Agents (used by JobAgent)
CHANGE: (6286) Default proxy length is 2048 bits
CHANGE: (6286) remove RFC argument for proxy generation

*WorkloadManagement

FIX: (6338) Log error if CE Platform is missing.
CHANGE: (6336) Remove CPU(MHz) from CacheSize(kB) job parameters

*Resources

FIX: (6320) Forward jobDesc to innerCE in PoolCE
FIX: (6318) Set HOME in SingularityCE
FIX: (6317) Return proper error if provider value missing in CloudCE
CHANGE: (6283) GFA2_SRM2 does not use anymore StorageElements/DefaultProtocols option in the CS (relies on specific OutputProtocols config)
CHANGE: (6283) explicit distinction between PluginName and ProtocolSection name in all the StorageElement components
CHANGE: (6283) API change for the StorageElement. See https://github.com/DIRACGrid/DIRAC/pull/6283 for details
CHANGE: (6283) StorageElement favors local plugins

*WorkloadManagementSystem

FIX: (6288) ElasticJobParametersDB: do not return parameters that are not needed

*tests

FIX: (6287) replication transformation test: avoid target==source
FEAT: (6287) transformation tests: allow -disk SEs, avoid RAL, RALPP, CESNET (don't work for me, YMMV)

*MonitoringSystem

FIX: (6286) ElasticSearchDB generates full index names based on UTC
FIX: (6286) DataOperationSender works on a copy of the documents
CHANGE: (6286) DataOperationSender fails if the master accounting system fails

[integration]

*ConfigurationSystem

CHANGE: (6285) Make GOCDB2CSAgent functionMap less private

*Core

CHANGE: (6281) Removed Core.Utilities.MemStat module (use psutil module instead)
CHANGE: (6281) Removed possibility to Register a DISET service with a secondary name

*Resources

FIX: (6275) Conditional FC ProxyPlugin checks the ThreadConfig

*DataManagementSystem

FIX: (6268) Fix FC change{group,owner,mode} for multiple LFNs

*WorkloadManagement

NEW: (6267) ContainerBind option can be set on the per-site and per-CE level

*WorkloadManagementSystem

FIX: (6256) Making getJobParameters handle specific parameters
CHANGE: (6178) removing filtering user sites by platform during the CHECKING state

*FrameworkSystem

NEW: (6248) Add StdoutJsonBackend
FIX: (6246) dirac_login HTML error hiding the actual error

[integration]

*Resources

FIX: (6242) remove REST information from the ARC6 pilot job references to bind them with jobs
FIX: (6222) dirac_resource_info - restored the SE lookup logic
FIX: (6218) ARCCE.getARCJob() becomes protected
FIX: (6215) ARC6 typo issues
FIX: (6215) ARC writeXRSL becomes protected

*WorkloadManagement

FIX: (6241) Fix Platform key error in SD when CheckPlatform=True

*DataManagementSystem

FIX: (6240) DatasetManager: bug fix in getDatasetsInDirectory() method
FIX: (6209) Built-in metakeys were incorrectly modified in a MultiVO environment. This fix keeps them intact while modifying user-supplied metadata names by append a VO suffix, as before.

*tests

NEW: (6239) added tests for the dirac-resource-info command

*Interfaces

CHANGE: (6228) Make tmpdir for local cache instead of using CWD directly when replicating

*Core

FIX: (6227) Increase VOMS command timeout
FIX: (6204) Fixing errors, adding tests for TimeUtilities module
FIX: (6128) Discovery of HTTPS methods from parent classes

*MonitoringSystem

FIX: (6214) Fixing errors blocking commission of data operations to Elasticsearch

*ResourceStatusSystem

FIX: (6201) convert ComputingElement to CE before searching in the configuration

*WorkloadManagementSystem

FIX: (6196) Fixing errors, adding mapping for ESJobParameters
CHANGE: (6193) new method to submit pilots from ARC
CHANGE: (6161) removing the possibility to get the optimizers to run with a vo plugin

[integration]

*CI

NEW: (6189) max open files for dirac CI container to 8192

*Core

CHANGE: (6188) Propagate original proxy strength when generating new proxies

*WorkloadManagementSystem

CHANGE: (6154) Addded timestamp to elasticjobparameters and changed parameter values into strings