Dirac

[integration]

*Core

FIX: (6341) createClient now creates subclasses of the original class
NEW: (6335) DIRAC.initialize now provides keyword arguments to customise initialisation
NEW: (6335) SecurityProperty enum for expressing security properties which can be assigned to groups/hosts
NEW: (6335) LogLevel enum for choosing the logging level
NEW: (6309) Introduce DReturnType/DOKReturnType/DErrorReturnType for DIRAC return structures
CHANGE: (6299) Set OpenSSL ciphers to "DEFAULT:SECLEVEL=1"
NEW: (6299) Experimental support for Python 3.10
NEW: (6299) Suppress InsecureRequestWarning when using dirac-configure with HTTPS servers
NEW: (6299) Support overriding ciphers and methods with environment variables when using HTTPS
CHANGE: (6298) Replace suds-jurko with suds
CHANGE: (6297) Use psutil.net_if_addrs for Network.discoverInterfaces
NEW: (6291) adding general flag for disabling Monitoring for Agents (used by JobAgent)
CHANGE: (6286) Default proxy length is 2048 bits
CHANGE: (6286) remove RFC argument for proxy generation

*WorkloadManagement

FIX: (6338) Log error if CE Platform is missing.
CHANGE: (6336) Remove CPU(MHz) from CacheSize(kB) job parameters

*Resources

FIX: (6320) Forward jobDesc to innerCE in PoolCE
FIX: (6318) Set HOME in SingularityCE
FIX: (6317) Return proper error if provider value missing in CloudCE
CHANGE: (6283) GFA2_SRM2 does not use anymore StorageElements/DefaultProtocols option in the CS (relies on specific OutputProtocols config)
CHANGE: (6283) explicit distinction between PluginName and ProtocolSection name in all the StorageElement components
CHANGE: (6283) API change for the StorageElement. See https://github.com/DIRACGrid/DIRAC/pull/6283 for details
CHANGE: (6283) StorageElement favors local plugins

*WorkloadManagementSystem

FIX: (6288) ElasticJobParametersDB: do not return parameters that are not needed

*tests

FIX: (6287) replication transformation test: avoid target==source
FEAT: (6287) transformation tests: allow -disk SEs, avoid RAL, RALPP, CESNET (don't work for me, YMMV)

*MonitoringSystem

FIX: (6286) ElasticSearchDB generates full index names based on UTC
FIX: (6286) DataOperationSender works on a copy of the documents
CHANGE: (6286) DataOperationSender fails if the master accounting system fails

[integration]

*ConfigurationSystem

CHANGE: (6285) Make GOCDB2CSAgent functionMap less private

*Core

CHANGE: (6281) Removed Core.Utilities.MemStat module (use psutil module instead)
CHANGE: (6281) Removed possibility to Register a DISET service with a secondary name

*Resources

FIX: (6275) Conditional FC ProxyPlugin checks the ThreadConfig

*DataManagementSystem

FIX: (6268) Fix FC change{group,owner,mode} for multiple LFNs

*WorkloadManagement

NEW: (6267) ContainerBind option can be set on the per-site and per-CE level

*WorkloadManagementSystem

FIX: (6256) Making getJobParameters handle specific parameters
CHANGE: (6178) removing filtering user sites by platform during the CHECKING state

*FrameworkSystem

NEW: (6248) Add StdoutJsonBackend
FIX: (6246) dirac_login HTML error hiding the actual error

[integration]

*Resources

FIX: (6242) remove REST information from the ARC6 pilot job references to bind them with jobs
FIX: (6222) dirac_resource_info - restored the SE lookup logic
FIX: (6218) ARCCE.getARCJob() becomes protected
FIX: (6215) ARC6 typo issues
FIX: (6215) ARC writeXRSL becomes protected

*WorkloadManagement

FIX: (6241) Fix Platform key error in SD when CheckPlatform=True

*DataManagementSystem

FIX: (6240) DatasetManager: bug fix in getDatasetsInDirectory() method
FIX: (6209) Built-in metakeys were incorrectly modified in a MultiVO environment. This fix keeps them intact while modifying user-supplied metadata names by append a VO suffix, as before.

*tests

NEW: (6239) added tests for the dirac-resource-info command

*Interfaces

CHANGE: (6228) Make tmpdir for local cache instead of using CWD directly when replicating

*Core

FIX: (6227) Increase VOMS command timeout
FIX: (6204) Fixing errors, adding tests for TimeUtilities module
FIX: (6128) Discovery of HTTPS methods from parent classes

*MonitoringSystem

FIX: (6214) Fixing errors blocking commission of data operations to Elasticsearch

*ResourceStatusSystem

FIX: (6201) convert ComputingElement to CE before searching in the configuration

*WorkloadManagementSystem

FIX: (6196) Fixing errors, adding mapping for ESJobParameters
CHANGE: (6193) new method to submit pilots from ARC
CHANGE: (6161) removing the possibility to get the optimizers to run with a vo plugin

[integration]

*CI

NEW: (6189) max open files for dirac CI container to 8192

*Core

CHANGE: (6188) Propagate original proxy strength when generating new proxies

*WorkloadManagementSystem

CHANGE: (6154) Addded timestamp to elasticjobparameters and changed parameter values into strings

[integration]

*Core

FIX: (6179) better report of error when importing Module
FIX: (6167) Replaced old vomses/vomsdir code to be able to handle cvmfs vomses. See https://github.com/DIRACGrid/DIRAC/issues/6150
FIX: (6151) Replace deprecated threading.activeCount with active_count
FIX: (6149) use AnyThreadEventLoopPolicy asyncio policy
FIX: (6144) Removing time dependence of TimeUtilities Module
NEW: (6137) add https to issuer address automatically
FIX: (6132) fix error reporting in TornadoService
CHANGE: (6091) changed name and simplified repetitive methods of Utilities.TIme
CHANGE: (6084) BaseRequestHandler encode/decode static methods

*MonitoringSystem

FIX: (6175) MonitoringReporter - late binding to MonitoringDB
FIX: (6076) Removing multiplication in timestamp when committing to monitoring

*WorkloadManagementSystem

FIX: (6173) restore /LocalSite/InstancePath for the case of JobAgents (see issue 6143)
FIX: (6162) use the JobMonitoringClient to check for job parameters (that can be in the ES backend)
NEW: (6134) adding a test to show the modifications made to the JDL before being inserted in the JobDB

*RequestManagementSystem

NEW: (6172) Add option to automatically cancel Scheduled requests after a defined time period. Default behaviour unchanged. Relates to 6136
CHANGE: (6114) Replace BLOBs with TEXTs for ReqDB

*docs

NEW: (6147) add info on installing non-released code from within the pilot

*Resources/Cloud

FIX: (6146) userData need to be in "ascii" string format to meet the requirement of "create_node" when creating VMs from cloud resources

*FrameworkSystem

FIX: (6138) dirac_login: avoid misleading error message, fixes issue 6096
FIX: (6135) dirac_login: Pass proxy as string to writeToProxyFile()
NEW: (6130) add the possibility to add a prefix before the actual log message for a given subLogger

[integration]

*Tests

FIX: (6122) restart Tornado after updating the configuration

*Core

FIX: (6120) don't truncate first character in cli doc strings
FIX: (6103) Race condition which causes multiple connectioons to be opened when the connection to an executor drops
CHANGE: (6103) Improve logging split between message/varmessage in execuctors
FIX: (6083) Grid: BDII make ldapsearch call forward compatible with newer ldapsearch versions

*DataManagementSystem

NEW: (6115) Introduces an option 'ForceIndexedMetadata' to disable the creation of unindexed metadata. Default: False, i.e. preserves the current default where both indexed and unindexed metadata are allowed. Relates to 6029.

*Resources

FIX: (6111) Error in the condition to get FlavorName
FIX: (6110) use identity instead of subject of proxy in CloudCE
FIX: (6102) remove decode from proxy in CloudCE
FIX: (6095) use ceType instead of ceName for CE parameters
FIX: (6088) autodetect cvmfs proxy in default cloudce template
FIX: (6083) ARCCE make ldapsearch call forward compatible with newer ldapsearch versions
FIX: (6080) Adapt StompMQConnector to Stomp 8

*tests

FIX: (6104) typo in testJobDefinitions for new Cloud test
NEW: (6081) added hackathon test for CloudComputingElement

*WorkloadManagement

FIX: (6103) Avoid producing massive log output in OptimizationMind

*Elasticsearch

NEW: (6094) option op_type in index API for Elasticsearch