Upgraded base container to python 3.10 on debian bullseye
v4.1.0.stable9
**Service Base**
- Added option to normalize paths and GUID mapping in process tree for dynamic analysis helper
v4.1.0.stable8
Base
- Fix Azure blob store save operation skipped even if we're forcing it to go through (this fixes the system safelist not being saved after being edited once)
Frontend
- Fix system tag_safelist editor size tracking
v4.1.0.stable7
Frontend
- Added a carousel viewer for image_section images
v4.1.0.stable6
Service Base:
- Revert image check on add_image()
v4.1.0.stable5
Service Base:
- Check if file is really an image before adding to ImageSection
v4.1.0.stable33
Base
- Added multi-item pop for queues
- Added flag to prevent AutoExportingCounters to export zero metrics
- Updated to Elastic 7.16.1 to fix Log4j Vulnerability
Core
- Multi-pop tasks in dispatcher
- Updated to Elastic 7.16.1 to fix Log4j Vulnerability
Service Server
- Batch service server statistics to reduce the load on the system
- Updated to Elastic 7.16.1 to fix Log4j Vulnerability
UI API Server
- Updated to Elastic 7.16.1 to fix Log4j Vulnerability
v4.1.0.stable32
Core
- Updater: Service updates will be run in parallel
v4.1.0.stable31
Service Client
- Hotfix: instantiate conditional variable whether if service requires a file or not
v4.1.0.stable30
Core
- Fix expiry's heartbeats and add maxed out state for archiving
v4.1.0.stable29
Service-Client
- Add a FILE_NOT_FOUND status
- On FILE_NOT_FOUND, mark as NON_RECOVERABLE
- Distinguish different types of exception messages/handling on download_file()
v4.1.0.stable28
Base
- Stop unnecessary filestore validation after write
Core
- Fix double counting of cache miss in ingester
- Only drop files in ingester if inflight queue is higher then 80%
- Use get_if_exists instead of get when saving results
- Add more APMs to dispatch and submission clients
Service Server
- Stop validating that the file is in the filestore since upload function already does that
- Use a multiget to get file infos instead of sequential gets
- Process embedded files in a threadpool
- Add APMs
v4.1.0.stable27
Base, UI, Service Server
- Allow the use of global configurations for services ( overridable)
- Global preferred_registry_type for services' registry_type (if absent on registration)
v4.1.0.stable26
Core
- Fix updates container credentials can get out of sync with service
- Cleanup docker controller for Docker-compose appliances
v4.1.0.stable25
Service Base
- Add exponential backoff for services to a max of 30s when trying to reach updaters
v4.1.0.stable24
Base
- Added max_queue_length support for services
- Fix data histogram validation
- Allow datastore archive_access to be forcibly turned on/off
Core
- Enforce service max_queue_size using plumber
- Use new archive_access param in datastore
- Fix docker controller to work with AL 4.1
- Fix Kubernetes controller update containers out-of-sync when restarting services
- Fix scaler and updater to work with docker-compose appliances
Frontend
- Show changed service settings and add reset buttons
- Allow service maximum queue length to be editted in the frontend
Service Base
- Fix SERVICE_API_KEY propagation
Service Server
- Fix SERVICE_API_KEY propagation
- Send events when registering services
UI API Server
- Added API to support service diff in the frontend
- Stop looking up archive when ingesting file via sha256
v4.1.0.stable23
Base
- Stop using stored_fields in datastore and use _source instead
- Optimize search / multiget combos to use search with full _source instead
- Expose default scaler config values
- Remove stream_search restriction that can be circumvented
- Fix histogram search deprecation warnings
- Change default alert returned fields
- Retry on FTP DNS errors
- More details on elasticsearch timeout and connection error
- Allow datastore transport timeouts to be changed
- Added ELF File and vector tags
Core
- Fix tests to work with _source changes
- Fix Dispatcher recovery when redis volatile restart
- Added TTLs to dispatcher's queues
Frontend
- Change the meaning of stored fields in the help page
- Adjust dashboard dispatcher card margins
- Force service sorting by name
Service Base
- Allow tasking to take place inside another folder
- Configurable update directory
- Configurable manifest folder
- Fix time used for caching in the service updater
- Fix default get_tool_version and get_service_version
- Cleanup update directories on start
Service Client
- Allow tasking to take place inside another folder
UI - API Server
- Optimize search / multiget combos to use search with full _source instead
v4.1.0.stable22
Core
- Rollback scaling changes that caused more harm then good
v4.1.0.stable21
Core
- Emergency bugfix over the previous build
v4.1.0.stable20
Core
- Allow pod/container over allocation in scaler
- Linear incremental increase/decrease of containers during scaling
v4.1.0.stable19
Frontend
- Fix service restore prompt not closing after restore
- Close search suggestions when clicking away form the search bar
- Notification Area is now color coded and displays the first letter of the message severity
v4.1.0.stable18
Frontend
- Fix service restore prompt not closing after restore
- Close search suggestions when clicking away form the search bar
- Notification Area is now color coded and displays the first letter of the message severity
v4.1.0.stable17
Dependencies
- Update base elasticsearch to 7.15.1
UI - API Server
- Added functionality to file hex API to:
- Set the length of bytes per line
- Only get the bytes without the offset or ascii
v4.1.0.stable16
Frontend
- Fix rendering issues when changing the theme
- Catch all exception and show a message to the user instead of a blank screen
v4.1.0.stable15
Base
- Add index name in the logs when an index is: Not Ready, Too Busy or Non-writeable
v4.1.0.stable14
Base
- Change fix_shards cli command so it can auto-fix cases where the current alias point <index_name>__fix_shards and <index_name>_hot has been delete.
v4.1.0.stable13
Frontend
- Fix error when user have mutiple consecutive spaces in their name
v4.1.0.stable12
Base
- Lookup software related attack_ids in the revoke map
- Updated Top level domains list
- Use document versions to get and save file atomically instead of using locks
Core
- Properly handle invalid submission classification
- Use document versions to get and save file atomically instead of using locks
- Perform archiving in chunks to prevent timeouts
- Pre-compute expiry daterange instead of using a datemath inside the expiry lucene query
Frontend
- Do not auto-retry on quota exceptions
- Auto-retry API calls when API server is down
- Make auto-retries use an exponential backoff
Sevice Base
- Stop polluting logs from empty extracted/supplementary files
UI - API Server
- Use one global filestore per process
- Re-use globally loaded config, filestore and classification engine
- Improve ingest API speed by removing extra filestore calls and by optimizing the sha256 submit usecase
v4.1.0.stable11
Base
- Remove v3 services compat tag map
- Added service cleanup method to CLI
Frontend
- Adjust submit page margins
- Make watermark clickthrough
Service Base
- Fix run_service_once handling of zeroized section flags
Service client
- Properly stop the task handler when the number of processed files has reached it's limit
v4.1.0.stable10
Frontend
- Preserve file name when downloading files from a submission