Security-monkey

----------------------------------------

- PR 666 - redixin - Use find_packages in setup.py to include nested packages.
- PR 667 - monkeysecurity - Explicitly adding `urllib3[secure]` to setup.py (REVERTED in 683)
- PR 668 - monkeysecurity - IPv6 support in security groups.
- PR 669 - monkeysecurity - Updating the security group auditor to treat `::/0` the same as `0.0.0.0/0`
- PR 671 - monkeysecurity - Enhancing PolicyDiff to be able to handle non-ascii strings.
- PR 673 - monkeysecurity - Fixing path to `aws_accounts.json`. (Broken my moving `manage.py`)
- PR 675 - monkeysecurity - Adding `package_data` and `data_files` sections to setup.py.
- PR 677 - willbengtson - Fixing the security trackable information.
- PR 682 - monkeysecurity - Updating packaged supervisor config to provide full path to `monkey`
- PR 681 - AlexCline - Add reference_policies for TLS transitional ELB security policies
- PR 684 - monkeysecurity - Disabling DB migration `b8ccf5b8089b`. Was freezing some `db upgrades`
- PR 683 - monkeysecurity - Reverted 667. Added `pip install --upgrade urllib3[secure]` to `quickstart` and `Dockerfile`.
- PR 685 - monkeysecurity - Running `docker-compose build` in Travis-CI.
- PR 688 - mcpeak - Add Bandit gate to Security Monkey.
- PR 687 - mikegrima - Fix for issue 680. (Unable to edit account names)
- PR 689 - mikegrima - Enhancements to Travis-CI: parallelized the workloads. (docker/python/dart in parallel)

Important Notes:
- This is a hotfix release to correct a number of installation difficulties reported since `0.9.0`.

Contributors:
- redixin
- AlexCline
- willbengtson
- mcpeak
- mikegrima
- monkeysecurity

----------------------------------------

- PR 500 - monkeysecurity - Updating ARN.py to look for StringEqualsIgnoreCase in policy condition blocks
- PR 511 - kalpatel01 - Fix KMSAuditor exceptions
- PR 510 - kalpatel01 - Add additional JIRA configurations
- PR 504 - redixin - Plugins support
- PR 515 - badraufran - Add ability to press enter to search in search bar component
- PR 514 - badraufran - Update dev_setup_osx.rst to get it up-to-date
- PR 513 / 545- mikegrima - Fix for S3 watcher errors.
- PR 516 - badraufran - Remove broken packages link
- PR 518 - badraufran - Update `dev_setup_osx` (Remove sudo)
- PR 519 - selmanj - Minor reformatting/style changes to Docker docs
- PR 512 / 521 - kalpatel01 - Organize tests into directories
- PR 524 - kalpatel01 - Remove DB mock class
- PR 522 - kalpatel01 - Optimize SQL for account delete
- PR 525 - kalpatel01 - Handle known kms boto exceptions
- PR 529 - mariusgrigaitis - Usage of `GOOGLE_HOSTED_DOMAIN` in sample configs
- PR 532 - kalpatel01 - Add sorting to account tables (UI)
- PR 538 - cu12 - Add more Docker envvars
- PR 536 / 540 - supertom - Add account type field to item, item details and search bar.
- PR 534 / 541 - kalpatel01 - Add bulk enable and disable account service
- PR 546 - supertom - GCP: fixed accounttypes typo.
- PR 547 - monkeysecurity - Delete deprecated Account fields
- PR 528 - kalpatel01 - Fix reaudit issue for watchers in different intervals
- PR 553 - mikegrima - Fixed bugs in the ES watcher
- PR 535 / 552 - kalpatel01 - Add support for overriding audit scores
- PR 560 / 587 - mikegrima - Bump CloudAux version
- PR 533 / 559 - kalpatel01 - Add Watcher configuration
- PR 562 - monkeysecurity - Re-adding reporter timing information to the logs.
- PR 557 - kalpatel01 - Add justified issues report
- PR 573 - monkeysecurity - fixing issue duplicate ARN issue…
- PR 564 - kalpatel01 - Fix justification preservation bug
- PR 565 - kalpatel01 - Handle unicode name tags
- PR 571 - kalpatel01 - Explicitly set export filename
- PR 572 - kalpatel01 - Fix minor watcher bugs
- PR 576 - kalpatel01 - Set user role via SSO profile
- PR 569 - kalpatel01 - Split `check_access_keys` method in the IAM User Auditor
- PR 566 - kalpatel01 - Convert watchers to boto3
- PR 568 - kalpatel01 - Replace ELBAuditor DB query with support watcher
- PR 567 - kalpatel01 - Reduce AWS managed policy audit noise
- PR 570 - kalpatel01 - Add support for custom watcher and auditor alerters
- PR 575 - kalpatel01 - Add functionality to clean up stale issues
- PR 582 - supertom - [GCP] Watchers/Auditors for GCP
- PR 588 - supertom - GCP docs: Draft of GCP changes
- PR 592 - monkeysecurity - SSO Role Modifications
- PR 597 - supertom - GCP: fixed issue where client wasn't receiving user-specified creds
- PR 598 - redixin - Implement `add_account_%s` for custom accounts
- PR 600 - supertom - GCP: fixed issue where bucket watcher wasn't sending credentials to Cloudaux
- PR 602 - crruthe - Added permission for DescribeVpnGateways missing
- PR 605 - monkeysecurity - ELB Auditor - Fixing reference to check_rfc_1918
- PR 610 - monkeysecurity - Adding Unique Index to TechName and AccountName
- PR 612 - carise - Add a section on using GCP Cloud SQL Postgres with Cloud SQL Proxy
- PR 613 - monkeysecurity - Setting Item.issue_count to deferred. Only joining tables in distinct if necessary.
- PR 614 - monkeysecurity - Increasing default timeout
- PR 607 - supertom - GCP: Set User Agent
- PR 609 - mikegrima - Added ephemeral section to S3 for "GrantReferences"
- PR 611 - roman-vynar - Quick start improvements
- PR 619 - mikegrima - Fix for plaintext passwords in DB if using CLI for user creation
- PR 622 - jonhadfield - Fix ACM certificate ImportedAt timestamp
- PR 616 - redixin - Fix docs and variable names related to custom alerters
- PR 502 - mikegrima - Batching support for watchers
- PR 631 - supertom - Added `__version__` property
- PR 632 - sysboy - Set the default value of SECURITY_REGISTERABLE to False
- PR 629 - BobPeterson1881 - Fix security group rule parsing
- PR 630 - BobPeterson1881 - Update dashboard view filter links
- PR 633 - sysboy - Log Warning when S3 ACL can't be retrieved.
- PR 639 - monkeysecurity - Removing reference to zerotodocker.
- PR 624 - mikegrima - Adding utilities to get S3 canonical IDs.
- PR 640 - supertom - GCP: fixed UI Account Type filtering
- PR 642 - monkeysecurity - Adding active and third_party flags to account view API
- PR 646 - monkeysecurity - Removing s3_name from exporter and renaming Account.number to identifier
- PR 648 - mikegrima - Fix for UI Account creation bug
- PR 657 658 - jeyglk - Fix Docker
- PR 655 - monkeysecurity - Updating quickstart/install documentation to simplify.
- PR 659 - monkeysecurity - Quickstart GCP Fixes
- PR 625 - bungoume - Fix principal KeyError
- PR 662 - monkeysecurity - Replacing `python manage.py` with `monkey`
- PR 660 - mcpeak - Adding an option to allow group write for logfiles
- PR 661 - shrikant0013 - Added doc on update/upgrade steps

Important Notes:

- `SECURITY_MONKEY_SETTINGS` is no longer a required environment variable.
- If supplied, security_monkey will respect the variable. Otherwise it will default to env-config/config.py
- `manage.py` has been moved inside the package and a `monkey` alias has been setup.
- Where you might once call `python manage.py <arguments>` you will now call `monkey <arguments>`
- Documentation has been converted from RST to Markdown.
- I will no longer be using readthedocs or RST.
- Quickstart guide has been largely re-written.
- Quickstart now instructs you to create and use a virtualenv (and how to get supervisor to work with it)
- This release contains [GCP Watcher Support](https://medium.com/Netflix_Techblog/netflix-security-monkey-on-google-cloud-platform-gcp-f221604c0cc7).
- Additional Permissions Required:
- ec2:DescribeVpnGateways

Contributors:
- kalpatel01
- redixin
- badraufran
- selmanj
- mariusgrigaitis
- cu12
- supertom
- crruthe
- carise
- roman-vynar
- jonhadfield
- sysboy
- jeyglk
- bungoume
- mcpeak
- shrikant0013
- mikegrima
- monkeysecurity

----------------------------------------

- PR \425 - crruthe - Fixed a few report hyperlinks.
- PR \428 - nagwww - Documentation fix. Renamed module: security\_monkey.auditors.elb to module: security\_monkey.auditors.elasticsearch\_service
- PR \424 - mikegrima - OS X Install doc updates for El Capitan and higher.
- PR \426 - mikegrima - Added "route53domains:getdomaindetail" to permissions doc.
- PR \427 - mikegrima - Fix for ARN parsing of cloudfront ARNs.
- PR \431 - mikegrima - Removed s3 ARN check for ElasticSearch Service.
- PR \448 - zollman - Fix exception logging in store\_exception.
- PR \444 - zollman - Adds exception logging listener for appscheduler.
- PR \454 - mikegrima - Updated S3 Permissions to reflect latest changes to cloudaux.
- PR \455 - zollman - Add Dashboard.
- PR \456 - zollman - Increase issue note size.
- PR \420 - crruthe - Added support for SSO OneLogin.
- PR \432 - robertoriv - Add pagination for whitelist and ignore list.
- PR \438 - AngeloCiffa - Pin moto==0.4.25. (TODO: Bump Jinja2 version.)
- PR \433 - jnbnyc - Added Docker/Docker Compose support for local dev.
- PR \408 - zollman - Add support for custom account metadata. (An important step that will allow us to support multiple cloud providers in the future.)
- PR \439 - monkeysecurity - Replace botor lib with Netflix CloudAux.
- PR \441 - monkeysecurity - Auditor ChangeItems now receive ARN.
- PR \446 - zollman - Fix item 'first\_seen' query .
- PR \447 - zollman - Refactor rdsdbcluster array params.
- PR \445 - zollman - Make misfire grace time and reporter start time configurable.
- PR \451 - monkeysecurity - Add coverage with Coveralls.io.
- PR \452 - monkeysecurity - Refactor & add tests for the PolicyDiff module.
- PR \449 - monkeysecurity - Refactoring s3 watcher to use Netflix CloudAux.
- PR \453 - monkeysecurity - Fixing two policy diff cases.
- PR \442 - monkeysecurity - Adding index to region. Dropping unused item.cloud.
- PR \450 - monkeysecurity - Moved test & onelogin requirements to the setup.py extras\_require section.
- PR \407 - zollman - Link together issues by enabling auditor dependencies.
- PR \419 - monkeysecurity - Auditor will now fix any issues that are not attached to an AuditorSetting.
- PR NONE - monkeysecurity - Item View no longer returns revision configuration bodies. Should improve UI for items with many revisions.
- PR NONE - monkeysecurity - Fixing bug where SSO arguments weren't passed along for branded sso. (Where the name is not google or ping or onelogin)
- PR \476 - markofu - Update aws\_accounts.json to add Canada and Ohio regions.
- PR NONE - monkeysecurity - Fixing manage.py::amazon\_accounts() to use new AccountType and adding delete\_unjustified\_issues().
- PR \480 - monkeysecurity - Making Gunicorn an optional import to help support dev on Windows.
- PR \481 - monkeysecurity - Fixing a couple dart warnings.
- PR \482 - monkeysecurity - Replacing Flask-Security with Flask-Security-Fork.
- PR \483 - monkeysecurity - issue \477 - Fixes IAM User Auditor login\_profile check.
- PR \484 - monkeysecurity - Bumping Jinja2 to \>=2.8.1
- PR \485 - robertoriv - New IAM Role Auditor feature - Check for unknown cross account assumerole.
- PR \487 - hyperbolist - issue \486 - Upgrade setuptools in Dockerfile.
- PR \489 - monkeysecurity - issue \251 - Fix IAM SSL Auditor regression. Issue should be raised if we cannot obtain cert issuer.
- PR \490 - monkeysecurity - issue \421 - Adding ephemeral field to RDS DB issue.
- PR \491 - monkeysecurity - Adding new RDS DB Cluster ephemeral field.
- PR \492 - monkeysecurity - issue \466 - Updating S3 Auditor to use the ARN class.
- PR NONE - monkeysecurity - Fixing typo in dart files.
- PR \495 - monkeysecurity - issue \494 - Refactoring to work with the new Flask-WTF.
- PR \493 - monkeysecurity - Windows 10 Development instructions.
- PR NONE - monkeysecurity - issue \496 - Bumping CloudAux to \>=1.0.7 to fix IAM User UploadDate field JSON serialization error.

Important Notes:

- New permissions required:
- s3:getaccelerateconfiguration
- s3:getbucketcors
- s3:getbucketnotification
- s3:getbucketwebsite
- s3:getreplicationconfiguration
- s3:getanalyticsconfiguration
- s3:getmetricsconfiguration
- s3:getinventoryconfiguration
- route53domains:getdomaindetail
- cloudtrail:gettrailstatus

Contributors:

- zollman
- robertoriv
- hyperbolist
- markofu
- AngeloCiffa
- jnbnyc
- crruthe
- nagwww
- mikegrima
- monkeysecurity

-------------------

- PR \410/\405 - zollman - Custom Watcher/Auditor Support. (Dynamic Loading)
- PR \412 - llange - Google SSO Fixes
- PR \409 - kyelberry - Fixed Report URLs in UI.
- PR \413 - markofu - Better handle IAM SSL certificates that we cannot parse.
- PR \411 - zollman - Many, many new watchers and auditors.

New Watchers:

> - CloudTrail
> - AWSConfig
> - AWSConfigRecorder
> - DirectConnect::Connection
> - EC2::EbsSnapshot
> - EC2::EbsVolume
> - EC2::Image
> - EC2::Instance
> - ENI
> - KMS::Grant
> - KMS::Key
> - Lambda
> - RDS::ClusterSnapshot
> - RDS::DBCluster
> - RDS::DBInstace
> - RDS::Snapshot
> - RDS::SubnetGroup
> - Route53
> - Route53Domains
> - TrustedAdvisor
> - VPC::DHCP
> - VPC::Endpoint
> - VPC::FlowLog
> - VPC::NatGateway
> - VPC::NetworkACL
> - VPC::Peering

Important Notes:

- New permissions required:
- cloudtrail:describetrails
- config:describeconfigrules
- config:describeconfigurationrecorders
- directconnect:describeconnections
- ec2:describeflowlogs
- ec2:describeimages
- ec2:describenatgateways
- ec2:describenetworkacls
- ec2:describenetworkinterfaces
- ec2:describesnapshots
- ec2:describevolumes
- ec2:describevpcendpoints
- ec2:describevpcpeeringconnections,
- iam:getaccesskeylastused
- iam:listattachedgrouppolicies
- iam:listattacheduserpolicies
- lambda:listfunctions
- rds:describedbclusters
- rds:describedbclustersnapshots
- rds:describedbinstances
- rds:describedbsnapshots
- rds:describedbsubnetgroups
- redshift:describeclusters
- route53domains:listdomains

Contributors:

- zollman
- kyleberry
- llange
- markofu
- monkeysecurity

-------------------

- issue \292 - PR \332 - Add ephemeral sections to the redshift watcher
- PR \338 - Added access key last used to IAM Users.
- Added an IAM User auditor check to look for access keys without use in past 90 days.
- PR \334 - alexcline - Route53 watcher and auditor. (Updated to use botor in PR \343)
- Logo updated. Weapon replaced with banana. Expect more logo changes soon.
- PR \345 - Ephemeral changes now update the latest revision. Revisions now have a date\_last\_ephemeral\_change column as well as a date\_created column.
- PR \349 - mikegrima - Install documentation updates
- PR \354 - Feature/SSO (YAY)
- PR \365 - alexcline - Added ACM (Amazon Certificate Manager) watcher/auditor
- PR \358/\370 - alexcline - Alex cline feature/kms
- Updated Dart/Angular dart versions.
- PR \362 - crruthe - Changed to dictConfig logging format
- PR \372 - ollytheninja - SQS principal bugfix
- PR \379 - bunjiboys - Adding Mumbai region
- PR \380 - bunjiboys - Adding Mumbai ELB Log AWS Account info
- PR \381 - ollytheninja - Adding tags to the S3 watcher
- Boto updates
- PR \376 - Adding item.arn field. Adding item.latest\_revision\_complete\_hash and item.latest\_revision\_durable\_hash. These are for the bananapeel rearchitecture.
- PR \386 - Shortening sessions from default value to 60 minutes. Setting Cookie HTTPONLY and SECURE flags.
- PR \389 - Adding CloudTrail table, linked to itemrevision. (To be used by bananapeel rearchitecture.)
- PR \390 - ollytheninja - Adding export CSV button.
- PR \394 - mikegrima - Saving exceptions to database table
- PR \402 - issue \401 - Adding new ELB Reference Policy ELBSecurityPolicy-2016-08

Hotfixes:

- Upgraded Cryptography to 1.3.1
- Updated docs to use sudo -E when calling manage.py amazon\_accounts.
- Updated the record\_exception decorator to allow the region to be overwritten. (Useful for region-less technology that likes to be recorded in the "universal" region.)
- issue \331 - IAMSSL watcher failed on elliptic curve certs

Important Notes:

- Route53 IgnoreList entries may match zone name or recordset name.
- Checkout the new log configuration format from PR \362. You may want to update your config.py.
- New permissions required:
- "acm:ListCertificates",
- "acm:DescribeCertificate",
- "kms:DescribeKey",
- "kms:GetKeyPolicy",
- "kms:ListKeys",
- "kms:ListAliases",
- "kms:ListGrants",
- "kms:ListKeyPolicies",
- "s3:GetBucketTagging"

- Some dependencies have been updated (cryptography, boto, boto3, botocore, botor, pyjwt). Please re-run python setup.py install.
- Please add the following lines to your config.py for more time-limited sessions:

~~~~ {.sourceCode .python}
PERMANENT_SESSION_LIFETIME=timedelta(minutes=60) Will logout users after period of inactivity.
SESSION_REFRESH_EACH_REQUEST=True
SESSION_COOKIE_SECURE=True
SESSION_COOKIE_HTTPONLY=True
PREFERRED_URL_SCHEME='https'

REMEMBER_COOKIE_DURATION=timedelta(minutes=60) Can make longer if you want remember_me to be useful
REMEMBER_COOKIE_SECURE=True
REMEMBER_COOKIE_HTTPONLY=True
~~~~

Contributors:

- alexcline
- crruthe
- ollytheninja
- bunjiboys
- mikegrima
- monkeysecurity

-------------------

- PR \286 - bunjiboys - Added Seoul region AWS Account IDs to import scripts
- PR \291 - sbasgall - Corrected ignore\_list.py variable names and help strings
- PR \284 - mikegrima - Fixed cross-account root reporting for ES service (Issue \283)
- PR \293 - mikegrima - Updated quickstart documentation to remove permission wildcards (Issue \287)
- PR \301 - monkeysecurity - iamrole watcher can now handle many more roles (1000+) and no longer times out.
- PR \316 - DenverJ - Handle database exceptions by cleaning up session.
- PR \289 - delikat - Persist custom role names on account creation
- PR \321 - monkeysecurity - Item List and Item View will no longer display disabled issues.
- PR \322 (PR \308) - llange - Ability to add AWS owned managed policies to ignore list by ARN (Issue \148)
- PR \323 - snixon - Breaks check\_securitygroup\_any into ingress and egress (Issue \239)
- PR \309 - DenverJ - Significant database query optimizations by tuning itemrevision retrievals
- PR \324 - mikegrima - Handling invalid ARNs more consistently between watchers (Issue \248)
- PR \317 - ollytheninja - Add Role Based Access Control
- PR \327 - monkeysecurity - Added Flask-Security's SECURITY\_TRACKABLE to backend and UI
- PR \328 - monkeysecurity - Added ability to parse AWS service "ARNs" like events.amazonaws.com as well as ARNS that use \* for the account number like arn:aws:s3:​\*:\*​:some-s3-bucket
- PR \314 - pdbogen - Update Logging to have the ability to log to stdout, useful for dockerizing.

Hotfixes:

- s3\_acl\_compare\_lowercase: AWS now returns S3 ACLs with a lowercased owner. security\_monkey now does a case insensitive compare
- longer\_resource\_ids. Updating DB to handle longer AWS resource IDs: <https://aws.amazon.com/blogs/aws/theyre-here-longer-ec2-resource-ids-now-available/>
- Removed requests from requirements.txt/setup.py as it was pinned to a very old version and not directly required (Issue \312)
- arn\_condition\_awssourcearn\_can\_be\_list. Updated security\_monkey to be able to handle a list of ARNS in a policy condition.
- ignore\_list\_fails\_on\_empty\_string: security\_monkey now properly handles an ignorelist entry containing a prefix string of length 0.
- protocol\_sslv2\_deprecation: AWS stopped returning whether an ELB listener supported SSLv2. Fixed security\_monkey to handle the new format correctly.

Important Notes:

- security\_monkey IAM roles now require a new permission: iam:listattachedrolepolicies
- Your security\_monkey config file should contain a new flag: SECURITY\_TRACKABLE = True
- You'll need to rerun python setup.py install to obtain the new dependencies.

Contributors:

- bunjiboys
- sbasgall
- mikegrima
- DenverJ
- delikat
- snixon
- ollytheninja
- pdbogen
- monkeysecurity