Detect-secrets

May 6th, 2024

We apologise for the extreme delay in publishing a new release for our beloved `detect-secrets`. We at Yelp appreciate your continued support and your contributions to this valuable project!

:newspaper: News
- We're adding support for Python 3.10, 3.11 and 3.12 and we dropped support for Python 3.6 and 3.7! We hope this won't be too disruptive for you all. Be aware that in a next release, we'll remove support for Python 3.8 too, as it'll reach EOL in October 2024.

:mega: Release Highlights
- Added support for OS-agnostic baseline files ([586])

:tada: New Features
- Added a detector for IP addresses ([692])
- Added a detector for GitLab tokens ([782])
- Added a detector for Telegram tokens ([808])
- Added a detector for Pypi and TestPypi tokens ([819])
- Added a detector for OpenAI tokens ([823])

:sparkles: Usability
- Added filenames in errors thrown when a plugin file specified in the `.secrets.baseline` is not found. ([719])
- Changed the wording of the audit prompt ([738])

:telescope: Accuracy
- Improved DiscordBotTokenDetector to reduce false negatives ([628])
- Improved KeywordDetector to reduce false positive for Golang ([675])
- Improved AWSKeyDetector by adding more access key formats ([796])

:bug: Bugfixes
- Fixed `NotImplementedError` in StatisticsAggregator ([678])
- Fixed bug in YAMLTransformer related to parsing YAML files with achors and tags ([679])
- Fixed `IndexError` in `is_prefixed_with_dollar_sign` caused by passing empty strings ([712])

:snake: Miscellaneous
- Dropped support for Python 3.6 ([672])
- Dropped support for Python 3.7 ([724])
- Added support for Python 3.10 ([724])
- Added support for Python 3.11 ([730])
- Added support for Python 3.12 ([810])
- Multiple dependency updates

[586]: https://github.com/Yelp/detect-secrets/pull/586
[628]: https://github.com/Yelp/detect-secrets/pull/628
[672]: https://github.com/Yelp/detect-secrets/pull/672
[675]: https://github.com/Yelp/detect-secrets/pull/675
[678]: https://github.com/Yelp/detect-secrets/pull/678
[679]: https://github.com/Yelp/detect-secrets/pull/679
[692]: https://github.com/Yelp/detect-secrets/pull/692
[712]: https://github.com/Yelp/detect-secrets/pull/712
[719]: https://github.com/Yelp/detect-secrets/pull/719
[724]: https://github.com/Yelp/detect-secrets/pull/724
[730]: https://github.com/Yelp/detect-secrets/pull/730
[738]: https://github.com/Yelp/detect-secrets/pull/738
[782]: https://github.com/Yelp/detect-secrets/pull/782
[796]: https://github.com/Yelp/detect-secrets/pull/796
[808]: https://github.com/Yelp/detect-secrets/pull/808
[810]: https://github.com/Yelp/detect-secrets/pull/810
[819]: https://github.com/Yelp/detect-secrets/pull/819
[823]: https://github.com/Yelp/detect-secrets/pull/823

October 4th, 2022

:newspaper: News
- We're dropping support for Python 3.6 starting v1.5.0! Python 3.6 reached EOL on December 23, 2021 and, therefore, is currently unsupported. We hope this announcement gives you plenty of time to upgrade your project, if needed.

:mega: Release Highlights
- Improved filtering by excluding secrets that have already been detected by a regex-based detector ([612])
:tada: New Features
- Added a detector for Discord bot tokens ([614])

:sparkles: Usability
- Improved the audit report to make it easier to parse programmatically ([619])
:telescope: Accuracy
- Improve ArtifactoryDetector plugin to reduce false positives ([499])

:bug: Bugfixes
- Fixed the verify flow in audit report by adding the code snippet of the verified secret ([620])
- Fixed deploy process to be environment configuration independent ([625])

:snake: Miscellaneous
- Added support for .NET packages.lock.json files in the heuristic filter ([593])
- Multiple dependency updates

[499]: https://github.com/Yelp/detect-secrets/pull/499
[556]: https://github.com/Yelp/detect-secrets/pull/556
[589]: https://github.com/Yelp/detect-secrets/pull/589
[593]: https://github.com/Yelp/detect-secrets/pull/593
[598]: https://github.com/Yelp/detect-secrets/pull/598
[612]: https://github.com/Yelp/detect-secrets/pull/612
[614]: https://github.com/Yelp/detect-secrets/pull/614
[615]: https://github.com/Yelp/detect-secrets/pull/615
[616]: https://github.com/Yelp/detect-secrets/pull/616
[619]: https://github.com/Yelp/detect-secrets/pull/619
[620]: https://github.com/Yelp/detect-secrets/pull/620
[625]: https://github.com/Yelp/detect-secrets/pull/625

July 22nd, 2022

:mega: Release Highlights
- Add Windows operating system to Github CI Action ([528])
- Enable dependabot for automated dependency updates built into GitHub ([531])
- Improve performance for array slice ([555])

:tada: New Features
- Improve keyword plugin to detect arrow key assignment ([567])
- Add command line argument for `detect-secrets-hook` to return output as json ([569])

:bug: Bugfixes
- Fix regex matching for `npm` plugin ([551])
- Fix `audit` crashing when secret is not found on specified line ([568])
- Fix ` pragma: allowlist nextline secret` secrets not filtered out of result set ([575])
- Fix `is_verified` flag not stored in `PotentialSecret` ([578])

:snake: Miscellaneous
- Only use ANSI color code in environments that support it ([523])
- Multiple dependency updates
- Make `is_likely_id_string` heuristic filter more strict to avoid eliminating true positives ([526])
- Refactor AWS access key regex to minimize false positives ([571])
- Correct spelling errors in code repository ([574])
- Add `py.typed` to enable type hints for package consumers ([579])

[523]: https://github.com/Yelp/detect-secrets/pull/523
[526]: https://github.com/Yelp/detect-secrets/pull/526
[528]: https://github.com/Yelp/detect-secrets/pull/528
[529]: https://github.com/Yelp/detect-secrets/pull/529
[530]: https://github.com/Yelp/detect-secrets/pull/530
[531]: https://github.com/Yelp/detect-secrets/pull/531
[532]: https://github.com/Yelp/detect-secrets/pull/532
[533]: https://github.com/Yelp/detect-secrets/pull/533
[535]: https://github.com/Yelp/detect-secrets/pull/535
[537]: https://github.com/Yelp/detect-secrets/pull/537
[538]: https://github.com/Yelp/detect-secrets/pull/538
[542]: https://github.com/Yelp/detect-secrets/pull/542
[543]: https://github.com/Yelp/detect-secrets/pull/543
[545]: https://github.com/Yelp/detect-secrets/pull/545
[546]: https://github.com/Yelp/detect-secrets/pull/546
[551]: https://github.com/Yelp/detect-secrets/pull/551
[555]: https://github.com/Yelp/detect-secrets/pull/555
[567]: https://github.com/Yelp/detect-secrets/pull/567
[568]: https://github.com/Yelp/detect-secrets/pull/568
[569]: https://github.com/Yelp/detect-secrets/pull/569
[571]: https://github.com/Yelp/detect-secrets/pull/571
[574]: https://github.com/Yelp/detect-secrets/pull/574
[575]: https://github.com/Yelp/detect-secrets/pull/575
[576]: https://github.com/Yelp/detect-secrets/pull/576
[578]: https://github.com/Yelp/detect-secrets/pull/578
[579]: https://github.com/Yelp/detect-secrets/pull/579

February 16th, 2022

:mega: Release Highlights
- Continuous integration github action added ([506])
- Release pipeline github action added ([513])

:tada: New Features

- New GitHub token plugin added ([465])
- New SendGrid plugin added ([463])
- More new ignored file extensions

:bug: Bugfixes
- Fixes catastrophic backtracking for indirect reference heuristic ([509])
- Fixes pre-commit hook secret equality checking causing updates to baseline with no real changes - only a timestamp update ([507])
- Fixes python 3.8 failing to load plugins on windows and macos ([505])
- Fixes yaml transformer inline dictionary index out of bounds exceptions ([501])
- Fixes regex for slack url ([477])
- Fixes `AttributeError: 'PotentialSecret' object has no attribute 'line_number'` by safely falling back to 0 if line_number isn't present. ([476])([472])
- Fixes gibberish-detector current version
- Fixes filtering ordering in .secrets.baseline

:snake: Miscellaneous

- Updated README due hook failing to interpret filenames with spaces ([470])
- Add CI github action badge to README
- Development dependency bumps ([519])

[463]: https://github.com/Yelp/detect-secrets/pull/463
[465]: https://github.com/Yelp/detect-secrets/pull/465
[470]: https://github.com/Yelp/detect-secrets/pull/470
[472]: https://github.com/Yelp/detect-secrets/pull/472
[476]: https://github.com/Yelp/detect-secrets/pull/476
[477]: https://github.com/Yelp/detect-secrets/pull/477
[501]: https://github.com/Yelp/detect-secrets/pull/501
[505]: https://github.com/Yelp/detect-secrets/pull/505
[506]: https://github.com/Yelp/detect-secrets/pull/506
[507]: https://github.com/Yelp/detect-secrets/pull/507
[509]: https://github.com/Yelp/detect-secrets/pull/509
[513]: https://github.com/Yelp/detect-secrets/pull/513
[519]: https://github.com/Yelp/detect-secrets/pull/519

April 14th, 2021

:mega: Release Highlights

- New gibberish filter added ([416])
- Multiprocessing support, for faster scans! ([441])
- Support for scanning different directories (rather than the current directory) ([440])

:tada: New Features

- `KeywordDetector` supports whitespace secrets ([414])
- `KeywordDetector` now supports prefix/suffixed keywords, and accuracy updates
- Adding alphanumerical filter to ensure secrets have at least one letter/number in them ([428])
- New filter added for ignoring common lock files ([417])
- More new ignored file extensions
- Adding filter to ignore swagger files
- Added `audit --report` to extract secret values with a baseline
([387], thanks [pablosantiagolopez], [syn-4ck])

:telescope: Accuracy

- `KeywordDetector` now defaults to requiring quotes around secrets ([448])
- `KeywordDetector` now searches for more keywords ([430])

:bug: Bugfixes

- Filter caches are cleared when swapping between different `Settings` objects ([444])
- Upgrading baselines from <0.12 migrates `exclude` to `exclude-files` rather than `exclude-lines`
([446])

:snake: Miscellaneous

- More verbose logging, to help with debugging issues ([432])
- YAMLTransformer handles binary entries differently

[387]: https://github.com/Yelp/detect-secrets/pull/387
[414]: https://github.com/Yelp/detect-secrets/pull/414
[416]: https://github.com/Yelp/detect-secrets/pull/416
[417]: https://github.com/Yelp/detect-secrets/pull/417
[428]: https://github.com/Yelp/detect-secrets/pull/428
[430]: https://github.com/Yelp/detect-secrets/pull/430
[432]: https://github.com/Yelp/detect-secrets/pull/432
[440]: https://github.com/Yelp/detect-secrets/pull/440
[441]: https://github.com/Yelp/detect-secrets/pull/441
[444]: https://github.com/Yelp/detect-secrets/pull/444
[446]: https://github.com/Yelp/detect-secrets/pull/446
[448]: https://github.com/Yelp/detect-secrets/pull/448
[syn-4ck]: https://github.com/syn-4ck

February 26th, 2021

:bug: Bugfixes

- Fixes `SecretsCollection` subtraction method, to handle non-overlapping files.
- Fixes installation for Windows environments ([412], thanks [pablosantiagolopez])

[412]: https://github.com/Yelp/detect-secrets/pull/412
[pablosantiagolopez]: https://github.com/pablosantiagolopez