Whoogle-search

It's been a while since the last official release! There have been some changes made since 0.7.4 that are fairly noteworthy, including:

- Complete overhaul of session management
- Fixed runaway session file creation for public instance maintainers
- Fixed failure to decrypt subsequent result page queries (i.e. page 2 query appearing as "gAAAAA....")
- Added ability to navigate through results with Tab and Shift+Tab
- Added an "Advanced Search" toggle in search results for quickly switching between countries
- Can be updated to support other search params, if requested
- Updated filter for sponsored results/advertising content
- Configured language + country is now included in search suggestion requests
- Fixed issues with bolding search terms in result pages
- Added ability to encode configuration in the URL
- Allows jumping between Whoogle instances (via Farside, for example) and maintaining the same configuration settings
- Fixed issues with result links to Google domains
- Reduced the amount of logging the Tor service outputs by default
- New translations:
- Thai
- Kurdish
- Welsh

And much more!

SourceHut

There's an official "mirror" of the repository on SourceHut at: https://sr.ht/~benbusby/whoogle-search now. It's not technically a "mirror" though, since it does more than just look for changes on GitHub. It's automatically synced with changes pushed to GitHub, but GitHub is also synced with any changes pushed to SourceHut. The two repos are completely interchangeable from a user perspective. If you prefer using SourceHut, feel free to mail in patches, open tickets, etc. If not, feel free to keep using GitHub.

Community Contributions
* Tor password authentication by MadcowOG in https://github.com/benbusby/whoogle-search/pull/746
* Fix a crash with protected Tor control port by CyberTailor in https://github.com/benbusby/whoogle-search/pull/785
* Improve newtab by jacr13 in https://github.com/benbusby/whoogle-search/pull/790
* Remove wildcard imports by jacr13 in https://github.com/benbusby/whoogle-search/pull/791
* Update translations.json by CAB233 in https://github.com/benbusby/whoogle-search/pull/794
* Add more OpenSearch definitions (for images etc.) by CyberTailor in https://github.com/benbusby/whoogle-search/pull/786
* Fixes handling of maps by jacr13 in https://github.com/benbusby/whoogle-search/pull/792
* Use X-Forwarded-Host as url_root when present by marekful in https://github.com/benbusby/whoogle-search/pull/799
* Tor Documentation by MadcowOG in https://github.com/benbusby/whoogle-search/pull/800
* added instance (search.wef.lol) by J2D9 in https://github.com/benbusby/whoogle-search/pull/807
* added new instance to misc/instances.txt by J2D9 in https://github.com/benbusby/whoogle-search/pull/808
* Tor password newline fix by MadcowOG in https://github.com/benbusby/whoogle-search/pull/801
* Fix typos by kianmeng in https://github.com/benbusby/whoogle-search/pull/813
* Fixes read_config_bool to allow several true params by jacr13 in https://github.com/benbusby/whoogle-search/pull/841
* Add Kurdish Translation by CYAXXX in https://github.com/benbusby/whoogle-search/pull/837
* Fixes google unsuported links bad internal redirection by jacr13 in https://github.com/benbusby/whoogle-search/pull/850
* Replaces default reading of environment by jacr13 in https://github.com/benbusby/whoogle-search/pull/848
* Add filter for ads in Vietnamese by BienAtGitHub in https://github.com/benbusby/whoogle-search/pull/847
* Add preferences parameter to url by jacr13 in https://github.com/benbusby/whoogle-search/pull/842
* Fix handling of bangs by jacr13 in https://github.com/benbusby/whoogle-search/pull/851
* Update lang_th by watchakorn-18k in https://github.com/benbusby/whoogle-search/pull/856
* Add ads thai in BLACKLIST by watchakorn-18k in https://github.com/benbusby/whoogle-search/pull/857
* Add ~vern instances (clearnet, onion, i2p) by gi-yt in https://github.com/benbusby/whoogle-search/pull/858
* Removal of search.wef.lol by Albonycal in https://github.com/benbusby/whoogle-search/pull/860
* Updating Readme.md ; Heroku fix and Removed Some Non-Working Public Instances by hrishikeshp7 in https://github.com/benbusby/whoogle-search/pull/864
* Fix autocomplete by jacr13 in https://github.com/benbusby/whoogle-search/pull/866
* changes regex to replace special characters and not chinese characters. Issue-845 by Abir10101 in https://github.com/benbusby/whoogle-search/pull/865
* Fix error preferences by jacr13 in https://github.com/benbusby/whoogle-search/pull/867
* Fix missing args in docstring by jacr13 in https://github.com/benbusby/whoogle-search/pull/872
* Fix color for light contrast text by jacr13 in https://github.com/benbusby/whoogle-search/pull/873
* Update zh-tw translation by xatier in https://github.com/benbusby/whoogle-search/pull/875
* Added Welsh Translation by curlpipe in https://github.com/benbusby/whoogle-search/pull/876
* Include new ad filter keyword by xaabi6 in https://github.com/benbusby/whoogle-search/pull/879

New Contributors
* MadcowOG made their first contribution in https://github.com/benbusby/whoogle-search/pull/746
* CAB233 made their first contribution in https://github.com/benbusby/whoogle-search/pull/794
* marekful made their first contribution in https://github.com/benbusby/whoogle-search/pull/799
* J2D9 made their first contribution in https://github.com/benbusby/whoogle-search/pull/807
* kianmeng made their first contribution in https://github.com/benbusby/whoogle-search/pull/813
* CYAXXX made their first contribution in https://github.com/benbusby/whoogle-search/pull/837
* BienAtGitHub made their first contribution in https://github.com/benbusby/whoogle-search/pull/847
* watchakorn-18k made their first contribution in https://github.com/benbusby/whoogle-search/pull/856
* gi-yt made their first contribution in https://github.com/benbusby/whoogle-search/pull/858
* hrishikeshp7 made their first contribution in https://github.com/benbusby/whoogle-search/pull/864
* Abir10101 made their first contribution in https://github.com/benbusby/whoogle-search/pull/865
* xaabi6 made their first contribution in https://github.com/benbusby/whoogle-search/pull/879

**Full Changelog**: https://github.com/benbusby/whoogle-search/compare/v0.7.4...v0.8.0

Very small, but important update for dark mode users.

Google made changes to the stylesheet returned in the result view for Whoogle search results, which broke certain aspects of dark mode themes. This version should fix the issues introduced by their changes and return the overall styling of Whoogle back to its normal appearance.

Minor update, with a few fixes:
- Instances of Whoogle installed via `pip` should no longer encounter the `UnpicklingError` issue
- Fixed "next page" button for searches with a leading slash (i.e. searching `/e/OS apps`)
- Fixed endpoints with leading slashes for instances deployed from sub path of a domain (i.e. `domain.com/whoogle`)
- Fixed broken behavior for the "open in new tab" config option

Community Contributions
* Fix pipx run by SuperSandro2000 in https://github.com/benbusby/whoogle-search/pull/738
* Remove "/" before endpoints & tags by invis-z in https://github.com/benbusby/whoogle-search/pull/734
* Add WHOOGLE_URL_PREFIX to app.json by invis-z in https://github.com/benbusby/whoogle-search/pull/737
* Update zh-tw translation by xatier in https://github.com/benbusby/whoogle-search/pull/736
* Update translations.json by hoschi1337 in https://github.com/benbusby/whoogle-search/pull/742
* Fix link in window by invis-z in https://github.com/benbusby/whoogle-search/pull/748
* Update Garudalinux instance, add dr460nf1r3.org instance by dr460nf1r3 in https://github.com/benbusby/whoogle-search/pull/750
* Fix image view by invis-z in https://github.com/benbusby/whoogle-search/pull/762
* Fix urls not open in new tab by jacr13 in https://github.com/benbusby/whoogle-search/pull/769
* added instance whoogle.privacydev.net by PrivacyDevel in https://github.com/benbusby/whoogle-search/pull/764
* Bump waitress from 2.1.1 to 2.1.2 by dependabot in https://github.com/benbusby/whoogle-search/pull/773

New Contributors
* SuperSandro2000 made their first contribution in https://github.com/benbusby/whoogle-search/pull/738
* invis-z made their first contribution in https://github.com/benbusby/whoogle-search/pull/734
* hoschi1337 made their first contribution in https://github.com/benbusby/whoogle-search/pull/742
* PrivacyDevel made their first contribution in https://github.com/benbusby/whoogle-search/pull/764

**Full Changelog**: https://github.com/benbusby/whoogle-search/compare/v0.7.2...v0.7.3

Features/Improvements
- Build images now pushed to [ghcr.io](https://github.com/benbusby/whoogle-search/pkgs/container/whoogle-search) (still pushed to Docker Hub as usual)
- Improved "minimal mode"
- Public instances now accessible in machine readable format via [misc/instances.txt](https://github.com/benbusby/whoogle-search/blob/main/misc/instances.txt)
- Fixed bugs w/ currency conversion
- Upgraded Python image in Dockerfile to `python:3.11.0a5-alpine`
- Updated ad filter
- Switched to [defusedxml](https://pypi.org/project/defusedxml/) for XML parsing
- Added ability to enable frontend alternatives, but selectively keep the default frontend for specific services
- For example, if site alts are enabled but you still want to visit `youtube.com` for YouTube results, set `WHOOGLE_ALT_YT` to an empty value (i.e. `WHOOGLE_ALT_YT=""` (all alt environment variables are documented in the README).
- Updated all remaining frontend alternatives to be redirected through [Farside](https://github.com/benbusby/farside) by default
- Fixed incorrect Chinese and Russian translations
- Added Korean translations
- Improved support for relative search results
- Note: If you're hosting Whoogle behind a path such as `mydomain.com/whoogle`, you can now set `WHOOGLE_URL_PREFIX` to the path prefix (in this example, `/whoogle`)
- Patched a minor vulnerability involving XSS on the Whoogle error template
- Passing valid javascript within a tag (i.e. "<script>alert(document.domain)</script>") to the `q` param for the (now removed) `/url` endpoint caused the javascript to be executed. This has been fixed by restoring message sanitizing on the error template, as well as removing the unused `/url` endpoint which provided the only way of accessing the bug.
- Added a (rough) implementation of Anonymous View
- Available as a config setting or environment variable (`WHOOGLE_CONFIG_ANON_VIEW`)
- Partially overrides the NoJS feature in order to keep result clutter to a minimum
- NoJS View is now accessible only if both Anonymous View and NoJS are enabled in the config, since it uses the same endpoint as Anonymous View
- This still has some issues to work through, but is available as a "beta" feature to test out now. Feel free to open an issue or reach out directly (or come to [my Twitch stream](https://twitch.tv/ben_busby)) if you find a specific site that anonymous view doesn't work with, or otherwise find a bug with it in general.

Community Contributions
* Update minimal mode for new Google formatting by DUOLabs333 in https://github.com/benbusby/whoogle-search/pull/637
* දෝෂ කිහිපයක් නිවරදි කිරීම by sayuri-gi in https://github.com/benbusby/whoogle-search/pull/594
* Improve rendering of the tabs by jacr13 in https://github.com/benbusby/whoogle-search/pull/535
* Clean "Show more results" of all site blocks by DUOLabs333 in https://github.com/benbusby/whoogle-search/pull/646
* added my instance :) by EsmailELBoBDev2 in https://github.com/benbusby/whoogle-search/pull/647
* Add gowogle.voring.me as public instance by ThatOneCalculator in https://github.com/benbusby/whoogle-search/pull/650
* Fix error with `remove_site_blocks` in the Images tab by DUOLabs333 in https://github.com/benbusby/whoogle-search/pull/651
* Fix 'collapse_sections' for 'MINIMAL_MODE' by DUOLabs333 in https://github.com/benbusby/whoogle-search/pull/654
* Give Accept-Language div its own class by nityy in https://github.com/benbusby/whoogle-search/pull/659
* Vulnerable Python image upgraded (critical) by Albonycal in https://github.com/benbusby/whoogle-search/pull/669
* Configure setup() using setup.cfg by CyberTailor in https://github.com/benbusby/whoogle-search/pull/667
* Increase /var/lib/tor tmpfs size to 12MB by CyberJack in https://github.com/benbusby/whoogle-search/pull/693
* whoogle.dcs0.hu cloudflare remove by domokosdcs0 in https://github.com/benbusby/whoogle-search/pull/696
* Fix incorrect translation (zh-TW & zh-CN) by xatier in https://github.com/benbusby/whoogle-search/pull/697
* Add "nofollow noopener noreferrer" to all links by 138138138 in https://github.com/benbusby/whoogle-search/pull/698
* add korean translation by green1052 in https://github.com/benbusby/whoogle-search/pull/700
* Do not offer opensearch.xml as attachment by gdm85 in https://github.com/benbusby/whoogle-search/pull/713
* [Chrome] Mention requirements to add a search engine via OpenSearch by gdm85 in https://github.com/benbusby/whoogle-search/pull/716
* Fix 'anon-view' KeyError by glitsj16 in https://github.com/benbusby/whoogle-search/pull/724
* Fix Russian translation by dsrev in https://github.com/benbusby/whoogle-search/pull/726
* Return 401 when token is invalid by gdm85 in https://github.com/benbusby/whoogle-search/pull/714
* Add support for relative search results by gdm85 in https://github.com/benbusby/whoogle-search/pull/715
* Fixes issue where 307 redirects on http not https by spitsw in https://github.com/benbusby/whoogle-search/pull/731

New Contributors
* EsmailELBoBDev2 made their first contribution in https://github.com/benbusby/whoogle-search/pull/647
* ThatOneCalculator made their first contribution in https://github.com/benbusby/whoogle-search/pull/650
* CyberTailor made their first contribution in https://github.com/benbusby/whoogle-search/pull/667
* CyberJack made their first contribution in https://github.com/benbusby/whoogle-search/pull/693
* xatier made their first contribution in https://github.com/benbusby/whoogle-search/pull/697
* 138138138 made their first contribution in https://github.com/benbusby/whoogle-search/pull/698
* green1052 made their first contribution in https://github.com/benbusby/whoogle-search/pull/700
* gdm85 made their first contribution in https://github.com/benbusby/whoogle-search/pull/713
* spitsw made their first contribution in https://github.com/benbusby/whoogle-search/pull/731

**Full Changelog**: https://github.com/benbusby/whoogle-search/compare/v0.7.1...v0.7.2

There are a few new Docker related improvements:
- Removed unneeded dependencies (cut compressed image size by ~50%)
- Image uses `whoogle` user, rather than `root` or `debian-tor`
- **WARNING**: If you are using `docker-compose` this release will break your instance unless you update your docker-compose.yml with the latest changes:
- https://github.com/benbusby/whoogle-search/blob/1af4566991f5248fd0d3576ca4fbcc0a84293004/docker-compose.yml#L14
- https://github.com/benbusby/whoogle-search/blob/1af4566991f5248fd0d3576ca4fbcc0a84293004/docker-compose.yml#L19-L22
- Tests for Docker and docker-compose have been added to the CI pipeline -- image tags will not be pushed/updated unless both tests pass.
- Nightly vulnerability scans for Docker images

There are also a number of various improvements and fixes provided by the community, defailed below. Thanks everyone!

Community Contributions
* Add missing environment variables by glitsj16 in https://github.com/benbusby/whoogle-search/pull/584
* Fix malformed systemd service file. by NunoSempere in https://github.com/benbusby/whoogle-search/pull/571
* alpine docker image by roylez in https://github.com/benbusby/whoogle-search/pull/573
* Updated screenshots by Albonycal in https://github.com/benbusby/whoogle-search/pull/583
* adding my instance! by domokosdcs0 in https://github.com/benbusby/whoogle-search/pull/604
* Add `WHOOGLE_ALT_IMG` for a replacement for imgur. by DUOLabs333 in https://github.com/benbusby/whoogle-search/pull/609
* Disable autocomplete.js if `WHOOGLE_AUTOCOMPLETE=0` by DUOLabs333 in https://github.com/benbusby/whoogle-search/pull/611
* Improve formatting of collapsible infobox by nityy in https://github.com/benbusby/whoogle-search/pull/612
* Allow Docker container to be run as non-root user by ras07 in https://github.com/benbusby/whoogle-search/pull/616
* Update Dockerfile by ras07 in https://github.com/benbusby/whoogle-search/pull/617
* Fix docker-compose.yml permission errors by nakoo in https://github.com/benbusby/whoogle-search/pull/623

New Contributors
* glitsj16 made their first contribution in https://github.com/benbusby/whoogle-search/pull/584
* NunoSempere made their first contribution in https://github.com/benbusby/whoogle-search/pull/571
* roylez made their first contribution in https://github.com/benbusby/whoogle-search/pull/573
* domokosdcs0 made their first contribution in https://github.com/benbusby/whoogle-search/pull/604
* nityy made their first contribution in https://github.com/benbusby/whoogle-search/pull/612
* ras07 made their first contribution in https://github.com/benbusby/whoogle-search/pull/616
* nakoo made their first contribution in https://github.com/benbusby/whoogle-search/pull/623

**Full Changelog**: https://github.com/benbusby/whoogle-search/compare/v0.7.0...v0.7.1

This release came around a little faster than normal, but still has some pretty worthwhile new features, improvements, and fixes.

Features/Improvements
- [Farside](https://github.com/benbusby/farside) integration
- This is a new service (developed by me) that provides a few new benefits that didn't quite fit into Whoogle itself, namely:
- "Smart" redirecting to available/working instances of alternative frontends like Nitter, Bibliogram, etc
- Allows quickly jumping to new Whoogle instances to resume your search if the one you're using is blocked
- All of your config settings should be parameterized in the generated Farside URL
- Switching to a valid Searx instance if your instance is blocked and there aren't any other Whoogle instances available (or you just feel like switching to Searx)
- Evenly distributing traffic to Nitter/Bibliogram/Libreddit instances rather than pooling users on popular instances and creating problems for the maintainer of the instance
- [Scribe](https://lists.sr.ht/~edwardloveall/scribe) replacements for Medium links
- New environment variables:
- `WHOOGLE_MINIMAL` to declutter the result page and only return "regular" result cards
- `WHOOGLE_RESULTS_PER_PAGE` to set the number of results per page (wow who would've guessed!)
- `WHOOGLE_AUTOCOMPLETE` defaults to "on", but can be set to `0` to turn off search suggestions
- Search terms are now **bold** in search results
- Support DDG bangs with the `!` at the end (for short-style bangs)
- Improved NoJS link formatting in results
- Instance now shows an "Update Available" message if the instance is not using the latest release
- Whoogle can now be deployed via Helm chart (https://github.com/benbusby/whoogle-search/pull/522)
- Improved public instance session management
- This is a dramatic improvement to how user sessions were handled previously. The new approach enforces that each new user to a Whoogle instance inherits an immutable set of default configuration parameters. Whoogle then validates if the user is allowing cookies to be set by forwarding them to a `/session` URL that checks for changes to their session ID. If the ID has changed, then the user likely has cookies disabled, and they are blocked from changing their config settings.
- This should help immensely on public instances where the default config was previously always changing.
- The "Country" config setting can now be used to emulate a search in that specified country
- Very useful for public instances that are hosted outside of your geographic location, but you still want relevant results for the country you live in

New Translations
- Russian

And here's a bulleted list of what's changed and who contributed! Thank you to everyone who has helped out. I really appreciate it.

What's Changed
* Implement scribe.rip replacement for medium.com by vacom13 in https://github.com/benbusby/whoogle-search/pull/463
* Fix typo by fredster33 in https://github.com/benbusby/whoogle-search/pull/478
* feat: block title or url by regex by YadominJinta in https://github.com/benbusby/whoogle-search/pull/473
* Feature to show clients ip address by vacom13 in https://github.com/benbusby/whoogle-search/pull/469
* Allow setting Accept-Language header by vacom13 in https://github.com/benbusby/whoogle-search/pull/483
* Add WHOOGLE_RESULTS_PER_PAGE by DUOLabs333 in https://github.com/benbusby/whoogle-search/pull/486
* Add a "Minimal Mode" by DUOLabs333 in https://github.com/benbusby/whoogle-search/pull/485
* Bold search query in results by DUOLabs333 in https://github.com/benbusby/whoogle-search/pull/487
* Ignore target_words that are not alphanumeric by DUOLabs333 in https://github.com/benbusby/whoogle-search/pull/496
* DDG style Bangs now support Bangs at the End by vacom13 in https://github.com/benbusby/whoogle-search/pull/503
* Change "skip bolding" check to regex to fix some edge cases by DUOLabs333 in https://github.com/benbusby/whoogle-search/pull/500
* Improve formatting with NoJS enabled by gripped in https://github.com/benbusby/whoogle-search/pull/509
* Remove css by gripped in https://github.com/benbusby/whoogle-search/pull/511
* Add Update Available warning by vacom13 in https://github.com/benbusby/whoogle-search/pull/517
* Fixes internal server error by fabianschilling in https://github.com/benbusby/whoogle-search/pull/521
* Added another public instance by KokoTheBest in https://github.com/benbusby/whoogle-search/pull/512
* New public instance added by Albonycal in https://github.com/benbusby/whoogle-search/pull/525
* feat: Simple Helm Chart by rblaine95 in https://github.com/benbusby/whoogle-search/pull/522
* Add Japanese translation by gnuhead-chieb in https://github.com/benbusby/whoogle-search/pull/528
* Fix for issue 299 by jacr13 in https://github.com/benbusby/whoogle-search/pull/534
* Improve public instance session management by benbusby in https://github.com/benbusby/whoogle-search/pull/480
* Add Russian translation by dsrev in https://github.com/benbusby/whoogle-search/pull/552
* Remove dead instance & add my onion instance by alefvanoon in https://github.com/benbusby/whoogle-search/pull/555
* docs(instance): search.sethforprivacy.com by curbengh in https://github.com/benbusby/whoogle-search/pull/562
* Currency Conversion by vacom13 in https://github.com/benbusby/whoogle-search/pull/536
* Farside integration by benbusby in https://github.com/benbusby/whoogle-search/pull/560

New Contributors
* vacom13 made their first contribution in https://github.com/benbusby/whoogle-search/pull/463
* YadominJinta made their first contribution in https://github.com/benbusby/whoogle-search/pull/473
* DUOLabs333 made their first contribution in https://github.com/benbusby/whoogle-search/pull/486
* fabianschilling made their first contribution in https://github.com/benbusby/whoogle-search/pull/521
* rblaine95 made their first contribution in https://github.com/benbusby/whoogle-search/pull/522
* gnuhead-chieb made their first contribution in https://github.com/benbusby/whoogle-search/pull/528
* dsrev made their first contribution in https://github.com/benbusby/whoogle-search/pull/552
* curbengh made their first contribution in https://github.com/benbusby/whoogle-search/pull/562

**Full Changelog**: https://github.com/benbusby/whoogle-search/compare/v0.6.0...v0.7.0