Scrapling

**This is an essential update for everyone to fully enjoy Scrapling as it's intended**

What's changed
1. Added the `-f` option to `scrapling install` to force reinstall browser dependencies. I recommend you do `scrapling install -f` now to enjoy the big speed performance `StealthyFetcher` just got with the new Camoufox browser version :)
2. Fixed a bug in `TextHandler` where slicing returned `TextHandlers` instead of `TextHandler` and fixed the type hint there (41 )
3. Fixed an issue where `scrapling install` might in some instances drop the user into a Python shell!

Thanks for all your support!

---

Big shoutout to our biggest Sponsor: [Scrapeless](https://www.scrapeless.com/?utm_source=github&utm_medium=ads&utm_campaign=scraping&utm_term=D4Vinci)
<a href="https://www.scrapeless.com/?utm_source=github&utm_medium=ads&utm_campaign=scraping&utm_term=D4Vinci"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/scrapeless.jpg" height="400" alt="Scrapeless Banner" ></a>

**This is an essential update for everyone to fully enjoy Scrapling as it's intended**

What's changed
1. Fixed a bug in `Fetcher` that made headers generated by the `stealthy_headers` argument overwrite some of the headers provided by the user like `Accept` (39 )
2. Improved the headers generation logic a bit so it should give a slight speed boost.

Thanks for all your support!

---

Shoutout to our biggest Sponsor: [Scrapeless](https://www.scrapeless.com/?utm_source=github&utm_medium=ads&utm_campaign=scraping&utm_term=D4Vinci)
<a href="https://www.scrapeless.com/?utm_source=github&utm_medium=ads&utm_campaign=scraping&utm_term=D4Vinci"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/scrapeless.jpg" height="400" alt="Scrapeless Banner" ></a>

**This is an essential update for everyone to fully enjoy Scrapling as it's intended**

What's changed
1. Added the `history` property to all fetchers to show redirections (32 )
2. Fixed the logic of the `case_sensitive` argument logic for all `re`/`re_first`. This may make your code return different results if you were using it (but you probably deserve it because you noticed it wasn't working as intended and didn't open an issue LOL)
3. Updated dependencies and enabled `coop` back again in the Camoufox engine (StealthyFetcher).

Thanks for all your support!

---

Shoutout to our biggest Sponsor: [Scrapeless](https://www.scrapeless.com/?utm_source=github&utm_medium=ads&utm_campaign=scraping&utm_term=D4Vinci)
<a href="https://www.scrapeless.com/?utm_source=github&utm_medium=ads&utm_campaign=scraping&utm_term=D4Vinci"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/scrapeless.jpg" height="500" alt="Scrapeless Banner" ></a>

**This is an essential update for everyone to fully enjoy Scrapling as it's intended**

What's changed
1. The return type is now consistent across all the parser engine so you will always get a return type as one of these `Adaptor`, `Adaptors`, `TextHandler`, `TextHandlers`, `None`, and a list in case you have mixed results like combined CSS selector. This allows a better coding experience with minimum manual type checking, makes the library more stable, and makes chaining methods always possible.
2. Most of the parser engine especially the `Adaptor` class got refactored to a cleaner version and most importantly a faster version. So now almost all the methods/properties, especially the searching methods, got a speed increase between 5-40%. Some methods got bigger speed boosts like `find_by_regex` got a ~60% speed boost! The automatch feature got a small ~5% speed boost.
3. Fixed logic bugs with the `find_all`/`find` methods that made the passed filters used in OR fashion and other times as an AND. So now all elements returned need to fulfill all filters you pass.
4. Now all regex-related methods return `TextHandler`/`TextHandlers` for easier methods chaining.
5. **Added a new** `below_elements` property that returns an `Adaptors` object of all elements under the current element in the DOM tree.
6. Now all methods/properties that were returning HTML source as string are now returning it as `TextHandler` so you can do regex easily on it etc...
7. StealthyFetcher is now a bit faster and more stealthy. Also, now it's possible to click Captchas in iframes like **Cloudflare Turnstile**.
8. The auto-completion and type hints improved a lot in nearly half the library. Especially `Adaptor`, `TextHandler`, and `TextHandlers`.
9. Now slicing `TextHandler`, accessing by index, or using the `split` method returns another `TextHandler` instead of the standard Python string. Now almost all standard string operations/methods return other `Texthandler` instead of standard string to make chaining methods/functions always possible.
10. Fixed some small bugs and typos. For example, the Fetcher async_put was doing post request instead of put request 😶‍🌫️
11. Improved the README a bit till I finish the documentation website.

This was supposed to be a small update till version 0.3 but thought to make it better.

Thanks for all your support!

---

Shoutout to our biggest Sponsor: [Scrapeless](https://www.scrapeless.com/?utm_source=github&utm_medium=ads&utm_campaign=scraping&utm_term=D4Vinci)
<a href="https://www.scrapeless.com/?utm_source=github&utm_medium=ads&utm_campaign=scraping&utm_term=D4Vinci"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/scrapeless.jpg" height="500" alt="Scrapeless Banner" ></a>

What's changed
- Now response returned by browser-based fetchers uses more reliable data sources in cases where the page loaded uses many Iframes.
- **Now installing `Scrapling` is made even easier, you install it with pip then run `scrapling install` in the terminal and you are ready!**
- Fixed an inaccurate type hint in the parser.

---

> [!NOTE]
> A friendly reminder that maintaining and improving `Scrapling` takes a lot of time and effort which I have been happily doing for months even though it's becoming harder. So, if you like `Scrapling` and want it to keep improving, you can help by supporting me through the [Sponsor button](https://github.com/sponsors/D4Vinci).

What's changed
- Fixed a bug where the logging fetch logging sentence was showing in the first request only.
- The default behavior for Playwright API while browsing a page is returning the first response that fulfills the load state given to the `goto` method `["load", "domcontentloaded", "networkidle"]` so if a website has a wait page like Cloudflare's one that redirects you to the real website afterward, Playwright will return the first status code which in this case would be something like 403. This update solves this issue for both `PlaywrightFetcher` and `StealthyFetcher` as both are using Playwright API so the result depends on Playwright's default behavior no more.
- Added support for proxies that use SOCKS proxies in the `Fetcher` class.
- Fixed the type hint for the `wait_selector_state` argument so now it will show the accurate values you should use while auto-completing.

---

> [!NOTE]
> A friendly reminder that maintaining and improving `Scrapling` takes a lot of time and effort which I have been happily doing for months even though it's becoming harder. So, if you like `Scrapling` and want it to keep improving, you can help by supporting me through the [Sponsor button](https://github.com/sponsors/D4Vinci).