Scrapling

**This is an essential update for everyone to fully enjoy Scrapling as it's intended**

What's changed
Various memory usage and speed optimizations
- Now all CSS selection methods' memory usage is lower by ~145% and the speed slightly increased.
- Implemented Lazy loading for all submodules of the library so now what you use is what you load, for example:
Before the update this import `from scrapling import Adaptor` was using `30-40mb` of RAM because it loaded all fetchers and stuff with it too, now it uses `~1.2mb`.
- The last update made the library use ~32% memory it used before with a large requests pool, now we adjusted the caching further to use even less than that.
- Overall speed increase in the parser by a slight 2-5%

**_Thanks for all your support and donations!_**

---

Big shoutout to our biggest Sponsor: [Scrapeless](https://www.scrapeless.com/en/product/deep-serp-api?utm_source=website&utm_medium=ads&utm_campaign=scraping&utm_term=d4vinci)
<a href="https://www.scrapeless.com/en/product/deep-serp-api?utm_source=website&utm_medium=ads&utm_campaign=scraping&utm_term=d4vinci"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/scrapeless.jpg" height="400" alt="Scrapeless Banner" ></a>

**This is an essential update for everyone to fully enjoy Scrapling as it's intended**

What's changed
Lower memory usage and small speed increase across all Fetchers.
- With new limitations across the library over caching size you will notice significantly lower memory usage than before while doing large numbers of requests/operations.
- Refactored big parts of the fetchers to easier maintainability and small speed increase.

Bugs fixed
- Fixed a bug in `TextHandler` where importing it alone and passing a non-string value converts it to an empty string. Now anything passed to `TextHandler` is automatically converted to a string before being converted to `TextHandler`, this is forced on any value passed -- `TextHandler` as the name implies is intended to work with strings only after all! (45 )
- Fixed a bug where the `retries` arguments weren't taken into account in most AsyncFetcher methods.

Miscellaneous
- Update type hints for most arguments in all fetchers to be clearer and more accurate.

**_Thanks for all your support and donations!_**

---

Big shoutout to our biggest Sponsor: [Scrapeless](https://www.scrapeless.com/en/product/deep-serp-api?utm_source=website&utm_medium=ads&utm_campaign=scraping&utm_term=d4vinci)
<a href="https://www.scrapeless.com/en/product/deep-serp-api?utm_source=website&utm_medium=ads&utm_campaign=scraping&utm_term=d4vinci"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/scrapeless.jpg" height="400" alt="Scrapeless Banner" ></a>

**This is an essential update for everyone to fully enjoy Scrapling as it's intended**

What's changed
1. Added the `-f` option to `scrapling install` to force reinstall browser dependencies. I recommend you do `scrapling install -f` now to enjoy the big speed performance `StealthyFetcher` just got with the new Camoufox browser version :)
2. Fixed a bug in `TextHandler` where slicing returned `TextHandlers` instead of `TextHandler` and fixed the type hint there (41 )
3. Fixed an issue where `scrapling install` might in some instances drop the user into a Python shell!

Thanks for all your support!

---

Big shoutout to our biggest Sponsor: [Scrapeless](https://www.scrapeless.com/en/product/deep-serp-api?utm_source=website&utm_medium=ads&utm_campaign=scraping&utm_term=d4vinci)
<a href="https://www.scrapeless.com/en/product/deep-serp-api?utm_source=website&utm_medium=ads&utm_campaign=scraping&utm_term=d4vinci"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/scrapeless.jpg" height="400" alt="Scrapeless Banner" ></a>

**This is an essential update for everyone to fully enjoy Scrapling as it's intended**

What's changed
1. Fixed a bug in `Fetcher` that made headers generated by the `stealthy_headers` argument overwrite some of the headers provided by the user like `Accept` (39 )
2. Improved the headers generation logic a bit so it should give a slight speed boost.

Thanks for all your support!

---

Shoutout to our biggest Sponsor: [Scrapeless](https://www.scrapeless.com/?utm_source=github&utm_medium=ads&utm_campaign=scraping&utm_term=D4Vinci)
<a href="https://www.scrapeless.com/?utm_source=github&utm_medium=ads&utm_campaign=scraping&utm_term=D4Vinci"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/scrapeless.jpg" height="400" alt="Scrapeless Banner" ></a>

**This is an essential update for everyone to fully enjoy Scrapling as it's intended**

What's changed
1. Added the `history` property to all fetchers to show redirections (32 )
2. Fixed the logic of the `case_sensitive` argument logic for all `re`/`re_first`. This may make your code return different results if you were using it (but you probably deserve it because you noticed it wasn't working as intended and didn't open an issue LOL)
3. Updated dependencies and enabled `coop` back again in the Camoufox engine (StealthyFetcher).

Thanks for all your support!

---

Shoutout to our biggest Sponsor: [Scrapeless](https://www.scrapeless.com/?utm_source=github&utm_medium=ads&utm_campaign=scraping&utm_term=D4Vinci)
<a href="https://www.scrapeless.com/?utm_source=github&utm_medium=ads&utm_campaign=scraping&utm_term=D4Vinci"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/scrapeless.jpg" height="500" alt="Scrapeless Banner" ></a>

**This is an essential update for everyone to fully enjoy Scrapling as it's intended**

What's changed
1. The return type is now consistent across all the parser engine so you will always get a return type as one of these `Adaptor`, `Adaptors`, `TextHandler`, `TextHandlers`, `None`, and a list in case you have mixed results like combined CSS selector. This allows a better coding experience with minimum manual type checking, makes the library more stable, and makes chaining methods always possible.
2. Most of the parser engine especially the `Adaptor` class got refactored to a cleaner version and most importantly a faster version. So now almost all the methods/properties, especially the searching methods, got a speed increase between 5-40%. Some methods got bigger speed boosts like `find_by_regex` got a ~60% speed boost! The automatch feature got a small ~5% speed boost.
3. Fixed logic bugs with the `find_all`/`find` methods that made the passed filters used in OR fashion and other times as an AND. So now all elements returned need to fulfill all filters you pass.
4. Now all regex-related methods return `TextHandler`/`TextHandlers` for easier methods chaining.
5. **Added a new** `below_elements` property that returns an `Adaptors` object of all elements under the current element in the DOM tree.
6. Now all methods/properties that were returning HTML source as string are now returning it as `TextHandler` so you can do regex easily on it etc...
7. StealthyFetcher is now a bit faster and more stealthy. Also, now it's possible to click Captchas in iframes like **Cloudflare Turnstile**.
8. The auto-completion and type hints improved a lot in nearly half the library. Especially `Adaptor`, `TextHandler`, and `TextHandlers`.
9. Now slicing `TextHandler`, accessing by index, or using the `split` method returns another `TextHandler` instead of the standard Python string. Now almost all standard string operations/methods return other `Texthandler` instead of standard string to make chaining methods/functions always possible.
10. Fixed some small bugs and typos. For example, the Fetcher async_put was doing post request instead of put request 😶‍🌫️
11. Improved the README a bit till I finish the documentation website.

This was supposed to be a small update till version 0.3 but thought to make it better.

Thanks for all your support!

---

Shoutout to our biggest Sponsor: [Scrapeless](https://www.scrapeless.com/?utm_source=github&utm_medium=ads&utm_campaign=scraping&utm_term=D4Vinci)
<a href="https://www.scrapeless.com/?utm_source=github&utm_medium=ads&utm_campaign=scraping&utm_term=D4Vinci"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/scrapeless.jpg" height="500" alt="Scrapeless Banner" ></a>