Advertools

-------------------

* Fixed
- Ensure meta crawl data included in URLs crawled by following links.

-------------------

* Added
- Enable the ``meta`` parameter for the crawl function for: arbitrary metadata,
custom request headers, and 3rd party plugins like playwright.
* Changed
- Raise an error when supplying a custom log format with supplying fields.

-------------------

* Fixed
- Make file path for ``emoji_df`` relative to advertools ``__path__``.
- Allow the extension ``.jsonl`` for crawling.

-------------------

* Added
- Enable supplying request headers in ``sitemap_to_df``, contributed by `joejoinerr <https://github.com/joejoinerr>`_
- New function ``crawlytics.compare`` for comparing two crawls.
- New function ``crawlytics.running_crawls`` for getting data on currently running crawl jobs (\*NIX only for now).
- New parameter ``date_format`` to ``logs_to_df`` for custom date formats.

* Changed
- Removed the `relatedSite` parameter from ``serp_goog`` - deprecated.
- Update emoji regex and functionality to v15.1.

* Fixed
- Use int64 instead of int for YouTube count columns, contributed by `DanielP77 <https://github.com/DanielP77>`_

-------------------

* Fixed
- Use ``pd.NA`` instead of ``np.nan`` for empty values in ``url_to_df``.

-------------------

* Changed
- Use a different XPath expression for `body_text` while crawling.