Pyrobotstxt

Latest version: v0.0.5

Safety actively analyzes 685670 Python packages for vulnerabilities to keep your Python projects secure.

0.0.3

List of Updates

- Updated Documentation
- Support for Crawl_delay: See 1 for more details
- Working Example. See 3 for more details

0.0.2

Similar to v0.0.1 but now available to pipy for public release.

0.0.1

Basic Robots.txt File creation Functionality

You can do the followings.

- Add Header/footer Comments
- Add Creation Date
- Add and Remove Allowed Items/Pages/Routes
- Add and Remove Disallowed Items/Pages/Routes
- Add/Remove Sitemaps
- Include Fancy ASCII Images

Releases

Has known vulnerabilities

0.0.3
0.0.2
0.0.1