Configuring Crawler Settings

The Crawler Settings section in the Scans tab lets you control how CookieHub scans your site.

You can configure:

• Automatic Cookie Scans – Runs monthly to keep your cookie declaration updated.
• Include localStorage and sessionStorage – Detects storage items alongside cookies.
• Publish After Successful Scan – Updates your widget automatically after a scan.
• Reset Consent After Scan – Asks all users to give consent again after a scan.
• Entry URL – Where the scan starts (default is / ).
• Maximum Scanned Pages – Limit the scan for large sites.
• Included URL Prefixes – Scan only certain parts of your site.
• Excluded URL Prefixes – Skip certain parts of your site.
• Authentication – Allow scans behind login walls.