If you don’t want your website to be audited using Insites, you can block the Insites crawler by adding a specific directive to your website’s robots.txt file.
Add the Insites directive
Open your existing robots.txt file or create a new plain-text file named robots.txt. Add the following lines:
User-agent: Insites
Disallow: /
The User-agent line identifies the Insites crawler, while Disallow: / tells Insites not to access any pages on your website.
Publish the robots.txt file
Upload the file to the top-level directory of your website. It should be available at an address like:
https://www.example.com/robots.txt
If your website already has a robots.txt file, add the Insites directive as a separate group without removing any existing rules.
Important
Insites must be named specifically in the User-agent directive. A wildcard rule using User-agent: * will not prevent Insites from auditing your website.
Once the specific directive has been published, Insites will respect it and will not audit the website.