Need to block search engine crawlers without setting up extra plugins or external tools? You’ve come to the right place.
A new feature we release helps you do that, right from the Webdock dashboard!
Note: This feature is available only for a server running our stack images (LEMP / LAMP)
How to do it?
Very simple.
Click on the pencil icon you see near the Site Web Root field on the server screen. This will take you to a brand new screen where you can manage access to your site.
Then look for the “Search Engine Crawling” card. Here you can configure access to crawling bots. You can either block access entirely or have a custom robots.txt file with your own access rules.
To setup custom robots text file, click on the “custom robots.txt file” link.
That’s it.
Hope this helps and takes away some of the headaches 🙂
