Agence de Développement Informatique du Nord » robots.txt http://blog.adin.pro Just another WordPress site Thu, 26 Dec 2019 08:54:31 +0000 en-US hourly 1 http://wordpress.org/?v=4.1.1 Robots.txt disallow all http://blog.adin.pro/2013-05-09/robots-txt-disallow-all/ http://blog.adin.pro/2013-05-09/robots-txt-disallow-all/#comments Thu, 09 May 2013 08:56:52 +0000 http://blog.adin.pro/?p=180 To prevent google to reference your dev website, you can add a robots.txt file to your web root directory and fill it with The “User-agent: *” means that the file applies to all robots. The “Disallow: /” means the robot … Continue reading →

The post Robots.txt disallow all appeared first on Agence de Développement Informatique du Nord.

]]> To prevent google to reference your dev website, you can add a robots.txt file to your web root directory and fill it with

User-agent: *
Disallow: /

The “User-agent: *” means that the file applies to all robots. The “Disallow: /” means the robot will not visit any pages on the website.

The post Robots.txt disallow all appeared first on Agence de Développement Informatique du Nord.

]]> http://blog.adin.pro/2013-05-09/robots-txt-disallow-all/feed/ 0