Robots Txt File For WordPress Blog

Many people are asking what is the best and common robots.txt file for a wordpress site/blog?

So here is the answer below. Copy this and paste in robots.txt file and save it in your wordpress root folder.

User-agent: *
Disallow: /cgi-bin
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-content
Disallow: /tag
Disallow: /author
Disallow: /wget/
Disallow: /httpd/
Disallow: /i/
Disallow: /f/
Disallow: /t/
Disallow: /c/
Disallow: /j/

User-agent: Mediapartners-Google
Allow: /

User-agent: AdsBot-Google
Disallow: /

User-agent: Googlebot-Image
Allow: /

User-agent: Googlebot-Mobile
Disallow: /

User-agent: ia_archiver-web.archive.org
Disallow: /
Sitemap: https://www.freedigitalworld.com/sitemap.xml

#NOte: please replace freedigitalworld.com with your sitename..

 

Leave a Reply

Your email address will not be published. Required fields are marked *

You May Also Like