Internet review - Lite Web-master Graphics Games Freeware Time Christmas Bible
Website Hosting Clouds Domains Resources Forms on the site Promotion
robots.txt SEO Validators Webmasters Internet Keywords Test Promotion Advertising Resources Internet speed Links Statistics Tests

Create robots.txt

robots.txt
Specifies to search robots what catalogs to take for indexing should not be. If it is empty or does not exist, then everything can be taken.
Details about robots.txt

Example

User-agent: *
Disallow:/news
# Prohibit all robots from indexing links,
# That start with/news

User-agent: StackRambler
User-agent: Aport
Disallow:/eng
Disallow:/news
#Rambler and Aport to forbid indexing of links,
# That start with/news and/eng

User-agent: Yandex
Disallow:
#Yandex alloud all.

FILE GENERATOR of ROBOTS.TXT

User-agent:
Disallow:

Result

META tag ROBOTS


Result highlight

Analyze robots.txt

  1. google.com/webmasters/tools/robots-testi... - Google, Robots.txt file verification tool
  2. websiteplanet.com/webtools/robots-txt/ - Website Planet

Webmasters

  1. webmaster.yandex.com/tools/robotstxt - Analyze robots.txt - Here you can find out what's wrong with us, if we have something wrong in the robots.txt file, learn more about robots.txt.

For a web master What are looking for? Site traffic statistics META tags .htaccess CHMOD

Mail Form Secure Email Colors in HTML Determinant of download speed Builder RSS stream in JavaScript Encrypt HTML in javascript



Mobile version

Terms of publication of the article
Advertising
About us
Graphics

Fonts
Logos
Brandbooks
Pictogramms
Heraldry

Popular

Check a website level
A website registration
How to creat a website
#1 on Google
Online Translators
Password

Internet top

©2005-2024, Web studio Ph4 - Internet Catalog for user, web-master and designer v. 6.0.3