Top Guidelines Of dana
the worth in the name attribute (robots) specifies which the rule applies to all crawlers. both of those the identify as well as the content material attributes are circumstance-insensitive. To address a particular crawler, change the robots worth of the title attribute With all the consumer agent token from the crawler that you are addressing. Goo