Ad

Asterisk In Robots.txt

- 1 answer

Wondering if following will work for google in robots.txt

Disallow: /*.action

I need to exclude all urls ending with .action.

Is this correct?

Ad

Answer

To block files of a specific file type (for example, .gif), use the following:

User-agent: Googlebot
Disallow: /*.gif$

So, you are close. Use Disallow: /*.action$ with a trailing "$"

Of course, that's merely what Google suggests: http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449

All bots are different.

Ad
source: stackoverflow.com
Ad