Ad

Robots.txt: How To Disallow All Subdirectories But Allow Parent Folder

- 1 answer

I'd like to disallow all subdirectories in my folder /search but allow indexing the search folder itself (I have content on /search).

Testing this does not work:

User-Agent: *
Allow: /search/
Disallow: /search/*
Ad

Answer

Your code appears correct. Try with a slight adjustment to Allow:

User-Agent: *
Disallow: /search/*
Allow: /search/$
Ad
source: stackoverflow.com
Ad