Ad
Robots.txt Cannot Be Located
I'm currently running a website (http://www.agents-world.com/) served by a node.js from heroku host. The problem is the robots.txt cannot be located whatever I do. I was wondering if heroku was caching something or more generally not deployment that file.
I've made changes to the app so I know Heroku deployed the latest version.
I would expect to see the content of the robots.txt file here http://www.agents-world.com/robots.txt
Any thoughts ?
Ad
Answer
It looks like you have no route pointing to robots.txt.
You have to serve this specific file, using for example an Express route:
app.get("/robots.txt", function(req, res){
res.sendFile(path.resolve("PATH_TO_ROBOTS.TXT"));
});
Ad
source: stackoverflow.com
Related Questions
- → Maximum call stack exceeded when instantiating class inside of a module
- → Browserify api: how to pass advanced option to script
- → Node.js Passing object from server.js to external modules?
- → gulp-rename makes copies, but does not replace
- → requiring RX.js in node.js
- → Remove an ObjectId from an array of objectId
- → Can not connect to Redis
- → React: How to publish page on server using React-starter-kit
- → Express - better pattern for passing data between middleware functions
- → Can't get plotly + node.js to stream data coming through POST requests
- → IsGenerator implementation
- → Async/Await not waiting
- → (Socket.io on nodejs) Updating div with mysql data stops without showing error
Ad