-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Description
Steps to Reproduce
npm i request robots-parse- Run the following code
const req = require('request');
req("https://optinmonster.com/robots.txt", (err, res, body) => {
if (err) {
console.log("Robots Parser Error");
console.error(err);
return reply.code(404).send({
error: "Couldn't find sitemap. Please provide sitemap URL",
});
}
const { parser } = require("robots-parse");
const object = parser(body);
console.log(object);
console.log(object.sitemaps);
}
);
- Input the following URL in request -
https://optinmonster.com/robots.txt
After doing these, I got the following error
TypeError: Cannot read property 'allow' of undefined
and my node server crashed.
Please check
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels