Issue #2446657 by rpayanm, er.pushpinderrana: Remove dead link from robots.txt
parent
c377b88b8e
commit
a577cc98e3
|
@ -12,9 +12,6 @@
|
|||
#
|
||||
# For more information about the robots.txt standard, see:
|
||||
# http://www.robotstxt.org/robotstxt.html
|
||||
#
|
||||
# For syntax checking, see:
|
||||
# http://www.frobee.com/robots-txt-check
|
||||
|
||||
User-agent: *
|
||||
Crawl-delay: 10
|
||||
|
|
Loading…
Reference in New Issue