oliver/landingpage-go
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Erstellung robots.txt

Browse source
This commit is contained in:
Oliver 2026-02-24 12:21:00 +01:00
parent a9f93b8537
commit f5d030d9c3
No known key found for this signature in database
1 changed files with 16 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

16
robots.txt Normal file
View file

@ -0,0 +1,16 @@
# robots.txt für https://go.vanityontour.de
# Erlaubt allen Crawlern das Durchsuchen der Seite
User-agent: *
Disallow:
# Optionale Verlangsamung für Crawler, falls der Server stark belastet wird
# (wird z.B. von Google ignoriert, aber von anderen Bots beachtet)
# Crawl-delay: 10
# Sonderregeln für bestimmte Crawler
User-agent: Googlebot
Disallow:
User-agent: Bingbot
Disallow: