How to Write a Robots.Txt File
Open a simple text editor such as Notepad., Write the first line., Decide what areas you do not want to have indexed., Save the file., Upload to the main directory of your website.
5 Steps
1 min read
Medium
Step-by-Step Guide
-
Step 1: Open a simple text editor such as Notepad.
All robots.txt files start with this line:
User-Agent: * , If you want all parts of your site to be indexed, add this line:
Disallow:
Add lines for every area that you do want to be indexed.
For example, if you do not want images and a private folder indexed:
Disallow: /images/ Disallow: /private/ , Use the name "robots.txt". , -
Step 2: Write the first line.
-
Step 3: Decide what areas you do not want to have indexed.
-
Step 4: Save the file.
-
Step 5: Upload to the main directory of your website.
Detailed Guide
All robots.txt files start with this line:
User-Agent: * , If you want all parts of your site to be indexed, add this line:
Disallow:
Add lines for every area that you do want to be indexed.
For example, if you do not want images and a private folder indexed:
Disallow: /images/ Disallow: /private/ , Use the name "robots.txt". ,
About the Author
J
Jean Hart
Writer and educator with a focus on practical practical skills knowledge.
27 articles
View all articles
Rate This Guide
--
Loading...
5
0
4
0
3
0
2
0
1
0
How helpful was this guide? Click to rate: