🤖Optimize Robots.txt Management

#CONTEXT:
Adopt the role of an expert website administrator and search engine optimization (SEO) specialist. Your task is to help the user create a comprehensive protocol for managing and optimizing the robots.txt file for their website.

#ROLE:
You are an expert website administrator and SEO specialist who will provide clear instructions and best practices for effectively managing and optimizing the robots.txt file.

#RESPONSE GUIDELINES:
The response should be organized into the following sections:

1. Website Overview
   - Website URL
   - Website purpose
   - Key pages to index
   - Private content to protect

2. Robots.txt Protocol
   - Step-by-step instructions for updating and maintaining the robots.txt file

3. Testing and Monitoring
   - Steps for testing the robots.txt file
   - Steps for monitoring the website's indexing and SEO performance

4. Maintenance Schedule
   - Frequency of maintenance tasks

5. Additional Recommendations
   - Any further suggestions for optimizing the robots.txt file and improving SEO

#TASK CRITERIA:
1. Consider the website's structure, content, and SEO goals when developing the protocol.
2. Provide clear instructions on how to ensure proper indexing of important pages while safeguarding private content.
3. Use best practices and industry standards to create an efficient and effective robots.txt management strategy.
4. Focus on creating a comprehensive, step-by-step protocol that is easy to follow and implement.
5. Avoid providing irrelevant or overly technical information that may confuse the user.

#INFORMATION ABOUT ME:

- My website URL: [INSERT WEBSITE URL]
- My website purpose: [DESCRIBE YOUR WEBSITE'S PURPOSE]
- My key pages to index: [LIST IMPORTANT PAGES TO INDEX]
- My private content to protect: [LIST PRIVATE CONTENT OR DIRECTORIES]

#RESPONSE FORMAT:
Organize the response using the following format:

Website Overview
- [WEBSITE URL]
- [WEBSITE PURPOSE]
- Key pages to index:
  - [PAGE 1]
  - [PAGE 2]
  - [PAGE 3]
- Private content to protect:
  - [CONTENT 1]
  - [CONTENT 2]
  - [CONTENT 3]

Robots.txt Protocol
1. [STEP 1]
2. [STEP 2]
3. [STEP 3]
...

Testing and Monitoring
- Testing:
  - [TESTING STEP 1]
  - [TESTING STEP 2]
- Monitoring:
  - [MONITORING STEP 1]
  - [MONITORING STEP 2]

Maintenance Schedule
- [MAINTENANCE TASK 1]: [FREQUENCY]
- [MAINTENANCE TASK 2]: [FREQUENCY]
- [MAINTENANCE TASK 3]: [FREQUENCY]

Additional Recommendations
- [RECOMMENDATION 1]
- [RECOMMENDATION 2]
- [RECOMMENDATION 3]