Skip to content

Commit acf1544

Browse files
author
Kyle Kirkby
committed
Removing pages from sitemap/robots.txt crawl
1 parent 0ef2f63 commit acf1544

File tree

5 files changed

+6
-0
lines changed

5 files changed

+6
-0
lines changed

Diff for: ‎_pages/blog_tags.md

+1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
---
22
title: Tagged Blog Posts
33
permalink: /blog/tags/
4+
sitemap: false
45
description: >
56
Filter blog posts by tag.
67
jumbotron:

Diff for: ‎_pages/blog_themes.md

+1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
---
22
title: Blog Posts based on Themes
33
permalink: /blog/themes/
4+
sitemap: false
45
description: >
56
Filter blog posts by theme.
67
jumbotron:

Diff for: ‎_pages/news_tags.md

+1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
---
22
title: Tagged News Posts
33
permalink: /news/tags/
4+
sitemap: false
45
description: >
56
Filter news posts by tag.
67
jumbotron:

Diff for: ‎_pages/news_themes.md

+1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
---
22
title: New posts based on Themes
33
permalink: /news/themes/
4+
sitemap: false
45
description: >
56
Filter blog posts by theme.
67
jumbotron:

Diff for: ‎robots.txt

+2
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,8 @@
33
{% if site.production %}
44
User-agent: *
55
Sitemap: {{site.url}}/sitemap.xml
6+
Disallow: /blog/tags/
7+
Disallow: /news/tags/
68
{% else %}
79
User-agent: *
810
Disallow: /

0 commit comments

Comments
 (0)