How to serve different robots.txt for http and https on same site?

I got a small site which served by Apache (I can't put Nginx in front nor change Apache to anything), and it is set up to serve the same site both over http and https (no redirects http->https is there so far, so both http and https versions are served in parallel).

What I need is to set up .htaccess so the same URI via http and via https to serve different text file?

Like http://example.com/proto.txt says "The site is over http" while https://example.com/proto.txt would say "The site served over https".

edited Nov 26 '18 at 9:37

Mr Shunz

2,23412122

asked Nov 26 '18 at 9:06

Kevin M

515

Don't quite have the time to check the exact way to write this (and so just a comment, not an answer), but another option if you don't want to or can't change the main Apache config, is to use a RewriteRule in your .htaccess with a condition on it being served over https.

– jcaron
Nov 26 '18 at 12:45

@dcaron, this is exactly I need, I just can't figure out how to do that exactly, and need an advice or (better) code snippet to do that :)

– Kevin M
Nov 27 '18 at 10:24

Your http and https sites are in different <VirtualHost>s. So you simply need to configure a different robots.txt in one of them. Around so: RewriteRule ^/robots.txt$ /path/to/alternative/robots.txt [L]

– peterh
Dec 12 '18 at 18:08

add a comment |

What I need is to set up .htaccess so the same URI via http and via https to serve different text file?

Like http://example.com/proto.txt says "The site is over http" while https://example.com/proto.txt would say "The site served over https".

edited Nov 26 '18 at 9:37

Mr Shunz

2,23412122

asked Nov 26 '18 at 9:06

Kevin M

515

Don't quite have the time to check the exact way to write this (and so just a comment, not an answer), but another option if you don't want to or can't change the main Apache config, is to use a RewriteRule in your .htaccess with a condition on it being served over https.

– jcaron
Nov 26 '18 at 12:45

@dcaron, this is exactly I need, I just can't figure out how to do that exactly, and need an advice or (better) code snippet to do that :)

– Kevin M
Nov 27 '18 at 10:24

Your http and https sites are in different <VirtualHost>s. So you simply need to configure a different robots.txt in one of them. Around so: RewriteRule ^/robots.txt$ /path/to/alternative/robots.txt [L]

– peterh
Dec 12 '18 at 18:08

add a comment |

What I need is to set up .htaccess so the same URI via http and via https to serve different text file?

Like http://example.com/proto.txt says "The site is over http" while https://example.com/proto.txt would say "The site served over https".

edited Nov 26 '18 at 9:37

Mr Shunz

2,23412122

asked Nov 26 '18 at 9:06

Kevin M

515

What I need is to set up .htaccess so the same URI via http and via https to serve different text file?

Like http://example.com/proto.txt says "The site is over http" while https://example.com/proto.txt would say "The site served over https".

apache-2.4 https http htpasswd

edited Nov 26 '18 at 9:37

Mr Shunz

2,23412122

asked Nov 26 '18 at 9:06

Kevin M

515

edited Nov 26 '18 at 9:37

Mr Shunz

2,23412122

asked Nov 26 '18 at 9:06

Kevin M

515

edited Nov 26 '18 at 9:37

Mr Shunz

2,23412122

edited Nov 26 '18 at 9:37

Mr Shunz

2,23412122

edited Nov 26 '18 at 9:37

Mr Shunz

2,23412122

asked Nov 26 '18 at 9:06

Kevin M

515

asked Nov 26 '18 at 9:06

Kevin M

515

asked Nov 26 '18 at 9:06

Kevin M

515

Don't quite have the time to check the exact way to write this (and so just a comment, not an answer), but another option if you don't want to or can't change the main Apache config, is to use a RewriteRule in your .htaccess with a condition on it being served over https.

– jcaron
Nov 26 '18 at 12:45

@dcaron, this is exactly I need, I just can't figure out how to do that exactly, and need an advice or (better) code snippet to do that :)

– Kevin M
Nov 27 '18 at 10:24

Your http and https sites are in different <VirtualHost>s. So you simply need to configure a different robots.txt in one of them. Around so: RewriteRule ^/robots.txt$ /path/to/alternative/robots.txt [L]

– peterh
Dec 12 '18 at 18:08

add a comment |

Don't quite have the time to check the exact way to write this (and so just a comment, not an answer), but another option if you don't want to or can't change the main Apache config, is to use a RewriteRule in your .htaccess with a condition on it being served over https.

– jcaron
Nov 26 '18 at 12:45

@dcaron, this is exactly I need, I just can't figure out how to do that exactly, and need an advice or (better) code snippet to do that :)

– Kevin M
Nov 27 '18 at 10:24

Your http and https sites are in different <VirtualHost>s. So you simply need to configure a different robots.txt in one of them. Around so: RewriteRule ^/robots.txt$ /path/to/alternative/robots.txt [L]

– peterh
Dec 12 '18 at 18:08

Don't quite have the time to check the exact way to write this (and so just a comment, not an answer), but another option if you don't want to or can't change the main Apache config, is to use a RewriteRule in your .htaccess with a condition on it being served over https.

– jcaron
Nov 26 '18 at 12:45

@dcaron, this is exactly I need, I just can't figure out how to do that exactly, and need an advice or (better) code snippet to do that :)

– Kevin M
Nov 27 '18 at 10:24

Your http and https sites are in different <VirtualHost>s. So you simply need to configure a different robots.txt in one of them. Around so: RewriteRule ^/robots.txt$ /path/to/alternative/robots.txt [L]

– peterh
Dec 12 '18 at 18:08

add a comment |

2 Answers
2

active

oldest

votes

Use an Alias

Create two files, robots.txt and robots_http.txt and add this to your http VirtualHost:

Alias "/robots.txt" "/path/to/documentroot/robots_http.txt"

edited Nov 26 '18 at 9:14

answered Nov 26 '18 at 9:09

Gerald Schneider

6,44112445

Can not modify vhost settings, can only edit .htaccess, this is the trick.

– Kevin M
Nov 27 '18 at 10:11

add a comment |

If you can't or won't change the "main" Apache config but need to do it in a .htaccess file, you can use a RewriteRule with a RewriteCond that checks for HTTPS.

Something along the lines of:

RewriteEngine On



RewriteCond %{HTTPS} "on"

RewriteRule robots.txt robots_https.txt [L]

should probably work (I didn't test it).

Note that this is based on Apache doing HTTPS termination itself. If HTTPS termination is done on a reverse proxy before it, then the condition will likely be different (and will depend on the configuration of the reverse proxy and Apache).

answered Nov 27 '18 at 11:10

jcaron

24617

add a comment |

Your Answer

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "2"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});

}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fserverfault.com%2fquestions%2f941613%2fhow-to-serve-different-robots-txt-for-http-and-https-on-same-site%23new-answer', 'question_page');
}
);

Post as a guest

Name

Required, but never shown

2 Answers
2

active

oldest

votes

2 Answers
2

active

oldest

votes

Use an Alias

Create two files, robots.txt and robots_http.txt and add this to your http VirtualHost:

Alias "/robots.txt" "/path/to/documentroot/robots_http.txt"

edited Nov 26 '18 at 9:14

answered Nov 26 '18 at 9:09

Gerald Schneider

6,44112445

Can not modify vhost settings, can only edit .htaccess, this is the trick.

– Kevin M
Nov 27 '18 at 10:11

add a comment |

Use an Alias

Create two files, robots.txt and robots_http.txt and add this to your http VirtualHost:

Alias "/robots.txt" "/path/to/documentroot/robots_http.txt"

edited Nov 26 '18 at 9:14

answered Nov 26 '18 at 9:09

Gerald Schneider

6,44112445

Can not modify vhost settings, can only edit .htaccess, this is the trick.

– Kevin M
Nov 27 '18 at 10:11

add a comment |

Use an Alias

Create two files, robots.txt and robots_http.txt and add this to your http VirtualHost:

Alias "/robots.txt" "/path/to/documentroot/robots_http.txt"

edited Nov 26 '18 at 9:14

answered Nov 26 '18 at 9:09

Gerald Schneider

6,44112445

Use an Alias

Create two files, robots.txt and robots_http.txt and add this to your http VirtualHost:

Alias "/robots.txt" "/path/to/documentroot/robots_http.txt"

edited Nov 26 '18 at 9:14

answered Nov 26 '18 at 9:09

Gerald Schneider

6,44112445

edited Nov 26 '18 at 9:14

answered Nov 26 '18 at 9:09

Gerald Schneider

6,44112445

answered Nov 26 '18 at 9:09

Gerald Schneider

6,44112445

answered Nov 26 '18 at 9:09

Gerald Schneider

6,44112445

Can not modify vhost settings, can only edit .htaccess, this is the trick.

– Kevin M
Nov 27 '18 at 10:11

add a comment |

Can not modify vhost settings, can only edit .htaccess, this is the trick.

– Kevin M
Nov 27 '18 at 10:11

Can not modify vhost settings, can only edit .htaccess, this is the trick.

– Kevin M
Nov 27 '18 at 10:11

add a comment |

If you can't or won't change the "main" Apache config but need to do it in a .htaccess file, you can use a RewriteRule with a RewriteCond that checks for HTTPS.

Something along the lines of:

RewriteEngine On



RewriteCond %{HTTPS} "on"

RewriteRule robots.txt robots_https.txt [L]

should probably work (I didn't test it).

answered Nov 27 '18 at 11:10

jcaron

24617

add a comment |

If you can't or won't change the "main" Apache config but need to do it in a .htaccess file, you can use a RewriteRule with a RewriteCond that checks for HTTPS.

Something along the lines of:

RewriteEngine On



RewriteCond %{HTTPS} "on"

RewriteRule robots.txt robots_https.txt [L]

should probably work (I didn't test it).

answered Nov 27 '18 at 11:10

jcaron

24617

add a comment |

If you can't or won't change the "main" Apache config but need to do it in a .htaccess file, you can use a RewriteRule with a RewriteCond that checks for HTTPS.

Something along the lines of:

RewriteEngine On



RewriteCond %{HTTPS} "on"

RewriteRule robots.txt robots_https.txt [L]

should probably work (I didn't test it).

answered Nov 27 '18 at 11:10

jcaron

24617

If you can't or won't change the "main" Apache config but need to do it in a .htaccess file, you can use a RewriteRule with a RewriteCond that checks for HTTPS.

Something along the lines of:

RewriteEngine On



RewriteCond %{HTTPS} "on"

RewriteRule robots.txt robots_https.txt [L]

should probably work (I didn't test it).

answered Nov 27 '18 at 11:10

jcaron

24617

answered Nov 27 '18 at 11:10

jcaron

24617

answered Nov 27 '18 at 11:10

jcaron

24617

answered Nov 27 '18 at 11:10

jcaron

24617

add a comment |

draft saved

draft discarded

Thanks for contributing an answer to Server Fault!

Please be sure to answer the question. Provide details and share your research!

But avoid …

Asking for help, clarification, or responding to other answers.

Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Name

Required, but never shown

Name

Required, but never shown

This page is only for reference, If you need detailed information, please check here

搜尋此網誌

Btukfyl