Elasticsearch cluster on kubernetes cluster. In or out?












1















I'm currently working on deploying an elasticseacrh cluster in K8s.
Can anyone help me understand what are the cons/pros of deploying the ES cluster inside our K8s cluster or outside? Thanks in advance!










share|improve this question



























    1















    I'm currently working on deploying an elasticseacrh cluster in K8s.
    Can anyone help me understand what are the cons/pros of deploying the ES cluster inside our K8s cluster or outside? Thanks in advance!










    share|improve this question

























      1












      1








      1


      1






      I'm currently working on deploying an elasticseacrh cluster in K8s.
      Can anyone help me understand what are the cons/pros of deploying the ES cluster inside our K8s cluster or outside? Thanks in advance!










      share|improve this question














      I'm currently working on deploying an elasticseacrh cluster in K8s.
      Can anyone help me understand what are the cons/pros of deploying the ES cluster inside our K8s cluster or outside? Thanks in advance!







      elasticsearch kubernetes






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked Nov 26 '18 at 14:34









      LCPALCPA

      61




      61
























          1 Answer
          1






          active

          oldest

          votes


















          0














          A big pro is data ingestion. If you have your ES cluster inside your k8s cluster, data ingestion will be faster.



          However, a big con is resources. ES will eat away your resources worse than google-chrome eats your ram. And I mean, a lot.



          And maintaining it can be quite cumbersome. Not sure about your use case but if it is logging (as in most cases), usually cloud providers have their own solution for that.



          If not, then:



          I would recommend having dedicated nodes for ES in your cluster, otherwise it might affect other pods if there are peaks and starts using a lot of node resources.



          Also make sure to familiarize yourself and optimize your cold-warm-hot data, it will save you a lot of time and resources.



          EDIT



          I haven't emphasized how important is this faster data ingestion so it might not seem like a good enough reason to deploy it inside the cluster. Bottom line is pretty obvious: Network latency and bandwidth.



          These things can really add up (picking up all those logs from all those pods, then scaling those same pods, then expanding the cluster, then again...), so every unit counts. If your VMs will not suffer from those two (meaning, they have same latency as any other node of the cluster), I think it won't make a huge difference.



          On the other hand, I see no big benefit in separating them from the cluster. It is a part of your infrastructure anyway.



          What if tomorrow you decide to switch to AWS or GKE? You would have to change your deployments, setup the whole thing again. On the other hand, if it's already a part of your cluster, just kubectl apply and 🤷



          I can also guess that you will try to setup an ELK stack. If time and good will allows, give fluentd a chance (it is 100% compatible with all logstash clients but much more lghtweight).






          share|improve this answer


























          • Hi Urosh, First of all, thank you for the answer! We are going to have 4 VMs exclusively for the ES cluster (2 masters and 2 data, for redundancy), so the resources will be contained to those machines. And you guessed right, it will be used to manage logging, and will be "on-prem" (not cloud). I was trying to figure if there are any benefits of placing this cluster directlly inside our k8s cluster, or if it is better to leave it as a separate cluster. Regarding this new information... care to share some more insights? :) Thanks in advance!

            – LCPA
            Nov 26 '18 at 16:15











          • Edited my answer, hope it helps

            – Urosh T.
            Nov 26 '18 at 19:49











          • Thx for you time, Urosh! :)

            – LCPA
            Nov 27 '18 at 14:39











          Your Answer






          StackExchange.ifUsing("editor", function () {
          StackExchange.using("externalEditor", function () {
          StackExchange.using("snippets", function () {
          StackExchange.snippets.init();
          });
          });
          }, "code-snippets");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "1"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53483369%2felasticsearch-cluster-on-kubernetes-cluster-in-or-out%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          0














          A big pro is data ingestion. If you have your ES cluster inside your k8s cluster, data ingestion will be faster.



          However, a big con is resources. ES will eat away your resources worse than google-chrome eats your ram. And I mean, a lot.



          And maintaining it can be quite cumbersome. Not sure about your use case but if it is logging (as in most cases), usually cloud providers have their own solution for that.



          If not, then:



          I would recommend having dedicated nodes for ES in your cluster, otherwise it might affect other pods if there are peaks and starts using a lot of node resources.



          Also make sure to familiarize yourself and optimize your cold-warm-hot data, it will save you a lot of time and resources.



          EDIT



          I haven't emphasized how important is this faster data ingestion so it might not seem like a good enough reason to deploy it inside the cluster. Bottom line is pretty obvious: Network latency and bandwidth.



          These things can really add up (picking up all those logs from all those pods, then scaling those same pods, then expanding the cluster, then again...), so every unit counts. If your VMs will not suffer from those two (meaning, they have same latency as any other node of the cluster), I think it won't make a huge difference.



          On the other hand, I see no big benefit in separating them from the cluster. It is a part of your infrastructure anyway.



          What if tomorrow you decide to switch to AWS or GKE? You would have to change your deployments, setup the whole thing again. On the other hand, if it's already a part of your cluster, just kubectl apply and 🤷



          I can also guess that you will try to setup an ELK stack. If time and good will allows, give fluentd a chance (it is 100% compatible with all logstash clients but much more lghtweight).






          share|improve this answer


























          • Hi Urosh, First of all, thank you for the answer! We are going to have 4 VMs exclusively for the ES cluster (2 masters and 2 data, for redundancy), so the resources will be contained to those machines. And you guessed right, it will be used to manage logging, and will be "on-prem" (not cloud). I was trying to figure if there are any benefits of placing this cluster directlly inside our k8s cluster, or if it is better to leave it as a separate cluster. Regarding this new information... care to share some more insights? :) Thanks in advance!

            – LCPA
            Nov 26 '18 at 16:15











          • Edited my answer, hope it helps

            – Urosh T.
            Nov 26 '18 at 19:49











          • Thx for you time, Urosh! :)

            – LCPA
            Nov 27 '18 at 14:39
















          0














          A big pro is data ingestion. If you have your ES cluster inside your k8s cluster, data ingestion will be faster.



          However, a big con is resources. ES will eat away your resources worse than google-chrome eats your ram. And I mean, a lot.



          And maintaining it can be quite cumbersome. Not sure about your use case but if it is logging (as in most cases), usually cloud providers have their own solution for that.



          If not, then:



          I would recommend having dedicated nodes for ES in your cluster, otherwise it might affect other pods if there are peaks and starts using a lot of node resources.



          Also make sure to familiarize yourself and optimize your cold-warm-hot data, it will save you a lot of time and resources.



          EDIT



          I haven't emphasized how important is this faster data ingestion so it might not seem like a good enough reason to deploy it inside the cluster. Bottom line is pretty obvious: Network latency and bandwidth.



          These things can really add up (picking up all those logs from all those pods, then scaling those same pods, then expanding the cluster, then again...), so every unit counts. If your VMs will not suffer from those two (meaning, they have same latency as any other node of the cluster), I think it won't make a huge difference.



          On the other hand, I see no big benefit in separating them from the cluster. It is a part of your infrastructure anyway.



          What if tomorrow you decide to switch to AWS or GKE? You would have to change your deployments, setup the whole thing again. On the other hand, if it's already a part of your cluster, just kubectl apply and 🤷



          I can also guess that you will try to setup an ELK stack. If time and good will allows, give fluentd a chance (it is 100% compatible with all logstash clients but much more lghtweight).






          share|improve this answer


























          • Hi Urosh, First of all, thank you for the answer! We are going to have 4 VMs exclusively for the ES cluster (2 masters and 2 data, for redundancy), so the resources will be contained to those machines. And you guessed right, it will be used to manage logging, and will be "on-prem" (not cloud). I was trying to figure if there are any benefits of placing this cluster directlly inside our k8s cluster, or if it is better to leave it as a separate cluster. Regarding this new information... care to share some more insights? :) Thanks in advance!

            – LCPA
            Nov 26 '18 at 16:15











          • Edited my answer, hope it helps

            – Urosh T.
            Nov 26 '18 at 19:49











          • Thx for you time, Urosh! :)

            – LCPA
            Nov 27 '18 at 14:39














          0












          0








          0







          A big pro is data ingestion. If you have your ES cluster inside your k8s cluster, data ingestion will be faster.



          However, a big con is resources. ES will eat away your resources worse than google-chrome eats your ram. And I mean, a lot.



          And maintaining it can be quite cumbersome. Not sure about your use case but if it is logging (as in most cases), usually cloud providers have their own solution for that.



          If not, then:



          I would recommend having dedicated nodes for ES in your cluster, otherwise it might affect other pods if there are peaks and starts using a lot of node resources.



          Also make sure to familiarize yourself and optimize your cold-warm-hot data, it will save you a lot of time and resources.



          EDIT



          I haven't emphasized how important is this faster data ingestion so it might not seem like a good enough reason to deploy it inside the cluster. Bottom line is pretty obvious: Network latency and bandwidth.



          These things can really add up (picking up all those logs from all those pods, then scaling those same pods, then expanding the cluster, then again...), so every unit counts. If your VMs will not suffer from those two (meaning, they have same latency as any other node of the cluster), I think it won't make a huge difference.



          On the other hand, I see no big benefit in separating them from the cluster. It is a part of your infrastructure anyway.



          What if tomorrow you decide to switch to AWS or GKE? You would have to change your deployments, setup the whole thing again. On the other hand, if it's already a part of your cluster, just kubectl apply and 🤷



          I can also guess that you will try to setup an ELK stack. If time and good will allows, give fluentd a chance (it is 100% compatible with all logstash clients but much more lghtweight).






          share|improve this answer















          A big pro is data ingestion. If you have your ES cluster inside your k8s cluster, data ingestion will be faster.



          However, a big con is resources. ES will eat away your resources worse than google-chrome eats your ram. And I mean, a lot.



          And maintaining it can be quite cumbersome. Not sure about your use case but if it is logging (as in most cases), usually cloud providers have their own solution for that.



          If not, then:



          I would recommend having dedicated nodes for ES in your cluster, otherwise it might affect other pods if there are peaks and starts using a lot of node resources.



          Also make sure to familiarize yourself and optimize your cold-warm-hot data, it will save you a lot of time and resources.



          EDIT



          I haven't emphasized how important is this faster data ingestion so it might not seem like a good enough reason to deploy it inside the cluster. Bottom line is pretty obvious: Network latency and bandwidth.



          These things can really add up (picking up all those logs from all those pods, then scaling those same pods, then expanding the cluster, then again...), so every unit counts. If your VMs will not suffer from those two (meaning, they have same latency as any other node of the cluster), I think it won't make a huge difference.



          On the other hand, I see no big benefit in separating them from the cluster. It is a part of your infrastructure anyway.



          What if tomorrow you decide to switch to AWS or GKE? You would have to change your deployments, setup the whole thing again. On the other hand, if it's already a part of your cluster, just kubectl apply and 🤷



          I can also guess that you will try to setup an ELK stack. If time and good will allows, give fluentd a chance (it is 100% compatible with all logstash clients but much more lghtweight).







          share|improve this answer














          share|improve this answer



          share|improve this answer








          edited Nov 26 '18 at 19:49

























          answered Nov 26 '18 at 15:04









          Urosh T.Urosh T.

          66711216




          66711216













          • Hi Urosh, First of all, thank you for the answer! We are going to have 4 VMs exclusively for the ES cluster (2 masters and 2 data, for redundancy), so the resources will be contained to those machines. And you guessed right, it will be used to manage logging, and will be "on-prem" (not cloud). I was trying to figure if there are any benefits of placing this cluster directlly inside our k8s cluster, or if it is better to leave it as a separate cluster. Regarding this new information... care to share some more insights? :) Thanks in advance!

            – LCPA
            Nov 26 '18 at 16:15











          • Edited my answer, hope it helps

            – Urosh T.
            Nov 26 '18 at 19:49











          • Thx for you time, Urosh! :)

            – LCPA
            Nov 27 '18 at 14:39



















          • Hi Urosh, First of all, thank you for the answer! We are going to have 4 VMs exclusively for the ES cluster (2 masters and 2 data, for redundancy), so the resources will be contained to those machines. And you guessed right, it will be used to manage logging, and will be "on-prem" (not cloud). I was trying to figure if there are any benefits of placing this cluster directlly inside our k8s cluster, or if it is better to leave it as a separate cluster. Regarding this new information... care to share some more insights? :) Thanks in advance!

            – LCPA
            Nov 26 '18 at 16:15











          • Edited my answer, hope it helps

            – Urosh T.
            Nov 26 '18 at 19:49











          • Thx for you time, Urosh! :)

            – LCPA
            Nov 27 '18 at 14:39

















          Hi Urosh, First of all, thank you for the answer! We are going to have 4 VMs exclusively for the ES cluster (2 masters and 2 data, for redundancy), so the resources will be contained to those machines. And you guessed right, it will be used to manage logging, and will be "on-prem" (not cloud). I was trying to figure if there are any benefits of placing this cluster directlly inside our k8s cluster, or if it is better to leave it as a separate cluster. Regarding this new information... care to share some more insights? :) Thanks in advance!

          – LCPA
          Nov 26 '18 at 16:15





          Hi Urosh, First of all, thank you for the answer! We are going to have 4 VMs exclusively for the ES cluster (2 masters and 2 data, for redundancy), so the resources will be contained to those machines. And you guessed right, it will be used to manage logging, and will be "on-prem" (not cloud). I was trying to figure if there are any benefits of placing this cluster directlly inside our k8s cluster, or if it is better to leave it as a separate cluster. Regarding this new information... care to share some more insights? :) Thanks in advance!

          – LCPA
          Nov 26 '18 at 16:15













          Edited my answer, hope it helps

          – Urosh T.
          Nov 26 '18 at 19:49





          Edited my answer, hope it helps

          – Urosh T.
          Nov 26 '18 at 19:49













          Thx for you time, Urosh! :)

          – LCPA
          Nov 27 '18 at 14:39





          Thx for you time, Urosh! :)

          – LCPA
          Nov 27 '18 at 14:39




















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Stack Overflow!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53483369%2felasticsearch-cluster-on-kubernetes-cluster-in-or-out%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          A CLEAN and SIMPLE way to add appendices to Table of Contents and bookmarks

          Calculate evaluation metrics using cross_val_predict sklearn

          Insert data from modal to MySQL (multiple modal on website)