I know that presently order by is not there in BeamSQL is there any work around for it












0
















This is present



PCollection<BeamRecord> rec = rec_out.apply(BeamSql.query(
"SELECT bnk_name,state_name,val from PCOLLECTION order by val desc limit 2"));


But I need



PCollection<BeamRecord> rec = rec_out.apply(BeamSql.query(
"SELECT bnk_name,state_name,val from PCOLLECTION order by val desc "));









share|improve this question
























  • What will you do with the data after it is output? What is the purpose of the order?
    – Kenn Knowles
    Nov 28 at 15:30
















0
















This is present



PCollection<BeamRecord> rec = rec_out.apply(BeamSql.query(
"SELECT bnk_name,state_name,val from PCOLLECTION order by val desc limit 2"));


But I need



PCollection<BeamRecord> rec = rec_out.apply(BeamSql.query(
"SELECT bnk_name,state_name,val from PCOLLECTION order by val desc "));









share|improve this question
























  • What will you do with the data after it is output? What is the purpose of the order?
    – Kenn Knowles
    Nov 28 at 15:30














0












0








0









This is present



PCollection<BeamRecord> rec = rec_out.apply(BeamSql.query(
"SELECT bnk_name,state_name,val from PCOLLECTION order by val desc limit 2"));


But I need



PCollection<BeamRecord> rec = rec_out.apply(BeamSql.query(
"SELECT bnk_name,state_name,val from PCOLLECTION order by val desc "));









share|improve this question

















This is present



PCollection<BeamRecord> rec = rec_out.apply(BeamSql.query(
"SELECT bnk_name,state_name,val from PCOLLECTION order by val desc limit 2"));


But I need



PCollection<BeamRecord> rec = rec_out.apply(BeamSql.query(
"SELECT bnk_name,state_name,val from PCOLLECTION order by val desc "));






apache google-cloud-platform google-cloud-dataflow apache-beam apache-beam-io






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Nov 28 at 15:27









Kenn Knowles

2,930814




2,930814










asked Nov 23 at 2:20









TEJASWAKUMAR SIRIGIRI

12




12












  • What will you do with the data after it is output? What is the purpose of the order?
    – Kenn Knowles
    Nov 28 at 15:30


















  • What will you do with the data after it is output? What is the purpose of the order?
    – Kenn Knowles
    Nov 28 at 15:30
















What will you do with the data after it is output? What is the purpose of the order?
– Kenn Knowles
Nov 28 at 15:30




What will you do with the data after it is output? What is the purpose of the order?
– Kenn Knowles
Nov 28 at 15:30












1 Answer
1






active

oldest

votes


















0














This actually wouldn't help. PCollections are unordered in Beam, so you cannot apply ordering to the whole output.



Beam does not have ordering because it is usually not necessary, often not possible. Most problems where ordering is used have another solution that does not require global sorting.



If you describe more about what you will do with the output of the SQL query, I will update my answer with advice.



If you cannot share more, then you have the option of using Beam's sorter extension outside of the SQL query, but again it is not really for global sorting, but per-key order after a shuffle.






share|improve this answer





















  • I will store my output Pcollection to a csv file or any database .
    – TEJASWAKUMAR SIRIGIRI
    Dec 1 at 13:19










  • That makes sense - if your data is small enough to store in a database then it is a great choice.
    – Kenn Knowles
    Dec 1 at 17:22










  • Ok thanks for the information and suggestions.
    – TEJASWAKUMAR SIRIGIRI
    Dec 2 at 15:58











Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














draft saved

draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53439933%2fi-know-that-presently-order-by-is-not-there-in-beamsql-is-there-any-work-around%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes









0














This actually wouldn't help. PCollections are unordered in Beam, so you cannot apply ordering to the whole output.



Beam does not have ordering because it is usually not necessary, often not possible. Most problems where ordering is used have another solution that does not require global sorting.



If you describe more about what you will do with the output of the SQL query, I will update my answer with advice.



If you cannot share more, then you have the option of using Beam's sorter extension outside of the SQL query, but again it is not really for global sorting, but per-key order after a shuffle.






share|improve this answer





















  • I will store my output Pcollection to a csv file or any database .
    – TEJASWAKUMAR SIRIGIRI
    Dec 1 at 13:19










  • That makes sense - if your data is small enough to store in a database then it is a great choice.
    – Kenn Knowles
    Dec 1 at 17:22










  • Ok thanks for the information and suggestions.
    – TEJASWAKUMAR SIRIGIRI
    Dec 2 at 15:58
















0














This actually wouldn't help. PCollections are unordered in Beam, so you cannot apply ordering to the whole output.



Beam does not have ordering because it is usually not necessary, often not possible. Most problems where ordering is used have another solution that does not require global sorting.



If you describe more about what you will do with the output of the SQL query, I will update my answer with advice.



If you cannot share more, then you have the option of using Beam's sorter extension outside of the SQL query, but again it is not really for global sorting, but per-key order after a shuffle.






share|improve this answer





















  • I will store my output Pcollection to a csv file or any database .
    – TEJASWAKUMAR SIRIGIRI
    Dec 1 at 13:19










  • That makes sense - if your data is small enough to store in a database then it is a great choice.
    – Kenn Knowles
    Dec 1 at 17:22










  • Ok thanks for the information and suggestions.
    – TEJASWAKUMAR SIRIGIRI
    Dec 2 at 15:58














0












0








0






This actually wouldn't help. PCollections are unordered in Beam, so you cannot apply ordering to the whole output.



Beam does not have ordering because it is usually not necessary, often not possible. Most problems where ordering is used have another solution that does not require global sorting.



If you describe more about what you will do with the output of the SQL query, I will update my answer with advice.



If you cannot share more, then you have the option of using Beam's sorter extension outside of the SQL query, but again it is not really for global sorting, but per-key order after a shuffle.






share|improve this answer












This actually wouldn't help. PCollections are unordered in Beam, so you cannot apply ordering to the whole output.



Beam does not have ordering because it is usually not necessary, often not possible. Most problems where ordering is used have another solution that does not require global sorting.



If you describe more about what you will do with the output of the SQL query, I will update my answer with advice.



If you cannot share more, then you have the option of using Beam's sorter extension outside of the SQL query, but again it is not really for global sorting, but per-key order after a shuffle.







share|improve this answer












share|improve this answer



share|improve this answer










answered Nov 28 at 15:29









Kenn Knowles

2,930814




2,930814












  • I will store my output Pcollection to a csv file or any database .
    – TEJASWAKUMAR SIRIGIRI
    Dec 1 at 13:19










  • That makes sense - if your data is small enough to store in a database then it is a great choice.
    – Kenn Knowles
    Dec 1 at 17:22










  • Ok thanks for the information and suggestions.
    – TEJASWAKUMAR SIRIGIRI
    Dec 2 at 15:58


















  • I will store my output Pcollection to a csv file or any database .
    – TEJASWAKUMAR SIRIGIRI
    Dec 1 at 13:19










  • That makes sense - if your data is small enough to store in a database then it is a great choice.
    – Kenn Knowles
    Dec 1 at 17:22










  • Ok thanks for the information and suggestions.
    – TEJASWAKUMAR SIRIGIRI
    Dec 2 at 15:58
















I will store my output Pcollection to a csv file or any database .
– TEJASWAKUMAR SIRIGIRI
Dec 1 at 13:19




I will store my output Pcollection to a csv file or any database .
– TEJASWAKUMAR SIRIGIRI
Dec 1 at 13:19












That makes sense - if your data is small enough to store in a database then it is a great choice.
– Kenn Knowles
Dec 1 at 17:22




That makes sense - if your data is small enough to store in a database then it is a great choice.
– Kenn Knowles
Dec 1 at 17:22












Ok thanks for the information and suggestions.
– TEJASWAKUMAR SIRIGIRI
Dec 2 at 15:58




Ok thanks for the information and suggestions.
– TEJASWAKUMAR SIRIGIRI
Dec 2 at 15:58


















draft saved

draft discarded




















































Thanks for contributing an answer to Stack Overflow!


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.





Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


Please pay close attention to the following guidance:


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53439933%2fi-know-that-presently-order-by-is-not-there-in-beamsql-is-there-any-work-around%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

A CLEAN and SIMPLE way to add appendices to Table of Contents and bookmarks

Calculate evaluation metrics using cross_val_predict sklearn

Insert data from modal to MySQL (multiple modal on website)