Writing output of String manipulation to Azure Data lake Store Item





.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box;
}







0















When I try to write an output of a String manipulation of Get-AzureRmDataLakeStoreItemContent output to a variable and try to pass it in a variable to New-AzureRmDataLakeStoreItem i am getting error "New-AzureRmDataLakeStoreItem : Invalid content passed in. Only byte and string content is supported."



I verified that the output of Get-command is an Object but I dont understand why i am not able to pass it. I am not sure whether I need some more transformation like a Hashtable to store into an output in Azure Data lake store. Or is it an Encoding problem. Please help in deciphering this error



Here is the code and i am also attaching a screenshot of the error.
Original input



1|2|3|a,b,
3|4|5|d,h,


Output of String manipulation



1|2|3|a,b
3|4|5|d,h

$data=((Get-AzureRmDataLakeStoreItemContent -Account $accountName -Path $myrootdir/V_FQP_ITC_11_VEHICLE/test.csv).ToString() -split("`r")).Trim() | ForEach-Object {$_.TrimEnd(",")}
New-AzureRmDataLakeStoreItem -Account $accountName -path $myrootdir/test_output.txt -Value $data


Error










share|improve this question























  • HI Joy, The command writes the Literal value "System-Object" to the output file" PS C:Users> New-AzureRmDataLakeStoreItem -Account $accountName -path $myrootdir/test_1.csv -Value $data.ToString() /landing_zone/rd/vedoc/096/data/test_1.csv PS C:Users> Get-AzureRmDataLakeStoreItemContent -Account $accountName -Path $myrootdir/test_1.csv System.Object

    – Naveen Venugopal
    Nov 29 '18 at 6:57











  • See the update of my answer.

    – Joy Wang
    Nov 29 '18 at 7:02


















0















When I try to write an output of a String manipulation of Get-AzureRmDataLakeStoreItemContent output to a variable and try to pass it in a variable to New-AzureRmDataLakeStoreItem i am getting error "New-AzureRmDataLakeStoreItem : Invalid content passed in. Only byte and string content is supported."



I verified that the output of Get-command is an Object but I dont understand why i am not able to pass it. I am not sure whether I need some more transformation like a Hashtable to store into an output in Azure Data lake store. Or is it an Encoding problem. Please help in deciphering this error



Here is the code and i am also attaching a screenshot of the error.
Original input



1|2|3|a,b,
3|4|5|d,h,


Output of String manipulation



1|2|3|a,b
3|4|5|d,h

$data=((Get-AzureRmDataLakeStoreItemContent -Account $accountName -Path $myrootdir/V_FQP_ITC_11_VEHICLE/test.csv).ToString() -split("`r")).Trim() | ForEach-Object {$_.TrimEnd(",")}
New-AzureRmDataLakeStoreItem -Account $accountName -path $myrootdir/test_output.txt -Value $data


Error










share|improve this question























  • HI Joy, The command writes the Literal value "System-Object" to the output file" PS C:Users> New-AzureRmDataLakeStoreItem -Account $accountName -path $myrootdir/test_1.csv -Value $data.ToString() /landing_zone/rd/vedoc/096/data/test_1.csv PS C:Users> Get-AzureRmDataLakeStoreItemContent -Account $accountName -Path $myrootdir/test_1.csv System.Object

    – Naveen Venugopal
    Nov 29 '18 at 6:57











  • See the update of my answer.

    – Joy Wang
    Nov 29 '18 at 7:02














0












0








0








When I try to write an output of a String manipulation of Get-AzureRmDataLakeStoreItemContent output to a variable and try to pass it in a variable to New-AzureRmDataLakeStoreItem i am getting error "New-AzureRmDataLakeStoreItem : Invalid content passed in. Only byte and string content is supported."



I verified that the output of Get-command is an Object but I dont understand why i am not able to pass it. I am not sure whether I need some more transformation like a Hashtable to store into an output in Azure Data lake store. Or is it an Encoding problem. Please help in deciphering this error



Here is the code and i am also attaching a screenshot of the error.
Original input



1|2|3|a,b,
3|4|5|d,h,


Output of String manipulation



1|2|3|a,b
3|4|5|d,h

$data=((Get-AzureRmDataLakeStoreItemContent -Account $accountName -Path $myrootdir/V_FQP_ITC_11_VEHICLE/test.csv).ToString() -split("`r")).Trim() | ForEach-Object {$_.TrimEnd(",")}
New-AzureRmDataLakeStoreItem -Account $accountName -path $myrootdir/test_output.txt -Value $data


Error










share|improve this question














When I try to write an output of a String manipulation of Get-AzureRmDataLakeStoreItemContent output to a variable and try to pass it in a variable to New-AzureRmDataLakeStoreItem i am getting error "New-AzureRmDataLakeStoreItem : Invalid content passed in. Only byte and string content is supported."



I verified that the output of Get-command is an Object but I dont understand why i am not able to pass it. I am not sure whether I need some more transformation like a Hashtable to store into an output in Azure Data lake store. Or is it an Encoding problem. Please help in deciphering this error



Here is the code and i am also attaching a screenshot of the error.
Original input



1|2|3|a,b,
3|4|5|d,h,


Output of String manipulation



1|2|3|a,b
3|4|5|d,h

$data=((Get-AzureRmDataLakeStoreItemContent -Account $accountName -Path $myrootdir/V_FQP_ITC_11_VEHICLE/test.csv).ToString() -split("`r")).Trim() | ForEach-Object {$_.TrimEnd(",")}
New-AzureRmDataLakeStoreItem -Account $accountName -path $myrootdir/test_output.txt -Value $data


Error







azure powershell azure-data-lake






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Nov 29 '18 at 6:41









Naveen VenugopalNaveen Venugopal

154




154













  • HI Joy, The command writes the Literal value "System-Object" to the output file" PS C:Users> New-AzureRmDataLakeStoreItem -Account $accountName -path $myrootdir/test_1.csv -Value $data.ToString() /landing_zone/rd/vedoc/096/data/test_1.csv PS C:Users> Get-AzureRmDataLakeStoreItemContent -Account $accountName -Path $myrootdir/test_1.csv System.Object

    – Naveen Venugopal
    Nov 29 '18 at 6:57











  • See the update of my answer.

    – Joy Wang
    Nov 29 '18 at 7:02



















  • HI Joy, The command writes the Literal value "System-Object" to the output file" PS C:Users> New-AzureRmDataLakeStoreItem -Account $accountName -path $myrootdir/test_1.csv -Value $data.ToString() /landing_zone/rd/vedoc/096/data/test_1.csv PS C:Users> Get-AzureRmDataLakeStoreItemContent -Account $accountName -Path $myrootdir/test_1.csv System.Object

    – Naveen Venugopal
    Nov 29 '18 at 6:57











  • See the update of my answer.

    – Joy Wang
    Nov 29 '18 at 7:02

















HI Joy, The command writes the Literal value "System-Object" to the output file" PS C:Users> New-AzureRmDataLakeStoreItem -Account $accountName -path $myrootdir/test_1.csv -Value $data.ToString() /landing_zone/rd/vedoc/096/data/test_1.csv PS C:Users> Get-AzureRmDataLakeStoreItemContent -Account $accountName -Path $myrootdir/test_1.csv System.Object

– Naveen Venugopal
Nov 29 '18 at 6:57





HI Joy, The command writes the Literal value "System-Object" to the output file" PS C:Users> New-AzureRmDataLakeStoreItem -Account $accountName -path $myrootdir/test_1.csv -Value $data.ToString() /landing_zone/rd/vedoc/096/data/test_1.csv PS C:Users> Get-AzureRmDataLakeStoreItemContent -Account $accountName -Path $myrootdir/test_1.csv System.Object

– Naveen Venugopal
Nov 29 '18 at 6:57













See the update of my answer.

– Joy Wang
Nov 29 '18 at 7:02





See the update of my answer.

– Joy Wang
Nov 29 '18 at 7:02












1 Answer
1






active

oldest

votes


















0














Seems you need to add a "" of $data, it works on my side.



New-AzureRmDataLakeStoreItem  -Account "joydatalake1" -path "/sss/test_output.txt" -Value "$data"


enter image description here






share|improve this answer


























  • Thanks Joy Once again. That was a silly mistake from my side. Thanks for correcting. One observation is that the out put has lost its formatting and hive external table is loading NULls. However, i will take some time to dig around see how to make it right.

    – Naveen Venugopal
    Nov 29 '18 at 7:14












Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














draft saved

draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53533224%2fwriting-output-of-string-manipulation-to-azure-data-lake-store-item%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes









0














Seems you need to add a "" of $data, it works on my side.



New-AzureRmDataLakeStoreItem  -Account "joydatalake1" -path "/sss/test_output.txt" -Value "$data"


enter image description here






share|improve this answer


























  • Thanks Joy Once again. That was a silly mistake from my side. Thanks for correcting. One observation is that the out put has lost its formatting and hive external table is loading NULls. However, i will take some time to dig around see how to make it right.

    – Naveen Venugopal
    Nov 29 '18 at 7:14
















0














Seems you need to add a "" of $data, it works on my side.



New-AzureRmDataLakeStoreItem  -Account "joydatalake1" -path "/sss/test_output.txt" -Value "$data"


enter image description here






share|improve this answer


























  • Thanks Joy Once again. That was a silly mistake from my side. Thanks for correcting. One observation is that the out put has lost its formatting and hive external table is loading NULls. However, i will take some time to dig around see how to make it right.

    – Naveen Venugopal
    Nov 29 '18 at 7:14














0












0








0







Seems you need to add a "" of $data, it works on my side.



New-AzureRmDataLakeStoreItem  -Account "joydatalake1" -path "/sss/test_output.txt" -Value "$data"


enter image description here






share|improve this answer















Seems you need to add a "" of $data, it works on my side.



New-AzureRmDataLakeStoreItem  -Account "joydatalake1" -path "/sss/test_output.txt" -Value "$data"


enter image description here







share|improve this answer














share|improve this answer



share|improve this answer








edited Nov 29 '18 at 6:59

























answered Nov 29 '18 at 6:49









Joy WangJoy Wang

8,5422315




8,5422315













  • Thanks Joy Once again. That was a silly mistake from my side. Thanks for correcting. One observation is that the out put has lost its formatting and hive external table is loading NULls. However, i will take some time to dig around see how to make it right.

    – Naveen Venugopal
    Nov 29 '18 at 7:14



















  • Thanks Joy Once again. That was a silly mistake from my side. Thanks for correcting. One observation is that the out put has lost its formatting and hive external table is loading NULls. However, i will take some time to dig around see how to make it right.

    – Naveen Venugopal
    Nov 29 '18 at 7:14

















Thanks Joy Once again. That was a silly mistake from my side. Thanks for correcting. One observation is that the out put has lost its formatting and hive external table is loading NULls. However, i will take some time to dig around see how to make it right.

– Naveen Venugopal
Nov 29 '18 at 7:14





Thanks Joy Once again. That was a silly mistake from my side. Thanks for correcting. One observation is that the out put has lost its formatting and hive external table is loading NULls. However, i will take some time to dig around see how to make it right.

– Naveen Venugopal
Nov 29 '18 at 7:14




















draft saved

draft discarded




















































Thanks for contributing an answer to Stack Overflow!


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53533224%2fwriting-output-of-string-manipulation-to-azure-data-lake-store-item%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

A CLEAN and SIMPLE way to add appendices to Table of Contents and bookmarks

Calculate evaluation metrics using cross_val_predict sklearn

Insert data from modal to MySQL (multiple modal on website)