Python script to write list of dict with multiple values to multiple files

I have a list of dicts as shown below. I wish to write the dicts to multiple excel or csv files depending on the keys. if the keys are the same they should be in one file.

my_list_of_dicts:

[{'john': ['0', '100']}, {'john': ['4', '101']}, {'john': ['0', '102']}, {'mary': ['2', '100']}, {'mary': ['5', '101']}, {'mary': ['4', '102']}, {'mary': ['1', '103']}, {'sam': ['4', '100']}, {'sam': ['3', '101']}, {'sam': ['12', '102']}, {'paul': ['2', '100']}, {'hay': ['2', '100']}, {'hay': ['1', '102']}, {'mercy': ['4', '101']}]

My code so far:

x = 

i = 0

for ii, line in enumerate(my_list_of_dicts):

    with open("out_%s.csv" % i, 'w+') as f:

        if line.keys() not in x:

            x.append(line.keys())

            i += 1

            pd.DataFrame.from_dict(data=line, orient='index').to_csv(f, header=False)

        else:

            pd.DataFrame.from_dict(data=line, orient='index').to_csv(f, header=False)

Result:

I am getting the desired number of files but not the content.

Expectation:

I expect to get files corresponding to each key i.e (john, mary, sam, jay, paul, hay, and mercy) with the below content. Using john as example:

john, 0, 100 

john, 4, 101

john, 0, 102

I am not sure how to proceed or if I even need enumerate. Thank you

edited Nov 27 '18 at 0:04

jpp

101k2163112

asked Nov 26 '18 at 23:44

Starter

717

Do you mean to use i instead of ii in your for loop?

– Jack Moody
Nov 26 '18 at 23:52

No that is why I said I was not sure if using enumerate was the right idea cos the ii was not used

– Starter
Nov 26 '18 at 23:55

Normally if you aren’t going to use something like that you would use _

– Jack Moody
Nov 26 '18 at 23:57

1

Okay now I know. Thank you

– Starter
Nov 27 '18 at 0:04

add a comment |

I have a list of dicts as shown below. I wish to write the dicts to multiple excel or csv files depending on the keys. if the keys are the same they should be in one file.

my_list_of_dicts:

[{'john': ['0', '100']}, {'john': ['4', '101']}, {'john': ['0', '102']}, {'mary': ['2', '100']}, {'mary': ['5', '101']}, {'mary': ['4', '102']}, {'mary': ['1', '103']}, {'sam': ['4', '100']}, {'sam': ['3', '101']}, {'sam': ['12', '102']}, {'paul': ['2', '100']}, {'hay': ['2', '100']}, {'hay': ['1', '102']}, {'mercy': ['4', '101']}]

My code so far:

x = 

i = 0

for ii, line in enumerate(my_list_of_dicts):

    with open("out_%s.csv" % i, 'w+') as f:

        if line.keys() not in x:

            x.append(line.keys())

            i += 1

            pd.DataFrame.from_dict(data=line, orient='index').to_csv(f, header=False)

        else:

            pd.DataFrame.from_dict(data=line, orient='index').to_csv(f, header=False)

Result:

I am getting the desired number of files but not the content.

Expectation:

I expect to get files corresponding to each key i.e (john, mary, sam, jay, paul, hay, and mercy) with the below content. Using john as example:

john, 0, 100 

john, 4, 101

john, 0, 102

I am not sure how to proceed or if I even need enumerate. Thank you

edited Nov 27 '18 at 0:04

jpp

101k2163112

asked Nov 26 '18 at 23:44

Starter

717

Do you mean to use i instead of ii in your for loop?

– Jack Moody
Nov 26 '18 at 23:52

No that is why I said I was not sure if using enumerate was the right idea cos the ii was not used

– Starter
Nov 26 '18 at 23:55

Normally if you aren’t going to use something like that you would use _

– Jack Moody
Nov 26 '18 at 23:57

1

Okay now I know. Thank you

– Starter
Nov 27 '18 at 0:04

add a comment |

I have a list of dicts as shown below. I wish to write the dicts to multiple excel or csv files depending on the keys. if the keys are the same they should be in one file.

my_list_of_dicts:

[{'john': ['0', '100']}, {'john': ['4', '101']}, {'john': ['0', '102']}, {'mary': ['2', '100']}, {'mary': ['5', '101']}, {'mary': ['4', '102']}, {'mary': ['1', '103']}, {'sam': ['4', '100']}, {'sam': ['3', '101']}, {'sam': ['12', '102']}, {'paul': ['2', '100']}, {'hay': ['2', '100']}, {'hay': ['1', '102']}, {'mercy': ['4', '101']}]

My code so far:

x = 

i = 0

for ii, line in enumerate(my_list_of_dicts):

    with open("out_%s.csv" % i, 'w+') as f:

        if line.keys() not in x:

            x.append(line.keys())

            i += 1

            pd.DataFrame.from_dict(data=line, orient='index').to_csv(f, header=False)

        else:

            pd.DataFrame.from_dict(data=line, orient='index').to_csv(f, header=False)

Result:

I am getting the desired number of files but not the content.

Expectation:

I expect to get files corresponding to each key i.e (john, mary, sam, jay, paul, hay, and mercy) with the below content. Using john as example:

john, 0, 100 

john, 4, 101

john, 0, 102

I am not sure how to proceed or if I even need enumerate. Thank you

edited Nov 27 '18 at 0:04

jpp

101k2163112

asked Nov 26 '18 at 23:44

Starter

717

I have a list of dicts as shown below. I wish to write the dicts to multiple excel or csv files depending on the keys. if the keys are the same they should be in one file.

my_list_of_dicts:

[{'john': ['0', '100']}, {'john': ['4', '101']}, {'john': ['0', '102']}, {'mary': ['2', '100']}, {'mary': ['5', '101']}, {'mary': ['4', '102']}, {'mary': ['1', '103']}, {'sam': ['4', '100']}, {'sam': ['3', '101']}, {'sam': ['12', '102']}, {'paul': ['2', '100']}, {'hay': ['2', '100']}, {'hay': ['1', '102']}, {'mercy': ['4', '101']}]

My code so far:

x = 

i = 0

for ii, line in enumerate(my_list_of_dicts):

    with open("out_%s.csv" % i, 'w+') as f:

        if line.keys() not in x:

            x.append(line.keys())

            i += 1

            pd.DataFrame.from_dict(data=line, orient='index').to_csv(f, header=False)

        else:

            pd.DataFrame.from_dict(data=line, orient='index').to_csv(f, header=False)

Result:

I am getting the desired number of files but not the content.

Expectation:

I expect to get files corresponding to each key i.e (john, mary, sam, jay, paul, hay, and mercy) with the below content. Using john as example:

john, 0, 100 

john, 4, 101

john, 0, 102

I am not sure how to proceed or if I even need enumerate. Thank you

python pandas csv pandas-groupby

edited Nov 27 '18 at 0:04

jpp

101k2163112

asked Nov 26 '18 at 23:44

Starter

717

edited Nov 27 '18 at 0:04

jpp

101k2163112

asked Nov 26 '18 at 23:44

Starter

717

edited Nov 27 '18 at 0:04

jpp

101k2163112

edited Nov 27 '18 at 0:04

jpp

101k2163112

edited Nov 27 '18 at 0:04

jpp

101k2163112

asked Nov 26 '18 at 23:44

Starter

717

asked Nov 26 '18 at 23:44

Starter

717

asked Nov 26 '18 at 23:44

Starter

717

Do you mean to use i instead of ii in your for loop?

– Jack Moody
Nov 26 '18 at 23:52

No that is why I said I was not sure if using enumerate was the right idea cos the ii was not used

– Starter
Nov 26 '18 at 23:55

Normally if you aren’t going to use something like that you would use _

– Jack Moody
Nov 26 '18 at 23:57

1

Okay now I know. Thank you

– Starter
Nov 27 '18 at 0:04

add a comment |

Do you mean to use i instead of ii in your for loop?

– Jack Moody
Nov 26 '18 at 23:52

No that is why I said I was not sure if using enumerate was the right idea cos the ii was not used

– Starter
Nov 26 '18 at 23:55

Normally if you aren’t going to use something like that you would use _

– Jack Moody
Nov 26 '18 at 23:57

1

Okay now I know. Thank you

– Starter
Nov 27 '18 at 0:04

Do you mean to use i instead of ii in your for loop?

– Jack Moody
Nov 26 '18 at 23:52

No that is why I said I was not sure if using enumerate was the right idea cos the ii was not used

– Starter
Nov 26 '18 at 23:55

Normally if you aren’t going to use something like that you would use _

– Jack Moody
Nov 26 '18 at 23:57

Okay now I know. Thank you

– Starter
Nov 27 '18 at 0:04

add a comment |

1 Answer
1

active

oldest

votes

A better idea is to aggregate your data into a single dataframe and then iterate a groupby object:

# construct dataframe from list of dictionaries

df = pd.DataFrame([[k, *v] for dct in L for k, v in dct.items()])

df[[1, 2]] = df[[1, 2]].apply(pd.to_numeric)



# iterate groupby object and export to separate CSV files

for key, df_key in df.groupby(0):

    df_key.to_csv(f'{key}.csv', index=False, header=False)

answered Nov 26 '18 at 23:51

jpp

101k2163112

add a comment |

Your Answer

StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});

}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53490738%2fpython-script-to-write-list-of-dict-with-multiple-values-to-multiple-files%23new-answer', 'question_page');
}
);

Post as a guest

Name

Required, but never shown

1 Answer
1

active

oldest

votes

1 Answer
1

active

oldest

votes

A better idea is to aggregate your data into a single dataframe and then iterate a groupby object:

# construct dataframe from list of dictionaries

df = pd.DataFrame([[k, *v] for dct in L for k, v in dct.items()])

df[[1, 2]] = df[[1, 2]].apply(pd.to_numeric)



# iterate groupby object and export to separate CSV files

for key, df_key in df.groupby(0):

    df_key.to_csv(f'{key}.csv', index=False, header=False)

answered Nov 26 '18 at 23:51

jpp

101k2163112

add a comment |

A better idea is to aggregate your data into a single dataframe and then iterate a groupby object:

# construct dataframe from list of dictionaries

df = pd.DataFrame([[k, *v] for dct in L for k, v in dct.items()])

df[[1, 2]] = df[[1, 2]].apply(pd.to_numeric)



# iterate groupby object and export to separate CSV files

for key, df_key in df.groupby(0):

    df_key.to_csv(f'{key}.csv', index=False, header=False)

answered Nov 26 '18 at 23:51

jpp

101k2163112

add a comment |

A better idea is to aggregate your data into a single dataframe and then iterate a groupby object:

# construct dataframe from list of dictionaries

df = pd.DataFrame([[k, *v] for dct in L for k, v in dct.items()])

df[[1, 2]] = df[[1, 2]].apply(pd.to_numeric)



# iterate groupby object and export to separate CSV files

for key, df_key in df.groupby(0):

    df_key.to_csv(f'{key}.csv', index=False, header=False)

answered Nov 26 '18 at 23:51

jpp

101k2163112

A better idea is to aggregate your data into a single dataframe and then iterate a groupby object:

# construct dataframe from list of dictionaries

df = pd.DataFrame([[k, *v] for dct in L for k, v in dct.items()])

df[[1, 2]] = df[[1, 2]].apply(pd.to_numeric)



# iterate groupby object and export to separate CSV files

for key, df_key in df.groupby(0):

    df_key.to_csv(f'{key}.csv', index=False, header=False)

answered Nov 26 '18 at 23:51

jpp

101k2163112

answered Nov 26 '18 at 23:51

jpp

101k2163112

answered Nov 26 '18 at 23:51

jpp

101k2163112

answered Nov 26 '18 at 23:51

jpp

101k2163112

add a comment |

draft saved

draft discarded

Thanks for contributing an answer to Stack Overflow!

Please be sure to answer the question. Provide details and share your research!

But avoid …

Asking for help, clarification, or responding to other answers.

Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Name

Required, but never shown

Name

Required, but never shown

This page is only for reference, If you need detailed information, please check here

搜尋此網誌

Btukfyl