Python - load an in-memory ZipFile object as bytes











up vote
1
down vote

favorite












I have a script which creates a closed in-memory ZipFile object that I need to post as a bytestring (using requests); how do I do that? I have tried opening the file, which fails with "TypeError: expected str, bytes or os.PathLike object, not ZipFile"



The script works just fine if I write the ZipFile to a file and then open that file for the post data. However it will probably iterate over a couple million files, and that seems like a lot of temp files, and disk activity.



import io
import zipfile
from PIL import Image

z = io.BytesIO()
zfile = zipfile.ZipFile(z,"a")

zipdict = {}

img_loc = "D:/Images/seasons-3.jpg"
im_original = Image.open(img_loc)
imfmt = im_original.format
im = im_original.copy()
im_original.close()
im_out = io.BytesIO()
im.save(im_out,imfmt)
zfile.writestr("seasons-3.jpg",im_out.getvalue())
im_out.close()
zipdict['seasons-3']=zfile
zfile.close()


running with error:



Python 3.6.3 (v3.6.3:2c5fed8, Oct  3 2017, 18:11:49) [MSC v.1900 64 bit (AMD64)] on win32
Type "copyright", "credits" or "license()" for more information.
>>>
>>> zipdict['seasons-3']
<zipfile.ZipFile [closed]>
>>> pl_data = open(zipdict['seasons-3'])
Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
pl_data = open(zipdict['seasons-3'])
TypeError: expected str, bytes or os.PathLike object, not ZipFile
>>>









share|improve this question









New contributor




Tim Achee is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
















  • 1




    Can you paste your code and error traceback?
    – Cheche
    yesterday






  • 2




    "closed in-memory ZipFile" - you're going to have to explain that some more. Did you wrap a ZipFile around a BytesIO or something?
    – user2357112
    yesterday










  • I'm not sure if it helps with an in-memory zip-file (never encountered one in the wild), but you can unzip a single file from an archive: stackoverflow.com/a/46423414/962190
    – Arne
    yesterday










  • @user2357112 that's pretty much exactly what I did. I created a ZipFile and used writestr to add a couple of BytesIO to the ZipFile. Then I added the ZipFile as value to a dict, with key as filename, and closed the ZipFile.
    – Tim Achee
    19 hours ago










  • @TimAchee: Nope, that still doesn't explain things. What, if anything, did you do to put the ZipFile itself in memory? What arguments did you pass to the ZipFile constructor?
    – user2357112
    19 hours ago

















up vote
1
down vote

favorite












I have a script which creates a closed in-memory ZipFile object that I need to post as a bytestring (using requests); how do I do that? I have tried opening the file, which fails with "TypeError: expected str, bytes or os.PathLike object, not ZipFile"



The script works just fine if I write the ZipFile to a file and then open that file for the post data. However it will probably iterate over a couple million files, and that seems like a lot of temp files, and disk activity.



import io
import zipfile
from PIL import Image

z = io.BytesIO()
zfile = zipfile.ZipFile(z,"a")

zipdict = {}

img_loc = "D:/Images/seasons-3.jpg"
im_original = Image.open(img_loc)
imfmt = im_original.format
im = im_original.copy()
im_original.close()
im_out = io.BytesIO()
im.save(im_out,imfmt)
zfile.writestr("seasons-3.jpg",im_out.getvalue())
im_out.close()
zipdict['seasons-3']=zfile
zfile.close()


running with error:



Python 3.6.3 (v3.6.3:2c5fed8, Oct  3 2017, 18:11:49) [MSC v.1900 64 bit (AMD64)] on win32
Type "copyright", "credits" or "license()" for more information.
>>>
>>> zipdict['seasons-3']
<zipfile.ZipFile [closed]>
>>> pl_data = open(zipdict['seasons-3'])
Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
pl_data = open(zipdict['seasons-3'])
TypeError: expected str, bytes or os.PathLike object, not ZipFile
>>>









share|improve this question









New contributor




Tim Achee is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
















  • 1




    Can you paste your code and error traceback?
    – Cheche
    yesterday






  • 2




    "closed in-memory ZipFile" - you're going to have to explain that some more. Did you wrap a ZipFile around a BytesIO or something?
    – user2357112
    yesterday










  • I'm not sure if it helps with an in-memory zip-file (never encountered one in the wild), but you can unzip a single file from an archive: stackoverflow.com/a/46423414/962190
    – Arne
    yesterday










  • @user2357112 that's pretty much exactly what I did. I created a ZipFile and used writestr to add a couple of BytesIO to the ZipFile. Then I added the ZipFile as value to a dict, with key as filename, and closed the ZipFile.
    – Tim Achee
    19 hours ago










  • @TimAchee: Nope, that still doesn't explain things. What, if anything, did you do to put the ZipFile itself in memory? What arguments did you pass to the ZipFile constructor?
    – user2357112
    19 hours ago















up vote
1
down vote

favorite









up vote
1
down vote

favorite











I have a script which creates a closed in-memory ZipFile object that I need to post as a bytestring (using requests); how do I do that? I have tried opening the file, which fails with "TypeError: expected str, bytes or os.PathLike object, not ZipFile"



The script works just fine if I write the ZipFile to a file and then open that file for the post data. However it will probably iterate over a couple million files, and that seems like a lot of temp files, and disk activity.



import io
import zipfile
from PIL import Image

z = io.BytesIO()
zfile = zipfile.ZipFile(z,"a")

zipdict = {}

img_loc = "D:/Images/seasons-3.jpg"
im_original = Image.open(img_loc)
imfmt = im_original.format
im = im_original.copy()
im_original.close()
im_out = io.BytesIO()
im.save(im_out,imfmt)
zfile.writestr("seasons-3.jpg",im_out.getvalue())
im_out.close()
zipdict['seasons-3']=zfile
zfile.close()


running with error:



Python 3.6.3 (v3.6.3:2c5fed8, Oct  3 2017, 18:11:49) [MSC v.1900 64 bit (AMD64)] on win32
Type "copyright", "credits" or "license()" for more information.
>>>
>>> zipdict['seasons-3']
<zipfile.ZipFile [closed]>
>>> pl_data = open(zipdict['seasons-3'])
Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
pl_data = open(zipdict['seasons-3'])
TypeError: expected str, bytes or os.PathLike object, not ZipFile
>>>









share|improve this question









New contributor




Tim Achee is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.











I have a script which creates a closed in-memory ZipFile object that I need to post as a bytestring (using requests); how do I do that? I have tried opening the file, which fails with "TypeError: expected str, bytes or os.PathLike object, not ZipFile"



The script works just fine if I write the ZipFile to a file and then open that file for the post data. However it will probably iterate over a couple million files, and that seems like a lot of temp files, and disk activity.



import io
import zipfile
from PIL import Image

z = io.BytesIO()
zfile = zipfile.ZipFile(z,"a")

zipdict = {}

img_loc = "D:/Images/seasons-3.jpg"
im_original = Image.open(img_loc)
imfmt = im_original.format
im = im_original.copy()
im_original.close()
im_out = io.BytesIO()
im.save(im_out,imfmt)
zfile.writestr("seasons-3.jpg",im_out.getvalue())
im_out.close()
zipdict['seasons-3']=zfile
zfile.close()


running with error:



Python 3.6.3 (v3.6.3:2c5fed8, Oct  3 2017, 18:11:49) [MSC v.1900 64 bit (AMD64)] on win32
Type "copyright", "credits" or "license()" for more information.
>>>
>>> zipdict['seasons-3']
<zipfile.ZipFile [closed]>
>>> pl_data = open(zipdict['seasons-3'])
Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
pl_data = open(zipdict['seasons-3'])
TypeError: expected str, bytes or os.PathLike object, not ZipFile
>>>






python python-requests zipfile bytesio






share|improve this question









New contributor




Tim Achee is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.











share|improve this question









New contributor




Tim Achee is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









share|improve this question




share|improve this question








edited 18 hours ago





















New contributor




Tim Achee is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









asked yesterday









Tim Achee

122




122




New contributor




Tim Achee is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.





New contributor





Tim Achee is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.






Tim Achee is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.








  • 1




    Can you paste your code and error traceback?
    – Cheche
    yesterday






  • 2




    "closed in-memory ZipFile" - you're going to have to explain that some more. Did you wrap a ZipFile around a BytesIO or something?
    – user2357112
    yesterday










  • I'm not sure if it helps with an in-memory zip-file (never encountered one in the wild), but you can unzip a single file from an archive: stackoverflow.com/a/46423414/962190
    – Arne
    yesterday










  • @user2357112 that's pretty much exactly what I did. I created a ZipFile and used writestr to add a couple of BytesIO to the ZipFile. Then I added the ZipFile as value to a dict, with key as filename, and closed the ZipFile.
    – Tim Achee
    19 hours ago










  • @TimAchee: Nope, that still doesn't explain things. What, if anything, did you do to put the ZipFile itself in memory? What arguments did you pass to the ZipFile constructor?
    – user2357112
    19 hours ago
















  • 1




    Can you paste your code and error traceback?
    – Cheche
    yesterday






  • 2




    "closed in-memory ZipFile" - you're going to have to explain that some more. Did you wrap a ZipFile around a BytesIO or something?
    – user2357112
    yesterday










  • I'm not sure if it helps with an in-memory zip-file (never encountered one in the wild), but you can unzip a single file from an archive: stackoverflow.com/a/46423414/962190
    – Arne
    yesterday










  • @user2357112 that's pretty much exactly what I did. I created a ZipFile and used writestr to add a couple of BytesIO to the ZipFile. Then I added the ZipFile as value to a dict, with key as filename, and closed the ZipFile.
    – Tim Achee
    19 hours ago










  • @TimAchee: Nope, that still doesn't explain things. What, if anything, did you do to put the ZipFile itself in memory? What arguments did you pass to the ZipFile constructor?
    – user2357112
    19 hours ago










1




1




Can you paste your code and error traceback?
– Cheche
yesterday




Can you paste your code and error traceback?
– Cheche
yesterday




2




2




"closed in-memory ZipFile" - you're going to have to explain that some more. Did you wrap a ZipFile around a BytesIO or something?
– user2357112
yesterday




"closed in-memory ZipFile" - you're going to have to explain that some more. Did you wrap a ZipFile around a BytesIO or something?
– user2357112
yesterday












I'm not sure if it helps with an in-memory zip-file (never encountered one in the wild), but you can unzip a single file from an archive: stackoverflow.com/a/46423414/962190
– Arne
yesterday




I'm not sure if it helps with an in-memory zip-file (never encountered one in the wild), but you can unzip a single file from an archive: stackoverflow.com/a/46423414/962190
– Arne
yesterday












@user2357112 that's pretty much exactly what I did. I created a ZipFile and used writestr to add a couple of BytesIO to the ZipFile. Then I added the ZipFile as value to a dict, with key as filename, and closed the ZipFile.
– Tim Achee
19 hours ago




@user2357112 that's pretty much exactly what I did. I created a ZipFile and used writestr to add a couple of BytesIO to the ZipFile. Then I added the ZipFile as value to a dict, with key as filename, and closed the ZipFile.
– Tim Achee
19 hours ago












@TimAchee: Nope, that still doesn't explain things. What, if anything, did you do to put the ZipFile itself in memory? What arguments did you pass to the ZipFile constructor?
– user2357112
19 hours ago






@TimAchee: Nope, that still doesn't explain things. What, if anything, did you do to put the ZipFile itself in memory? What arguments did you pass to the ZipFile constructor?
– user2357112
19 hours ago














1 Answer
1






active

oldest

votes

















up vote
0
down vote













zfile is closed. It's useless to you. The thing you need to use now is z, the file-like object that was managing the underlying binary storage for the ZipFile.



You can use z.getvalue() to get a bytestring representing the contents of z, just like you did with im_out, or you can seek back to the beginning with z.seek(0) and use it with the parts of requests that take file-like objects.






share|improve this answer





















  • thank you user2357112, but I need to post the data as a bytestring of a zipped file so I'm not sure this solution would work out.
    – Tim Achee
    17 hours ago










  • @TimAchee: What makes you think this wouldn't work? It sounds like you're misunderstanding the role of the ZipFile object.
    – user2357112
    7 hours ago











Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});






Tim Achee is a new contributor. Be nice, and check out our Code of Conduct.










 

draft saved


draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53397725%2fpython-load-an-in-memory-zipfile-object-as-bytes%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes








up vote
0
down vote













zfile is closed. It's useless to you. The thing you need to use now is z, the file-like object that was managing the underlying binary storage for the ZipFile.



You can use z.getvalue() to get a bytestring representing the contents of z, just like you did with im_out, or you can seek back to the beginning with z.seek(0) and use it with the parts of requests that take file-like objects.






share|improve this answer





















  • thank you user2357112, but I need to post the data as a bytestring of a zipped file so I'm not sure this solution would work out.
    – Tim Achee
    17 hours ago










  • @TimAchee: What makes you think this wouldn't work? It sounds like you're misunderstanding the role of the ZipFile object.
    – user2357112
    7 hours ago















up vote
0
down vote













zfile is closed. It's useless to you. The thing you need to use now is z, the file-like object that was managing the underlying binary storage for the ZipFile.



You can use z.getvalue() to get a bytestring representing the contents of z, just like you did with im_out, or you can seek back to the beginning with z.seek(0) and use it with the parts of requests that take file-like objects.






share|improve this answer





















  • thank you user2357112, but I need to post the data as a bytestring of a zipped file so I'm not sure this solution would work out.
    – Tim Achee
    17 hours ago










  • @TimAchee: What makes you think this wouldn't work? It sounds like you're misunderstanding the role of the ZipFile object.
    – user2357112
    7 hours ago













up vote
0
down vote










up vote
0
down vote









zfile is closed. It's useless to you. The thing you need to use now is z, the file-like object that was managing the underlying binary storage for the ZipFile.



You can use z.getvalue() to get a bytestring representing the contents of z, just like you did with im_out, or you can seek back to the beginning with z.seek(0) and use it with the parts of requests that take file-like objects.






share|improve this answer












zfile is closed. It's useless to you. The thing you need to use now is z, the file-like object that was managing the underlying binary storage for the ZipFile.



You can use z.getvalue() to get a bytestring representing the contents of z, just like you did with im_out, or you can seek back to the beginning with z.seek(0) and use it with the parts of requests that take file-like objects.







share|improve this answer












share|improve this answer



share|improve this answer










answered 17 hours ago









user2357112

147k12149237




147k12149237












  • thank you user2357112, but I need to post the data as a bytestring of a zipped file so I'm not sure this solution would work out.
    – Tim Achee
    17 hours ago










  • @TimAchee: What makes you think this wouldn't work? It sounds like you're misunderstanding the role of the ZipFile object.
    – user2357112
    7 hours ago


















  • thank you user2357112, but I need to post the data as a bytestring of a zipped file so I'm not sure this solution would work out.
    – Tim Achee
    17 hours ago










  • @TimAchee: What makes you think this wouldn't work? It sounds like you're misunderstanding the role of the ZipFile object.
    – user2357112
    7 hours ago
















thank you user2357112, but I need to post the data as a bytestring of a zipped file so I'm not sure this solution would work out.
– Tim Achee
17 hours ago




thank you user2357112, but I need to post the data as a bytestring of a zipped file so I'm not sure this solution would work out.
– Tim Achee
17 hours ago












@TimAchee: What makes you think this wouldn't work? It sounds like you're misunderstanding the role of the ZipFile object.
– user2357112
7 hours ago




@TimAchee: What makes you think this wouldn't work? It sounds like you're misunderstanding the role of the ZipFile object.
– user2357112
7 hours ago










Tim Achee is a new contributor. Be nice, and check out our Code of Conduct.










 

draft saved


draft discarded


















Tim Achee is a new contributor. Be nice, and check out our Code of Conduct.













Tim Achee is a new contributor. Be nice, and check out our Code of Conduct.












Tim Achee is a new contributor. Be nice, and check out our Code of Conduct.















 


draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53397725%2fpython-load-an-in-memory-zipfile-object-as-bytes%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

A CLEAN and SIMPLE way to add appendices to Table of Contents and bookmarks

Calculate evaluation metrics using cross_val_predict sklearn

Insert data from modal to MySQL (multiple modal on website)