Dropbox API not returning all files in list












0















I am trying to get a full file list from my dropbox account using the python dropbox API, to later compare the local and remote folders and download only the changed files on my Raspberry PI. So for now I need the current file list on my account. I am trying to accomplish that by using this code:



client = dropbox.Dropbox("ACCESS_TOKEN")

metadata = client.files_list_folder("", recursive=True, include_deleted=False, include_media_info=False)

flist = 

if metadata.has_more == True:

    m1 = metadata.entries

    cur = metadata.cursor

    for i in m1:

        if isinstance(i, dropbox.files.FileMetadata):

            flist.append([i.name])

    # flist now has 2000 items

    m2 = client.files_list_folder_continue(cur)

    while m2.has_more == True:

        for i in m2.entries:

            if isinstance(i, dropbox.files.FileMetadata):

                flist.append([i.name])

        cur = m2.cursor

        m2 = client.files_list_folder_continue(cur)


Which gives me a total of 4971 files. But there are 5252 files in a lot of different folders (probably). I am getting a lot of different numbers. Windows gives me 5252 files using console, using the explorer it gives me 2344 files. Unix gives me 5260 files using console and 5288 files using fhe file manager...

I noticed that a lot of files are missing I added/edited recently (few days ago). After that I tried a slightly different code:



dbx = dropbox.Dropbox("ACCESS_TOKEN")

result = dbx.files_list_folder("", recursive=True, include_deleted=False, include_media_info=False)

file_list = 



def process_entries(entries):

    for entry in entries:

        if isinstance(entry, dropbox.files.FileMetadata):

            file_list.append([entry.name])



process_entries(result.entries)



while result.has_more:

    result = dbx.files_list_folder_continue(result.cursor)

    process_entries(result.entries)



print(len(file_list))


This code gives me only 1997 files, but the list is containing the new files. I don't get why this is happening, as the two codes are doing the same, but neither is capable of giving me the correct list of files.



Edit: Changed the second code accordingly to Gregs comment. Getting to many files now compared to the synchronized locale Dropbox folder.






python dropbox-api







share|improve this question

























  • When you say "there are 5252 files in a lot of different folders (probably)", what you do you mean by the "probably"?

    – Greg
    Nov 26 '18 at 19:08











  • Also, note that not all files from the local filesystem are synced to Dropbox. E.g., see the "Ignored files" section of: dropbox.com/help/syncing-uploads/files-not-syncing That will result in different total numbers. For that reason, comparing with the local filesystem won't work well. Likewise, are you counting folders from the local filesystem? Your API code is only counting files, ignoring the folders themselves.

    – Greg
    Nov 26 '18 at 19:08











  • Finally, I see you copied the second code from dropbox.com/developers/reference/…, but your second call to process_entries has the wrong indentation. The way you have it, it will only run for the last page of results, and not earlier ones.

    – Greg
    Nov 26 '18 at 19:08











  • True @Greg. I changed the indentation of process_entries because Python gave me an indentation error for that line and I couldn't really figure out why.

    – BallerNacken
    Nov 26 '18 at 20:48











  • I guess there is no simple way to mirror the whole dropbox using the dropbox api on the Raspberry Pi? If yes, I would not need to try it this way.

    – BallerNacken
    Nov 26 '18 at 21:00
















0















I am trying to get a full file list from my dropbox account using the python dropbox API, to later compare the local and remote folders and download only the changed files on my Raspberry PI. So for now I need the current file list on my account. I am trying to accomplish that by using this code:



client = dropbox.Dropbox("ACCESS_TOKEN")

metadata = client.files_list_folder("", recursive=True, include_deleted=False, include_media_info=False)

flist = 

if metadata.has_more == True:

    m1 = metadata.entries

    cur = metadata.cursor

    for i in m1:

        if isinstance(i, dropbox.files.FileMetadata):

            flist.append([i.name])

    # flist now has 2000 items

    m2 = client.files_list_folder_continue(cur)

    while m2.has_more == True:

        for i in m2.entries:

            if isinstance(i, dropbox.files.FileMetadata):

                flist.append([i.name])

        cur = m2.cursor

        m2 = client.files_list_folder_continue(cur)


Which gives me a total of 4971 files. But there are 5252 files in a lot of different folders (probably). I am getting a lot of different numbers. Windows gives me 5252 files using console, using the explorer it gives me 2344 files. Unix gives me 5260 files using console and 5288 files using fhe file manager...

I noticed that a lot of files are missing I added/edited recently (few days ago). After that I tried a slightly different code:



dbx = dropbox.Dropbox("ACCESS_TOKEN")

result = dbx.files_list_folder("", recursive=True, include_deleted=False, include_media_info=False)

file_list = 



def process_entries(entries):

    for entry in entries:

        if isinstance(entry, dropbox.files.FileMetadata):

            file_list.append([entry.name])



process_entries(result.entries)



while result.has_more:

    result = dbx.files_list_folder_continue(result.cursor)

    process_entries(result.entries)



print(len(file_list))


This code gives me only 1997 files, but the list is containing the new files. I don't get why this is happening, as the two codes are doing the same, but neither is capable of giving me the correct list of files.



Edit: Changed the second code accordingly to Gregs comment. Getting to many files now compared to the synchronized locale Dropbox folder.






python dropbox-api







share|improve this question

























  • When you say "there are 5252 files in a lot of different folders (probably)", what you do you mean by the "probably"?

    – Greg
    Nov 26 '18 at 19:08











  • Also, note that not all files from the local filesystem are synced to Dropbox. E.g., see the "Ignored files" section of: dropbox.com/help/syncing-uploads/files-not-syncing That will result in different total numbers. For that reason, comparing with the local filesystem won't work well. Likewise, are you counting folders from the local filesystem? Your API code is only counting files, ignoring the folders themselves.

    – Greg
    Nov 26 '18 at 19:08











  • Finally, I see you copied the second code from dropbox.com/developers/reference/…, but your second call to process_entries has the wrong indentation. The way you have it, it will only run for the last page of results, and not earlier ones.

    – Greg
    Nov 26 '18 at 19:08











  • True @Greg. I changed the indentation of process_entries because Python gave me an indentation error for that line and I couldn't really figure out why.

    – BallerNacken
    Nov 26 '18 at 20:48











  • I guess there is no simple way to mirror the whole dropbox using the dropbox api on the Raspberry Pi? If yes, I would not need to try it this way.

    – BallerNacken
    Nov 26 '18 at 21:00














0












0








0








I am trying to get a full file list from my dropbox account using the python dropbox API, to later compare the local and remote folders and download only the changed files on my Raspberry PI. So for now I need the current file list on my account. I am trying to accomplish that by using this code:



client = dropbox.Dropbox("ACCESS_TOKEN")

metadata = client.files_list_folder("", recursive=True, include_deleted=False, include_media_info=False)

flist = 

if metadata.has_more == True:

    m1 = metadata.entries

    cur = metadata.cursor

    for i in m1:

        if isinstance(i, dropbox.files.FileMetadata):

            flist.append([i.name])

    # flist now has 2000 items

    m2 = client.files_list_folder_continue(cur)

    while m2.has_more == True:

        for i in m2.entries:

            if isinstance(i, dropbox.files.FileMetadata):

                flist.append([i.name])

        cur = m2.cursor

        m2 = client.files_list_folder_continue(cur)


Which gives me a total of 4971 files. But there are 5252 files in a lot of different folders (probably). I am getting a lot of different numbers. Windows gives me 5252 files using console, using the explorer it gives me 2344 files. Unix gives me 5260 files using console and 5288 files using fhe file manager...

I noticed that a lot of files are missing I added/edited recently (few days ago). After that I tried a slightly different code:



dbx = dropbox.Dropbox("ACCESS_TOKEN")

result = dbx.files_list_folder("", recursive=True, include_deleted=False, include_media_info=False)

file_list = 



def process_entries(entries):

    for entry in entries:

        if isinstance(entry, dropbox.files.FileMetadata):

            file_list.append([entry.name])



process_entries(result.entries)



while result.has_more:

    result = dbx.files_list_folder_continue(result.cursor)

    process_entries(result.entries)



print(len(file_list))


This code gives me only 1997 files, but the list is containing the new files. I don't get why this is happening, as the two codes are doing the same, but neither is capable of giving me the correct list of files.



Edit: Changed the second code accordingly to Gregs comment. Getting to many files now compared to the synchronized locale Dropbox folder.






python dropbox-api







share|improve this question
















I am trying to get a full file list from my dropbox account using the python dropbox API, to later compare the local and remote folders and download only the changed files on my Raspberry PI. So for now I need the current file list on my account. I am trying to accomplish that by using this code:



client = dropbox.Dropbox("ACCESS_TOKEN")

metadata = client.files_list_folder("", recursive=True, include_deleted=False, include_media_info=False)

flist = 

if metadata.has_more == True:

    m1 = metadata.entries

    cur = metadata.cursor

    for i in m1:

        if isinstance(i, dropbox.files.FileMetadata):

            flist.append([i.name])

    # flist now has 2000 items

    m2 = client.files_list_folder_continue(cur)

    while m2.has_more == True:

        for i in m2.entries:

            if isinstance(i, dropbox.files.FileMetadata):

                flist.append([i.name])

        cur = m2.cursor

        m2 = client.files_list_folder_continue(cur)


Which gives me a total of 4971 files. But there are 5252 files in a lot of different folders (probably). I am getting a lot of different numbers. Windows gives me 5252 files using console, using the explorer it gives me 2344 files. Unix gives me 5260 files using console and 5288 files using fhe file manager...

I noticed that a lot of files are missing I added/edited recently (few days ago). After that I tried a slightly different code:



dbx = dropbox.Dropbox("ACCESS_TOKEN")

result = dbx.files_list_folder("", recursive=True, include_deleted=False, include_media_info=False)

file_list = 



def process_entries(entries):

    for entry in entries:

        if isinstance(entry, dropbox.files.FileMetadata):

            file_list.append([entry.name])



process_entries(result.entries)



while result.has_more:

    result = dbx.files_list_folder_continue(result.cursor)

    process_entries(result.entries)



print(len(file_list))


This code gives me only 1997 files, but the list is containing the new files. I don't get why this is happening, as the two codes are doing the same, but neither is capable of giving me the correct list of files.



Edit: Changed the second code accordingly to Gregs comment. Getting to many files now compared to the synchronized locale Dropbox folder.






python dropbox-api




python dropbox-api






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Nov 26 '18 at 20:59







BallerNacken

















asked Nov 25 '18 at 1:37









BallerNackenBallerNacken

73110




73110













  • When you say "there are 5252 files in a lot of different folders (probably)", what you do you mean by the "probably"?

    – Greg
    Nov 26 '18 at 19:08











  • Also, note that not all files from the local filesystem are synced to Dropbox. E.g., see the "Ignored files" section of: dropbox.com/help/syncing-uploads/files-not-syncing That will result in different total numbers. For that reason, comparing with the local filesystem won't work well. Likewise, are you counting folders from the local filesystem? Your API code is only counting files, ignoring the folders themselves.

    – Greg
    Nov 26 '18 at 19:08











  • Finally, I see you copied the second code from dropbox.com/developers/reference/…, but your second call to process_entries has the wrong indentation. The way you have it, it will only run for the last page of results, and not earlier ones.

    – Greg
    Nov 26 '18 at 19:08











  • True @Greg. I changed the indentation of process_entries because Python gave me an indentation error for that line and I couldn't really figure out why.

    – BallerNacken
    Nov 26 '18 at 20:48











  • I guess there is no simple way to mirror the whole dropbox using the dropbox api on the Raspberry Pi? If yes, I would not need to try it this way.

    – BallerNacken
    Nov 26 '18 at 21:00



















  • When you say "there are 5252 files in a lot of different folders (probably)", what you do you mean by the "probably"?

    – Greg
    Nov 26 '18 at 19:08











  • Also, note that not all files from the local filesystem are synced to Dropbox. E.g., see the "Ignored files" section of: dropbox.com/help/syncing-uploads/files-not-syncing That will result in different total numbers. For that reason, comparing with the local filesystem won't work well. Likewise, are you counting folders from the local filesystem? Your API code is only counting files, ignoring the folders themselves.

    – Greg
    Nov 26 '18 at 19:08











  • Finally, I see you copied the second code from dropbox.com/developers/reference/…, but your second call to process_entries has the wrong indentation. The way you have it, it will only run for the last page of results, and not earlier ones.

    – Greg
    Nov 26 '18 at 19:08











  • True @Greg. I changed the indentation of process_entries because Python gave me an indentation error for that line and I couldn't really figure out why.

    – BallerNacken
    Nov 26 '18 at 20:48











  • I guess there is no simple way to mirror the whole dropbox using the dropbox api on the Raspberry Pi? If yes, I would not need to try it this way.

    – BallerNacken
    Nov 26 '18 at 21:00

















When you say "there are 5252 files in a lot of different folders (probably)", what you do you mean by the "probably"?

– Greg
Nov 26 '18 at 19:08





When you say "there are 5252 files in a lot of different folders (probably)", what you do you mean by the "probably"?

– Greg
Nov 26 '18 at 19:08













Also, note that not all files from the local filesystem are synced to Dropbox. E.g., see the "Ignored files" section of: dropbox.com/help/syncing-uploads/files-not-syncing That will result in different total numbers. For that reason, comparing with the local filesystem won't work well. Likewise, are you counting folders from the local filesystem? Your API code is only counting files, ignoring the folders themselves.

– Greg
Nov 26 '18 at 19:08





Also, note that not all files from the local filesystem are synced to Dropbox. E.g., see the "Ignored files" section of: dropbox.com/help/syncing-uploads/files-not-syncing That will result in different total numbers. For that reason, comparing with the local filesystem won't work well. Likewise, are you counting folders from the local filesystem? Your API code is only counting files, ignoring the folders themselves.

– Greg
Nov 26 '18 at 19:08













Finally, I see you copied the second code from dropbox.com/developers/reference/…, but your second call to process_entries has the wrong indentation. The way you have it, it will only run for the last page of results, and not earlier ones.

– Greg
Nov 26 '18 at 19:08





Finally, I see you copied the second code from dropbox.com/developers/reference/…, but your second call to process_entries has the wrong indentation. The way you have it, it will only run for the last page of results, and not earlier ones.

– Greg
Nov 26 '18 at 19:08













True @Greg. I changed the indentation of process_entries because Python gave me an indentation error for that line and I couldn't really figure out why.

– BallerNacken
Nov 26 '18 at 20:48





True @Greg. I changed the indentation of process_entries because Python gave me an indentation error for that line and I couldn't really figure out why.

– BallerNacken
Nov 26 '18 at 20:48













I guess there is no simple way to mirror the whole dropbox using the dropbox api on the Raspberry Pi? If yes, I would not need to try it this way.

– BallerNacken
Nov 26 '18 at 21:00





I guess there is no simple way to mirror the whole dropbox using the dropbox api on the Raspberry Pi? If yes, I would not need to try it this way.

– BallerNacken
Nov 26 '18 at 21:00












0






active

oldest

votes











Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














draft saved

draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53463959%2fdropbox-api-not-returning-all-files-in-list%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























0






active

oldest

votes








0






active

oldest

votes









active

oldest

votes






active

oldest

votes
















draft saved

draft discarded




















































Thanks for contributing an answer to Stack Overflow!


  • Please be sure to answer the question. Provide details and share your research!

But avoid …



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53463959%2fdropbox-api-not-returning-all-files-in-list%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







-

Popular posts from this blog

A CLEAN and SIMPLE way to add appendices to Table of Contents and bookmarks

Image
0 I found a lot of questions abount appendices and ToC. Many users want appendices to be grouped in an Appendix part, however some problems arise with ToC, hyperref, PDF viewer bookmarks, and so on. There are different solutions which require extra packages, command patching and other extra code, however none of them satisfies me. I almost found an easy way to accomplish a good result, where appendices are added to bookmarks in the right way and hyperref links point to the right page. However, the number of the "Appendix" part page is wrong (it's the number of appendix A). Is there any EASY way to fix that? This is a MWE: documentclass{book} usepackage[nottoc,notlot,notlof]{tocbibind} usepackage{hyperref} begin{document} frontmatter tableofcontents mainmatter part{First} chapter{...
Read more

Calculate evaluation metrics using cross_val_predict sklearn

Image
1 In the sklearn.model_selection.cross_val_predict page it is stated: Generate cross-validated estimates for each input data point. It is not appropriate to pass these predictions into an evaluation metric. Can someone explain what does it mean? If this gives estimate of Y (y prediction) for every Y (true Y), why can't I calculate metrics such as RMSE or coefficient of determination using these results? python scikit-learn cross-validation share | improve this question edited Nov 28 '18 at 17:52 desertnaut 20.3k 7 43 79 ...
Read more

Insert data from modal to MySQL (multiple modal on website)

Image
0 I have problem with inserting data to MySQL from modal. My modal: <a href="#" class="badge badge-pill badge-success">6 komentarzy</a> <a href="#" class="badge badge-pill badge-danger">brak komentarzy</a> <a data-toggle="modal" href="#add_desk_comm_{$desk_['desk_id']}" data-target="#add_desk_comm_{$desk_['desk_id']}" class="ediiit">(dodaj)</a> <div class="modal fade" id="add_desk_comm_{$desk_['desk_id']}" tabindex="-1" role="dialog" aria-labelledby="edit_printer" aria-hidden="true"> <div class="modal-dialog" role="document"> ...
Read more