Btukfyl

I am using a lot of data returned from a WEB-api.The function below calls the API 22 times, decodes and loads json into python format. Then I store the results in a big list of 22 pages with each 100 art objects as data.

fourteen_list = return_14th_century_works_list() 

To limit the necessary API-calls, I want to build a function that stores this list as a file if it is not present, and when it is present I want to load the file from my computer. I came up with the following:

with open('fourteenth_century_list.txt', 'w') as fourteenth_century_file:

    print(fourteen_list, file=fourteenth_century_file)



try:

    with open('fourteenth_century_list.txt', 'r') as fourteenth_century_file:

        fourteenth_list_cache = fourteenth_century_file.read()  

        count_objects(fourteenth_list_cache) 

except FileNotFoundError:

    fourteenth_list = return_14th_century_works_list() Calls API again

    count_objects(fourteen_list)

I use the count_objects function to check if everything still works, but the file that is opened in the try block doesn't seem to return in the same way I saved it; When I run this code, the function call in the try-block returns a type-error. For me this is an indication that the file opened from disk is in a somewhat different format then if I load it directly from the API.

When i call the function count_objects() with the non-cached version of my list, so fourteen_list in this case,works fine.

Does with_open(filename, 'w') and then with_open(filename, 'r') mutate your data, and if not what am I doing wrong here?

The issue here is that when you print the list of dictionaries to a file, you create a string representation of the list. You then read that string back and pass it to count_objects(), but that falls over because it expects a list of dictionaries, not a big string.

Rather than printing, a better approach would be to serialize the list to back JSON - which would preserve its structure. You also want to write the list to the cache in the except block after you've retrieved the data from the API.

import json



try:

    with open('fourteenth_century_list.json', 'r') as fourteenth_century_file:

        fourteenth_list_cache = json.load(fourteenth_century_file)

        count_objects(fourteenth_list_cache) 

except FileNotFoundError:

    # Calls API again

    fourteenth_list = return_14th_century_works_list() 

    count_objects(fourteen_list)



    # Cache the API data

    with open('fourteenth_century_list.json', 'w') as fourteenth_century_file:

        json.dump(fourteen_list, fourteenth_century_file)