Btukfyl

A webscraper written in Python extracts waterleveldata. One read per hour.
When written to a .txt-file using the code below each line is appended with datetime, thus each line takes up something like 20 characters.
Example: "01/01-2010 11:10,-32"

Using the code below results in a file containing data from 01/01-2010 00:10 to 28/02-2010 23:50 which equals something like 60 days. 60 days, with a reading per hour results in 1440 lines and approx. 30000 characters. Microsoft word, however, tell me the file contains 830000 characters on 42210 lines, which fits very well with an observed filesize of 893 kB.

Apparently some lines and characters are hidden somewhere. I cant seem to find them anywhere.

import requests

import time



totaldata =

filnavn='Vandstandsdata_Esbjerg_KDI_TEST_recode.txt'

file = open(filnavn,'w')

file.write("")

file.close()



from datetime import timedelta, date

from bs4 import BeautifulSoup



def daterange(start_date, end_date):

    for n in range(int ((end_date - start_date).days)):

        yield start_date + timedelta(n)



start_date = date(2010, 1, 1)

end_date = date(2010, 3, 1)

values=

datoer=

for single_date in daterange(start_date, end_date):

    valuesTemp=

    datoerTemp=

    dato = single_date.strftime("%d-%m-%y")

    url = "http://kysterne.kyst.dk/pages/10852/waves/showData.asp?targetDay="+dato+"&ident=6401&subGroupGuid=16410"

    page = requests.get(url)

    if page.status_code == 200:

        soup = BeautifulSoup(page.content, 'html.parser')

        dataliste = list(soup.find_all(class_="dataTable"))

        #dataliste =list(dataliste.find_all('td'))

        #dataliste =dataliste[0].getText()

        print(url)

        dataliste = str(dataliste)

        dataliste = dataliste.splitlines()

        dataliste = dataliste[6:] #18

        #print(dataliste[0])

        #print(dataliste[1])

        for e in range (0,len(dataliste),4): #4

            #print(dataliste[e])

            datoerTemp.append(dataliste[e])

            #print(" -------- n")

        for e in range (1,len(dataliste),4): #4

            valuesTemp.append(dataliste[e])

        for e in valuesTemp:

            #print (e)

            e=e[4:]

            e=e[:-5]

            values.append(e)

        for e in datoerTemp:

            #print (e)

            e=e[4:]

            e=e[:-5]

            datoer.append(e)



    file = open(filnavn,'a')

    for i in range(0,len(datoer),6):

        file.write(datoer[i]+","+values[i]+"n")            

    print("- skrevet til filn")

file.close()

print("done")

Ah, heureka.

Seconds before posting this question I realized I forgot to reset the list.
I added:

datoer=

everything now works as intended.
The old code would write data from a given day and all data of all previous days, for each loop in the code.
I hope someone can use this newbie-experience.