How to apply best fit line to time series in python
2
I am trying to apply a best fit line to time series showing NDVI over time but I keep running into errors. my x, in this case, are different dates as strings that are not evenly spaced and y is the NDVI value for use each date.
When I use the poly1d function in numpy I get the following error:
TypeError: ufunc 'add' did not contain a loop with signature matching types
dtype('<U32') dtype('<U32') dtype('<U32')
I have attached a sample of the data set I am working with
# plot Data and and models
plt.subplots(figsize=(20, 10))
plt.xticks(rotation=90)
plt.plot(x,y,'-', color= 'blue')
plt.title('WSC-10-50')
plt.ylabel('NDVI')
plt.xlabel('Date')
plt.plot(np.unique(x), np.poly1d(np.polyfit(x, y, 1))(np.unique(y)))
plt.legend(loc='upper right')
Any help fixing my code or a better way I can get the best fit line for my data?
python time-series best-fit-curve
asked Nov 26 '18 at 1:23
Selena ChavezSelena Chavez
275
add a comment |
2
I am trying to apply a best fit line to time series showing NDVI over time but I keep running into errors. my x, in this case, are different dates as strings that are not evenly spaced and y is the NDVI value for use each date.
When I use the poly1d function in numpy I get the following error:
TypeError: ufunc 'add' did not contain a loop with signature matching types
dtype('<U32') dtype('<U32') dtype('<U32')
I have attached a sample of the data set I am working with
# plot Data and and models
plt.subplots(figsize=(20, 10))
plt.xticks(rotation=90)
plt.plot(x,y,'-', color= 'blue')
plt.title('WSC-10-50')
plt.ylabel('NDVI')
plt.xlabel('Date')
plt.plot(np.unique(x), np.poly1d(np.polyfit(x, y, 1))(np.unique(y)))
plt.legend(loc='upper right')
Any help fixing my code or a better way I can get the best fit line for my data?
python time-series best-fit-curve
asked Nov 26 '18 at 1:23
Selena ChavezSelena Chavez
275
add a comment |
2
2
2
I am trying to apply a best fit line to time series showing NDVI over time but I keep running into errors. my x, in this case, are different dates as strings that are not evenly spaced and y is the NDVI value for use each date.
When I use the poly1d function in numpy I get the following error:
TypeError: ufunc 'add' did not contain a loop with signature matching types
dtype('<U32') dtype('<U32') dtype('<U32')
I have attached a sample of the data set I am working with
# plot Data and and models
plt.subplots(figsize=(20, 10))
plt.xticks(rotation=90)
plt.plot(x,y,'-', color= 'blue')
plt.title('WSC-10-50')
plt.ylabel('NDVI')
plt.xlabel('Date')
plt.plot(np.unique(x), np.poly1d(np.polyfit(x, y, 1))(np.unique(y)))
plt.legend(loc='upper right')
Any help fixing my code or a better way I can get the best fit line for my data?
python time-series best-fit-curve
asked Nov 26 '18 at 1:23
Selena ChavezSelena Chavez
275
I am trying to apply a best fit line to time series showing NDVI over time but I keep running into errors. my x, in this case, are different dates as strings that are not evenly spaced and y is the NDVI value for use each date.
When I use the poly1d function in numpy I get the following error:
TypeError: ufunc 'add' did not contain a loop with signature matching types
dtype('<U32') dtype('<U32') dtype('<U32')
I have attached a sample of the data set I am working with
# plot Data and and models
plt.subplots(figsize=(20, 10))
plt.xticks(rotation=90)
plt.plot(x,y,'-', color= 'blue')
plt.title('WSC-10-50')
plt.ylabel('NDVI')
plt.xlabel('Date')
plt.plot(np.unique(x), np.poly1d(np.polyfit(x, y, 1))(np.unique(y)))
plt.legend(loc='upper right')
Any help fixing my code or a better way I can get the best fit line for my data?
python time-series best-fit-curve
python time-series best-fit-curve
asked Nov 26 '18 at 1:23
Selena ChavezSelena Chavez
275
asked Nov 26 '18 at 1:23
Selena ChavezSelena Chavez
275
edited Nov 26 '18 at 1:42
Selena Chavez
asked Nov 26 '18 at 1:23
Selena ChavezSelena Chavez
275
asked Nov 26 '18 at 1:23
Selena ChavezSelena Chavez
275
asked Nov 26 '18 at 1:23
Selena ChavezSelena Chavez
275
275
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
1
When I apply a best fit line to time series data, I create an evenly spaced line that represents the dates to simplify the regression. So I use np.linspace() to create a set of intervals equal to the number of dates.
Code:
from io import StringIO
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
data = StringIO("""
date value
24-Jan-16 0.786
25-Feb-16 0.781
29-Apr-16 0.786
15-May-16 0.761
16-Jun-16 0.762
04-Sep-16 0.783
22-Oct-16 0.797
""")
df = pd.read_table(data, delim_whitespace=True)
# To read from csv use:
# df = pd.read_csv("/path/to/file.csv")
df.loc[:, "date"] = pd.to_datetime(df.loc[:, "date"], format="%d-%b-%y")
y_values = df.loc[:, "value"]
x_values = np.linspace(0,1,len(df.loc[:, "value"]))
poly_degree = 3
coeffs = np.polyfit(x_values, y_values, poly_degree)
poly_eqn = np.poly1d(coeffs)
y_hat = poly_eqn(x_values)
plt.figure(figsize=(12,8))
plt.plot(df.loc[:, "date"], df.loc[:,"value"], "ro")
plt.plot(df.loc[:, "date"],y_hat)
plt.title('WSC-10-50')
plt.ylabel('NDVI')
plt.xlabel('Date')
plt.savefig("NDVI_plot.png")
Output:
answered Nov 26 '18 at 2:50
DodgeDodge
1,4191922
1
Thank you this is exactly what I needed. I have never used io before. is there a way I could import my date and values from a csv or excel file?
– Selena Chavez
Nov 26 '18 at 19:40
Yup, I just useioto make copying and pasting sample data easy. I will edit the code to include a line for reading fromcsv.
– Dodge
Nov 26 '18 at 19:52
@SelenaChavez it is customary to accept an answer that solves your problem. Please consider clicking the checkmark associated with this post. You can read more on the following help page stackoverflow.com/help/someone-answers
– piRSquared
Nov 27 '18 at 13:59
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53473666%2fhow-to-apply-best-fit-line-to-time-series-in-python%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
1
When I apply a best fit line to time series data, I create an evenly spaced line that represents the dates to simplify the regression. So I use np.linspace() to create a set of intervals equal to the number of dates.
Code:
from io import StringIO
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
data = StringIO("""
date value
24-Jan-16 0.786
25-Feb-16 0.781
29-Apr-16 0.786
15-May-16 0.761
16-Jun-16 0.762
04-Sep-16 0.783
22-Oct-16 0.797
""")
df = pd.read_table(data, delim_whitespace=True)
# To read from csv use:
# df = pd.read_csv("/path/to/file.csv")
df.loc[:, "date"] = pd.to_datetime(df.loc[:, "date"], format="%d-%b-%y")
y_values = df.loc[:, "value"]
x_values = np.linspace(0,1,len(df.loc[:, "value"]))
poly_degree = 3
coeffs = np.polyfit(x_values, y_values, poly_degree)
poly_eqn = np.poly1d(coeffs)
y_hat = poly_eqn(x_values)
plt.figure(figsize=(12,8))
plt.plot(df.loc[:, "date"], df.loc[:,"value"], "ro")
plt.plot(df.loc[:, "date"],y_hat)
plt.title('WSC-10-50')
plt.ylabel('NDVI')
plt.xlabel('Date')
plt.savefig("NDVI_plot.png")
Output:
answered Nov 26 '18 at 2:50
DodgeDodge
1,4191922
1
Thank you this is exactly what I needed. I have never used io before. is there a way I could import my date and values from a csv or excel file?
– Selena Chavez
Nov 26 '18 at 19:40
Yup, I just useioto make copying and pasting sample data easy. I will edit the code to include a line for reading fromcsv.
– Dodge
Nov 26 '18 at 19:52
@SelenaChavez it is customary to accept an answer that solves your problem. Please consider clicking the checkmark associated with this post. You can read more on the following help page stackoverflow.com/help/someone-answers
– piRSquared
Nov 27 '18 at 13:59
add a comment |
1
When I apply a best fit line to time series data, I create an evenly spaced line that represents the dates to simplify the regression. So I use np.linspace() to create a set of intervals equal to the number of dates.
Code:
from io import StringIO
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
data = StringIO("""
date value
24-Jan-16 0.786
25-Feb-16 0.781
29-Apr-16 0.786
15-May-16 0.761
16-Jun-16 0.762
04-Sep-16 0.783
22-Oct-16 0.797
""")
df = pd.read_table(data, delim_whitespace=True)
# To read from csv use:
# df = pd.read_csv("/path/to/file.csv")
df.loc[:, "date"] = pd.to_datetime(df.loc[:, "date"], format="%d-%b-%y")
y_values = df.loc[:, "value"]
x_values = np.linspace(0,1,len(df.loc[:, "value"]))
poly_degree = 3
coeffs = np.polyfit(x_values, y_values, poly_degree)
poly_eqn = np.poly1d(coeffs)
y_hat = poly_eqn(x_values)
plt.figure(figsize=(12,8))
plt.plot(df.loc[:, "date"], df.loc[:,"value"], "ro")
plt.plot(df.loc[:, "date"],y_hat)
plt.title('WSC-10-50')
plt.ylabel('NDVI')
plt.xlabel('Date')
plt.savefig("NDVI_plot.png")
Output:
answered Nov 26 '18 at 2:50
DodgeDodge
1,4191922
1
Thank you this is exactly what I needed. I have never used io before. is there a way I could import my date and values from a csv or excel file?
– Selena Chavez
Nov 26 '18 at 19:40
Yup, I just useioto make copying and pasting sample data easy. I will edit the code to include a line for reading fromcsv.
– Dodge
Nov 26 '18 at 19:52
@SelenaChavez it is customary to accept an answer that solves your problem. Please consider clicking the checkmark associated with this post. You can read more on the following help page stackoverflow.com/help/someone-answers
– piRSquared
Nov 27 '18 at 13:59
add a comment |
1
1
1
When I apply a best fit line to time series data, I create an evenly spaced line that represents the dates to simplify the regression. So I use np.linspace() to create a set of intervals equal to the number of dates.
Code:
from io import StringIO
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
data = StringIO("""
date value
24-Jan-16 0.786
25-Feb-16 0.781
29-Apr-16 0.786
15-May-16 0.761
16-Jun-16 0.762
04-Sep-16 0.783
22-Oct-16 0.797
""")
df = pd.read_table(data, delim_whitespace=True)
# To read from csv use:
# df = pd.read_csv("/path/to/file.csv")
df.loc[:, "date"] = pd.to_datetime(df.loc[:, "date"], format="%d-%b-%y")
y_values = df.loc[:, "value"]
x_values = np.linspace(0,1,len(df.loc[:, "value"]))
poly_degree = 3
coeffs = np.polyfit(x_values, y_values, poly_degree)
poly_eqn = np.poly1d(coeffs)
y_hat = poly_eqn(x_values)
plt.figure(figsize=(12,8))
plt.plot(df.loc[:, "date"], df.loc[:,"value"], "ro")
plt.plot(df.loc[:, "date"],y_hat)
plt.title('WSC-10-50')
plt.ylabel('NDVI')
plt.xlabel('Date')
plt.savefig("NDVI_plot.png")
Output:
answered Nov 26 '18 at 2:50
DodgeDodge
1,4191922
When I apply a best fit line to time series data, I create an evenly spaced line that represents the dates to simplify the regression. So I use np.linspace() to create a set of intervals equal to the number of dates.
Code:
from io import StringIO
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
data = StringIO("""
date value
24-Jan-16 0.786
25-Feb-16 0.781
29-Apr-16 0.786
15-May-16 0.761
16-Jun-16 0.762
04-Sep-16 0.783
22-Oct-16 0.797
""")
df = pd.read_table(data, delim_whitespace=True)
# To read from csv use:
# df = pd.read_csv("/path/to/file.csv")
df.loc[:, "date"] = pd.to_datetime(df.loc[:, "date"], format="%d-%b-%y")
y_values = df.loc[:, "value"]
x_values = np.linspace(0,1,len(df.loc[:, "value"]))
poly_degree = 3
coeffs = np.polyfit(x_values, y_values, poly_degree)
poly_eqn = np.poly1d(coeffs)
y_hat = poly_eqn(x_values)
plt.figure(figsize=(12,8))
plt.plot(df.loc[:, "date"], df.loc[:,"value"], "ro")
plt.plot(df.loc[:, "date"],y_hat)
plt.title('WSC-10-50')
plt.ylabel('NDVI')
plt.xlabel('Date')
plt.savefig("NDVI_plot.png")
Output:
answered Nov 26 '18 at 2:50
DodgeDodge
1,4191922
edited Nov 26 '18 at 20:49
answered Nov 26 '18 at 2:50
DodgeDodge
1,4191922
answered Nov 26 '18 at 2:50
DodgeDodge
1,4191922
answered Nov 26 '18 at 2:50
DodgeDodge
1,4191922
1,4191922
1
Thank you this is exactly what I needed. I have never used io before. is there a way I could import my date and values from a csv or excel file?
– Selena Chavez
Nov 26 '18 at 19:40
Yup, I just useioto make copying and pasting sample data easy. I will edit the code to include a line for reading fromcsv.
– Dodge
Nov 26 '18 at 19:52
@SelenaChavez it is customary to accept an answer that solves your problem. Please consider clicking the checkmark associated with this post. You can read more on the following help page stackoverflow.com/help/someone-answers
– piRSquared
Nov 27 '18 at 13:59
add a comment |
1
Thank you this is exactly what I needed. I have never used io before. is there a way I could import my date and values from a csv or excel file?
– Selena Chavez
Nov 26 '18 at 19:40
Yup, I just useioto make copying and pasting sample data easy. I will edit the code to include a line for reading fromcsv.
– Dodge
Nov 26 '18 at 19:52
@SelenaChavez it is customary to accept an answer that solves your problem. Please consider clicking the checkmark associated with this post. You can read more on the following help page stackoverflow.com/help/someone-answers
– piRSquared
Nov 27 '18 at 13:59
1
1
Thank you this is exactly what I needed. I have never used io before. is there a way I could import my date and values from a csv or excel file?
– Selena Chavez
Nov 26 '18 at 19:40
Thank you this is exactly what I needed. I have never used io before. is there a way I could import my date and values from a csv or excel file?
– Selena Chavez
Nov 26 '18 at 19:40
Yup, I just use
io to make copying and pasting sample data easy. I will edit the code to include a line for reading from csv.– Dodge
Nov 26 '18 at 19:52
Yup, I just use
io to make copying and pasting sample data easy. I will edit the code to include a line for reading from csv.– Dodge
Nov 26 '18 at 19:52
@SelenaChavez it is customary to accept an answer that solves your problem. Please consider clicking the checkmark associated with this post. You can read more on the following help page stackoverflow.com/help/someone-answers
– piRSquared
Nov 27 '18 at 13:59
@SelenaChavez it is customary to accept an answer that solves your problem. Please consider clicking the checkmark associated with this post. You can read more on the following help page stackoverflow.com/help/someone-answers
– piRSquared
Nov 27 '18 at 13:59
add a comment |
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53473666%2fhow-to-apply-best-fit-line-to-time-series-in-python%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
