Frequency change irregular time series
0
I have an irregular time series where the frequency should be 2 sec but it's occasionally 4 seconds. When I try to change the time step in minutes I have an error... the desired output should be
"2017-04-01 00:03:00 5.6179"
Only the full minute should be displayed but instead I got two values
DateTime<-c("2017-04-01 00:02:38 CEST", "2017-04-01 00:02:40 CEST","2017-04-01 00:02:42 CEST",
"2017-04-01 00:02:46 CEST","2017-04-01 00:02:48 CEST" ,"2017-04-01 00:02:50 CEST" ,
"2017-04-01 00:02:52 CEST","2017-04-01 00:02:54 CEST","2017-04-01 00:02:56 CEST" ,
"2017-04-01 00:02:58 CEST","2017-04-01 00:03:00 CEST", "2017-04-01 00:03:02 CEST")
Time<-as.POSIXct(DateTime)
set.seed(1)
Values<-runif(12,5,8)
df<-data.frame(Time, Values)
library(xts)
dfSeconds <- xts(df[,2], order.by=as.POSIXct(df[,1], format='%y/%m/%d %H:%M: %S'))
dfMinutes<-to.minutes(dfSeconds,OHLC=FALSE,indexAt="startof")
dfMinutes
[,1]
2017-04-01 00:02:58 6.692016
2017-04-01 00:03:02 7.518400
Not sure how to solve the issue in the huge data frame.
r time
asked Nov 27 '18 at 16:15
kelamahimkelamahim
931110
add a comment |
0
I have an irregular time series where the frequency should be 2 sec but it's occasionally 4 seconds. When I try to change the time step in minutes I have an error... the desired output should be
"2017-04-01 00:03:00 5.6179"
Only the full minute should be displayed but instead I got two values
DateTime<-c("2017-04-01 00:02:38 CEST", "2017-04-01 00:02:40 CEST","2017-04-01 00:02:42 CEST",
"2017-04-01 00:02:46 CEST","2017-04-01 00:02:48 CEST" ,"2017-04-01 00:02:50 CEST" ,
"2017-04-01 00:02:52 CEST","2017-04-01 00:02:54 CEST","2017-04-01 00:02:56 CEST" ,
"2017-04-01 00:02:58 CEST","2017-04-01 00:03:00 CEST", "2017-04-01 00:03:02 CEST")
Time<-as.POSIXct(DateTime)
set.seed(1)
Values<-runif(12,5,8)
df<-data.frame(Time, Values)
library(xts)
dfSeconds <- xts(df[,2], order.by=as.POSIXct(df[,1], format='%y/%m/%d %H:%M: %S'))
dfMinutes<-to.minutes(dfSeconds,OHLC=FALSE,indexAt="startof")
dfMinutes
[,1]
2017-04-01 00:02:58 6.692016
2017-04-01 00:03:02 7.518400
Not sure how to solve the issue in the huge data frame.
r time
asked Nov 27 '18 at 16:15
kelamahimkelamahim
931110
add a comment |
0
0
0
I have an irregular time series where the frequency should be 2 sec but it's occasionally 4 seconds. When I try to change the time step in minutes I have an error... the desired output should be
"2017-04-01 00:03:00 5.6179"
Only the full minute should be displayed but instead I got two values
DateTime<-c("2017-04-01 00:02:38 CEST", "2017-04-01 00:02:40 CEST","2017-04-01 00:02:42 CEST",
"2017-04-01 00:02:46 CEST","2017-04-01 00:02:48 CEST" ,"2017-04-01 00:02:50 CEST" ,
"2017-04-01 00:02:52 CEST","2017-04-01 00:02:54 CEST","2017-04-01 00:02:56 CEST" ,
"2017-04-01 00:02:58 CEST","2017-04-01 00:03:00 CEST", "2017-04-01 00:03:02 CEST")
Time<-as.POSIXct(DateTime)
set.seed(1)
Values<-runif(12,5,8)
df<-data.frame(Time, Values)
library(xts)
dfSeconds <- xts(df[,2], order.by=as.POSIXct(df[,1], format='%y/%m/%d %H:%M: %S'))
dfMinutes<-to.minutes(dfSeconds,OHLC=FALSE,indexAt="startof")
dfMinutes
[,1]
2017-04-01 00:02:58 6.692016
2017-04-01 00:03:02 7.518400
Not sure how to solve the issue in the huge data frame.
r time
asked Nov 27 '18 at 16:15
kelamahimkelamahim
931110
I have an irregular time series where the frequency should be 2 sec but it's occasionally 4 seconds. When I try to change the time step in minutes I have an error... the desired output should be
"2017-04-01 00:03:00 5.6179"
Only the full minute should be displayed but instead I got two values
DateTime<-c("2017-04-01 00:02:38 CEST", "2017-04-01 00:02:40 CEST","2017-04-01 00:02:42 CEST",
"2017-04-01 00:02:46 CEST","2017-04-01 00:02:48 CEST" ,"2017-04-01 00:02:50 CEST" ,
"2017-04-01 00:02:52 CEST","2017-04-01 00:02:54 CEST","2017-04-01 00:02:56 CEST" ,
"2017-04-01 00:02:58 CEST","2017-04-01 00:03:00 CEST", "2017-04-01 00:03:02 CEST")
Time<-as.POSIXct(DateTime)
set.seed(1)
Values<-runif(12,5,8)
df<-data.frame(Time, Values)
library(xts)
dfSeconds <- xts(df[,2], order.by=as.POSIXct(df[,1], format='%y/%m/%d %H:%M: %S'))
dfMinutes<-to.minutes(dfSeconds,OHLC=FALSE,indexAt="startof")
dfMinutes
[,1]
2017-04-01 00:02:58 6.692016
2017-04-01 00:03:02 7.518400
Not sure how to solve the issue in the huge data frame.
r time
r time
asked Nov 27 '18 at 16:15
kelamahimkelamahim
931110
asked Nov 27 '18 at 16:15
kelamahimkelamahim
931110
asked Nov 27 '18 at 16:15
kelamahimkelamahim
931110
asked Nov 27 '18 at 16:15
kelamahimkelamahim
931110
asked Nov 27 '18 at 16:15
kelamahimkelamahim
931110
931110
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
1
If you do not want to round, but only remove the seconds, this could help, removing also the useless column:
df$Time <- floor_date(df$Time,unit ="minute")
But if you want to have only the row with the minute, you can try this:
library(dplyr)
library(lubridate)
dat <- df %>%
filter(second(Time)==0) %>% # filter only the minutes with 00 seconds
data.frame()
Then convert to an xts object:
library(xts)
xts(dat[,2], order.by=as.POSIXct(dat[,1], format='%y/%m/%d %H:%M:%S'))
[,1]
2017-04-01 00:03:00 5.617924
This mean that you'll not have data if the seconds are all different from 00, but looking at your example, it seems you need that.
answered Nov 28 '18 at 8:27
s_ts_t
3,50621032
Thank you for the answer but I want to have just the minute frequency so from all the data the output should be just the value at the exact full minute ""2017-04-01 00:03:00 5.6179""
– kelamahim
Nov 28 '18 at 9:09
You are welcome! Sorry I do not understand, could you point out what on the column Time_rounded is wrong? Or simply you need two columns, so that is the mistake? However I've edited, maybe you mean that.
– s_t
Nov 28 '18 at 9:11
It's acctually just changing the time step of the data on 1 minutes interval and deleting all the other values so the new data frame should have only a single value:""2017-04-01 00:03:00 5.6179""
– kelamahim
Nov 28 '18 at 9:19
Edited, a bit hacky, but it seems to fit, let me know.
– s_t
Nov 28 '18 at 9:49
1
Works perfectly fine though need to check it on huge dataset
– kelamahim
Nov 28 '18 at 10:12
|
show 2 more comments
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53503824%2ffrequency-change-irregular-time-series%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
1
If you do not want to round, but only remove the seconds, this could help, removing also the useless column:
df$Time <- floor_date(df$Time,unit ="minute")
But if you want to have only the row with the minute, you can try this:
library(dplyr)
library(lubridate)
dat <- df %>%
filter(second(Time)==0) %>% # filter only the minutes with 00 seconds
data.frame()
Then convert to an xts object:
library(xts)
xts(dat[,2], order.by=as.POSIXct(dat[,1], format='%y/%m/%d %H:%M:%S'))
[,1]
2017-04-01 00:03:00 5.617924
This mean that you'll not have data if the seconds are all different from 00, but looking at your example, it seems you need that.
answered Nov 28 '18 at 8:27
s_ts_t
3,50621032
Thank you for the answer but I want to have just the minute frequency so from all the data the output should be just the value at the exact full minute ""2017-04-01 00:03:00 5.6179""
– kelamahim
Nov 28 '18 at 9:09
You are welcome! Sorry I do not understand, could you point out what on the column Time_rounded is wrong? Or simply you need two columns, so that is the mistake? However I've edited, maybe you mean that.
– s_t
Nov 28 '18 at 9:11
It's acctually just changing the time step of the data on 1 minutes interval and deleting all the other values so the new data frame should have only a single value:""2017-04-01 00:03:00 5.6179""
– kelamahim
Nov 28 '18 at 9:19
Edited, a bit hacky, but it seems to fit, let me know.
– s_t
Nov 28 '18 at 9:49
1
Works perfectly fine though need to check it on huge dataset
– kelamahim
Nov 28 '18 at 10:12
|
show 2 more comments
1
If you do not want to round, but only remove the seconds, this could help, removing also the useless column:
df$Time <- floor_date(df$Time,unit ="minute")
But if you want to have only the row with the minute, you can try this:
library(dplyr)
library(lubridate)
dat <- df %>%
filter(second(Time)==0) %>% # filter only the minutes with 00 seconds
data.frame()
Then convert to an xts object:
library(xts)
xts(dat[,2], order.by=as.POSIXct(dat[,1], format='%y/%m/%d %H:%M:%S'))
[,1]
2017-04-01 00:03:00 5.617924
This mean that you'll not have data if the seconds are all different from 00, but looking at your example, it seems you need that.
answered Nov 28 '18 at 8:27
s_ts_t
3,50621032
Thank you for the answer but I want to have just the minute frequency so from all the data the output should be just the value at the exact full minute ""2017-04-01 00:03:00 5.6179""
– kelamahim
Nov 28 '18 at 9:09
You are welcome! Sorry I do not understand, could you point out what on the column Time_rounded is wrong? Or simply you need two columns, so that is the mistake? However I've edited, maybe you mean that.
– s_t
Nov 28 '18 at 9:11
It's acctually just changing the time step of the data on 1 minutes interval and deleting all the other values so the new data frame should have only a single value:""2017-04-01 00:03:00 5.6179""
– kelamahim
Nov 28 '18 at 9:19
Edited, a bit hacky, but it seems to fit, let me know.
– s_t
Nov 28 '18 at 9:49
1
Works perfectly fine though need to check it on huge dataset
– kelamahim
Nov 28 '18 at 10:12
|
show 2 more comments
1
1
1
If you do not want to round, but only remove the seconds, this could help, removing also the useless column:
df$Time <- floor_date(df$Time,unit ="minute")
But if you want to have only the row with the minute, you can try this:
library(dplyr)
library(lubridate)
dat <- df %>%
filter(second(Time)==0) %>% # filter only the minutes with 00 seconds
data.frame()
Then convert to an xts object:
library(xts)
xts(dat[,2], order.by=as.POSIXct(dat[,1], format='%y/%m/%d %H:%M:%S'))
[,1]
2017-04-01 00:03:00 5.617924
This mean that you'll not have data if the seconds are all different from 00, but looking at your example, it seems you need that.
answered Nov 28 '18 at 8:27
s_ts_t
3,50621032
If you do not want to round, but only remove the seconds, this could help, removing also the useless column:
df$Time <- floor_date(df$Time,unit ="minute")
But if you want to have only the row with the minute, you can try this:
library(dplyr)
library(lubridate)
dat <- df %>%
filter(second(Time)==0) %>% # filter only the minutes with 00 seconds
data.frame()
Then convert to an xts object:
library(xts)
xts(dat[,2], order.by=as.POSIXct(dat[,1], format='%y/%m/%d %H:%M:%S'))
[,1]
2017-04-01 00:03:00 5.617924
This mean that you'll not have data if the seconds are all different from 00, but looking at your example, it seems you need that.
answered Nov 28 '18 at 8:27
s_ts_t
3,50621032
edited Nov 28 '18 at 9:49
answered Nov 28 '18 at 8:27
s_ts_t
3,50621032
answered Nov 28 '18 at 8:27
s_ts_t
3,50621032
answered Nov 28 '18 at 8:27
s_ts_t
3,50621032
3,50621032
Thank you for the answer but I want to have just the minute frequency so from all the data the output should be just the value at the exact full minute ""2017-04-01 00:03:00 5.6179""
– kelamahim
Nov 28 '18 at 9:09
You are welcome! Sorry I do not understand, could you point out what on the column Time_rounded is wrong? Or simply you need two columns, so that is the mistake? However I've edited, maybe you mean that.
– s_t
Nov 28 '18 at 9:11
It's acctually just changing the time step of the data on 1 minutes interval and deleting all the other values so the new data frame should have only a single value:""2017-04-01 00:03:00 5.6179""
– kelamahim
Nov 28 '18 at 9:19
Edited, a bit hacky, but it seems to fit, let me know.
– s_t
Nov 28 '18 at 9:49
1
Works perfectly fine though need to check it on huge dataset
– kelamahim
Nov 28 '18 at 10:12
|
show 2 more comments
Thank you for the answer but I want to have just the minute frequency so from all the data the output should be just the value at the exact full minute ""2017-04-01 00:03:00 5.6179""
– kelamahim
Nov 28 '18 at 9:09
You are welcome! Sorry I do not understand, could you point out what on the column Time_rounded is wrong? Or simply you need two columns, so that is the mistake? However I've edited, maybe you mean that.
– s_t
Nov 28 '18 at 9:11
It's acctually just changing the time step of the data on 1 minutes interval and deleting all the other values so the new data frame should have only a single value:""2017-04-01 00:03:00 5.6179""
– kelamahim
Nov 28 '18 at 9:19
Edited, a bit hacky, but it seems to fit, let me know.
– s_t
Nov 28 '18 at 9:49
1
Works perfectly fine though need to check it on huge dataset
– kelamahim
Nov 28 '18 at 10:12
Thank you for the answer but I want to have just the minute frequency so from all the data the output should be just the value at the exact full minute ""2017-04-01 00:03:00 5.6179""
– kelamahim
Nov 28 '18 at 9:09
Thank you for the answer but I want to have just the minute frequency so from all the data the output should be just the value at the exact full minute ""2017-04-01 00:03:00 5.6179""
– kelamahim
Nov 28 '18 at 9:09
You are welcome! Sorry I do not understand, could you point out what on the column Time_rounded is wrong? Or simply you need two columns, so that is the mistake? However I've edited, maybe you mean that.
– s_t
Nov 28 '18 at 9:11
You are welcome! Sorry I do not understand, could you point out what on the column Time_rounded is wrong? Or simply you need two columns, so that is the mistake? However I've edited, maybe you mean that.
– s_t
Nov 28 '18 at 9:11
It's acctually just changing the time step of the data on 1 minutes interval and deleting all the other values so the new data frame should have only a single value:""2017-04-01 00:03:00 5.6179""
– kelamahim
Nov 28 '18 at 9:19
It's acctually just changing the time step of the data on 1 minutes interval and deleting all the other values so the new data frame should have only a single value:""2017-04-01 00:03:00 5.6179""
– kelamahim
Nov 28 '18 at 9:19
Edited, a bit hacky, but it seems to fit, let me know.
– s_t
Nov 28 '18 at 9:49
Edited, a bit hacky, but it seems to fit, let me know.
– s_t
Nov 28 '18 at 9:49
1
1
Works perfectly fine though need to check it on huge dataset
– kelamahim
Nov 28 '18 at 10:12
Works perfectly fine though need to check it on huge dataset
– kelamahim
Nov 28 '18 at 10:12
|
show 2 more comments
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53503824%2ffrequency-change-irregular-time-series%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
