Why LOSS is constantly increasing in tensorflow Object Detection API?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box;
}
2
I am using faster_rcnn_resnet50_coco model from the model zoo of tensorflow. I trained the model on my own data (solar images) and as I am observing the training (in tensorboard) the total loss is constantly increasing.
This is surprising to me because the model seems to be pretty successful in detecting the objects.
tensorflow tensorboard object-detection-api
asked Nov 29 '18 at 1:28
AzimAzim
384116
add a comment |
2
I am using faster_rcnn_resnet50_coco model from the model zoo of tensorflow. I trained the model on my own data (solar images) and as I am observing the training (in tensorboard) the total loss is constantly increasing.
This is surprising to me because the model seems to be pretty successful in detecting the objects.
tensorflow tensorboard object-detection-api
asked Nov 29 '18 at 1:28
AzimAzim
384116
Could you give information about which layers you are freezing, and how many classes you are trying to predict?
– thelogicalkoan
Nov 29 '18 at 17:20
As I mentioned, I am training the model on my own data, so no transfer learning is taking place here (that I am aware of, at least). About your other question, I am trying to "detect" (not predict) only one single object type.
– Azim
Nov 30 '18 at 18:13
The total loss is the validation loss. I met the same issue and I tried different learning rates. And if I increase the initial learning rate, the validation loss would drop.
– danyfang
Mar 5 at 10:45
add a comment |
2
2
2
1
I am using faster_rcnn_resnet50_coco model from the model zoo of tensorflow. I trained the model on my own data (solar images) and as I am observing the training (in tensorboard) the total loss is constantly increasing.
This is surprising to me because the model seems to be pretty successful in detecting the objects.
tensorflow tensorboard object-detection-api
asked Nov 29 '18 at 1:28
AzimAzim
384116
I am using faster_rcnn_resnet50_coco model from the model zoo of tensorflow. I trained the model on my own data (solar images) and as I am observing the training (in tensorboard) the total loss is constantly increasing.
This is surprising to me because the model seems to be pretty successful in detecting the objects.
tensorflow tensorboard object-detection-api
tensorflow tensorboard object-detection-api
asked Nov 29 '18 at 1:28
AzimAzim
384116
asked Nov 29 '18 at 1:28
AzimAzim
384116
asked Nov 29 '18 at 1:28
AzimAzim
384116
asked Nov 29 '18 at 1:28
AzimAzim
384116
asked Nov 29 '18 at 1:28
AzimAzim
384116
384116
Could you give information about which layers you are freezing, and how many classes you are trying to predict?
– thelogicalkoan
Nov 29 '18 at 17:20
As I mentioned, I am training the model on my own data, so no transfer learning is taking place here (that I am aware of, at least). About your other question, I am trying to "detect" (not predict) only one single object type.
– Azim
Nov 30 '18 at 18:13
The total loss is the validation loss. I met the same issue and I tried different learning rates. And if I increase the initial learning rate, the validation loss would drop.
– danyfang
Mar 5 at 10:45
add a comment |
Could you give information about which layers you are freezing, and how many classes you are trying to predict?
– thelogicalkoan
Nov 29 '18 at 17:20
As I mentioned, I am training the model on my own data, so no transfer learning is taking place here (that I am aware of, at least). About your other question, I am trying to "detect" (not predict) only one single object type.
– Azim
Nov 30 '18 at 18:13
The total loss is the validation loss. I met the same issue and I tried different learning rates. And if I increase the initial learning rate, the validation loss would drop.
– danyfang
Mar 5 at 10:45
Could you give information about which layers you are freezing, and how many classes you are trying to predict?
– thelogicalkoan
Nov 29 '18 at 17:20
Could you give information about which layers you are freezing, and how many classes you are trying to predict?
– thelogicalkoan
Nov 29 '18 at 17:20
As I mentioned, I am training the model on my own data, so no transfer learning is taking place here (that I am aware of, at least). About your other question, I am trying to "detect" (not predict) only one single object type.
– Azim
Nov 30 '18 at 18:13
As I mentioned, I am training the model on my own data, so no transfer learning is taking place here (that I am aware of, at least). About your other question, I am trying to "detect" (not predict) only one single object type.
– Azim
Nov 30 '18 at 18:13
The total loss is the validation loss. I met the same issue and I tried different learning rates. And if I increase the initial learning rate, the validation loss would drop.
– danyfang
Mar 5 at 10:45
The total loss is the validation loss. I met the same issue and I tried different learning rates. And if I increase the initial learning rate, the validation loss would drop.
– danyfang
Mar 5 at 10:45
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53530561%2fwhy-loss-is-constantly-increasing-in-tensorflow-object-detection-api%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53530561%2fwhy-loss-is-constantly-increasing-in-tensorflow-object-detection-api%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Could you give information about which layers you are freezing, and how many classes you are trying to predict?
– thelogicalkoan
Nov 29 '18 at 17:20
As I mentioned, I am training the model on my own data, so no transfer learning is taking place here (that I am aware of, at least). About your other question, I am trying to "detect" (not predict) only one single object type.
– Azim
Nov 30 '18 at 18:13
The total loss is the validation loss. I met the same issue and I tried different learning rates. And if I increase the initial learning rate, the validation loss would drop.
– danyfang
Mar 5 at 10:45