TensorFlow: What is the easiest way to incorporate predictions from one model in the training of a new model?
What is the simplest way to use tf.estimator trained model A during the training of another model B?
The weights in model A are fixed. In model B, I would like to take some inputs, compute, feed these results into model A, then do some more computations on the output.
A simple example:
ModelA returns tf.matmul(input,weights)
In ModelB, I would like to do the following:
x1 = tf.matmul(new_inputs,new_weights1)
x2 = modelA(x1) # with fixed weights
return tf.matmul(x2,new_weights2)
But with more complicated models A and B, each of which is trained as a tf.estimator (though I'm happy to not use estimators if there's another easy solution -- I'm using them because I would like to use ML Engine).
This question is related, but the proposed solution does not work for training model B, because the gradients of tf.py_func are [None]. I have tried registering a gradient for tf.py_func, but this fails with
Unsupported object type Tensor
I have also tried tf.import_graph_def for model A, but this seems to load the pretrained graph, but not the actual weights.
python tensorflow google-cloud-ml
edited Nov 30 '18 at 7:52
spicyramen
3,48323872
asked Nov 28 '18 at 18:53
bclearybcleary
84
add a comment |
What is the simplest way to use tf.estimator trained model A during the training of another model B?
The weights in model A are fixed. In model B, I would like to take some inputs, compute, feed these results into model A, then do some more computations on the output.
A simple example:
ModelA returns tf.matmul(input,weights)
In ModelB, I would like to do the following:
x1 = tf.matmul(new_inputs,new_weights1)
x2 = modelA(x1) # with fixed weights
return tf.matmul(x2,new_weights2)
But with more complicated models A and B, each of which is trained as a tf.estimator (though I'm happy to not use estimators if there's another easy solution -- I'm using them because I would like to use ML Engine).
This question is related, but the proposed solution does not work for training model B, because the gradients of tf.py_func are [None]. I have tried registering a gradient for tf.py_func, but this fails with
Unsupported object type Tensor
I have also tried tf.import_graph_def for model A, but this seems to load the pretrained graph, but not the actual weights.
python tensorflow google-cloud-ml
edited Nov 30 '18 at 7:52
spicyramen
3,48323872
asked Nov 28 '18 at 18:53
bclearybcleary
84
add a comment |
What is the simplest way to use tf.estimator trained model A during the training of another model B?
The weights in model A are fixed. In model B, I would like to take some inputs, compute, feed these results into model A, then do some more computations on the output.
A simple example:
ModelA returns tf.matmul(input,weights)
In ModelB, I would like to do the following:
x1 = tf.matmul(new_inputs,new_weights1)
x2 = modelA(x1) # with fixed weights
return tf.matmul(x2,new_weights2)
But with more complicated models A and B, each of which is trained as a tf.estimator (though I'm happy to not use estimators if there's another easy solution -- I'm using them because I would like to use ML Engine).
This question is related, but the proposed solution does not work for training model B, because the gradients of tf.py_func are [None]. I have tried registering a gradient for tf.py_func, but this fails with
Unsupported object type Tensor
I have also tried tf.import_graph_def for model A, but this seems to load the pretrained graph, but not the actual weights.
python tensorflow google-cloud-ml
edited Nov 30 '18 at 7:52
spicyramen
3,48323872
asked Nov 28 '18 at 18:53
bclearybcleary
84
What is the simplest way to use tf.estimator trained model A during the training of another model B?
The weights in model A are fixed. In model B, I would like to take some inputs, compute, feed these results into model A, then do some more computations on the output.
A simple example:
ModelA returns tf.matmul(input,weights)
In ModelB, I would like to do the following:
x1 = tf.matmul(new_inputs,new_weights1)
x2 = modelA(x1) # with fixed weights
return tf.matmul(x2,new_weights2)
But with more complicated models A and B, each of which is trained as a tf.estimator (though I'm happy to not use estimators if there's another easy solution -- I'm using them because I would like to use ML Engine).
This question is related, but the proposed solution does not work for training model B, because the gradients of tf.py_func are [None]. I have tried registering a gradient for tf.py_func, but this fails with
Unsupported object type Tensor
I have also tried tf.import_graph_def for model A, but this seems to load the pretrained graph, but not the actual weights.
python tensorflow google-cloud-ml
python tensorflow google-cloud-ml
edited Nov 30 '18 at 7:52
spicyramen
3,48323872
asked Nov 28 '18 at 18:53
bclearybcleary
84
edited Nov 30 '18 at 7:52
spicyramen
3,48323872
asked Nov 28 '18 at 18:53
bclearybcleary
84
edited Nov 30 '18 at 7:52
spicyramen
3,48323872
edited Nov 30 '18 at 7:52
spicyramen
3,48323872
edited Nov 30 '18 at 7:52
spicyramen
3,48323872
3,48323872
asked Nov 28 '18 at 18:53
bclearybcleary
84
asked Nov 28 '18 at 18:53
bclearybcleary
84
asked Nov 28 '18 at 18:53
bclearybcleary
84
84
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
For model composability, Keras works a whole lot better. You can convert a Keras model to estimator:
https://cloud.google.com/blog/products/gcp/new-in-tensorflow-14-converting-a-keras-model-to-a-tensorflow-estimator
So you can still train on ML Engine.
With Keras, it is then just a matter of loading the intermediate layers' weights and biases from a checkpoint and make that layer non-trainable. See:
Is it possible to save a trained layer to use layer on Keras?
answered Nov 30 '18 at 16:34
LakLak
1,692715
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53526233%2ftensorflow-what-is-the-easiest-way-to-incorporate-predictions-from-one-model-in%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
For model composability, Keras works a whole lot better. You can convert a Keras model to estimator:
https://cloud.google.com/blog/products/gcp/new-in-tensorflow-14-converting-a-keras-model-to-a-tensorflow-estimator
So you can still train on ML Engine.
With Keras, it is then just a matter of loading the intermediate layers' weights and biases from a checkpoint and make that layer non-trainable. See:
Is it possible to save a trained layer to use layer on Keras?
answered Nov 30 '18 at 16:34
LakLak
1,692715
add a comment |
For model composability, Keras works a whole lot better. You can convert a Keras model to estimator:
https://cloud.google.com/blog/products/gcp/new-in-tensorflow-14-converting-a-keras-model-to-a-tensorflow-estimator
So you can still train on ML Engine.
With Keras, it is then just a matter of loading the intermediate layers' weights and biases from a checkpoint and make that layer non-trainable. See:
Is it possible to save a trained layer to use layer on Keras?
answered Nov 30 '18 at 16:34
LakLak
1,692715
add a comment |
For model composability, Keras works a whole lot better. You can convert a Keras model to estimator:
https://cloud.google.com/blog/products/gcp/new-in-tensorflow-14-converting-a-keras-model-to-a-tensorflow-estimator
So you can still train on ML Engine.
With Keras, it is then just a matter of loading the intermediate layers' weights and biases from a checkpoint and make that layer non-trainable. See:
Is it possible to save a trained layer to use layer on Keras?
answered Nov 30 '18 at 16:34
LakLak
1,692715
For model composability, Keras works a whole lot better. You can convert a Keras model to estimator:
https://cloud.google.com/blog/products/gcp/new-in-tensorflow-14-converting-a-keras-model-to-a-tensorflow-estimator
So you can still train on ML Engine.
With Keras, it is then just a matter of loading the intermediate layers' weights and biases from a checkpoint and make that layer non-trainable. See:
Is it possible to save a trained layer to use layer on Keras?
answered Nov 30 '18 at 16:34
LakLak
1,692715
answered Nov 30 '18 at 16:34
LakLak
1,692715
answered Nov 30 '18 at 16:34
LakLak
1,692715
answered Nov 30 '18 at 16:34
LakLak
1,692715
1,692715
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53526233%2ftensorflow-what-is-the-easiest-way-to-incorporate-predictions-from-one-model-in%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
