Incompatible shapes between op input and calculated input gradient: conv1d_transpose
I am having the following error when computing the gradient.
ValueError: Incompatible shapes between op input and calculated input gradient. Forward operation: write_8/conv1d_transpose. Input index: 2. Original input shape: (100, 1, 10, 10, 100). Calculated input gradient shape: (100, 1, 13, 10)
def conv1d_layer(inp, filters, stride, output_shape=None, transpose=False):
if output_shape is not None:
output_shape = tf.constant(output_shape)
if transpose: return tf.contrib.nn.conv1d_transpose(inp,
filters, output_shape, stride, padding='SAME')
return tf.nn.conv1d(inp, filters, stride, 'SAME')
def forward(input, batch_size=100):
output = tf.reshape(output, [-1, 10, 10, 100])
init = tf.truncated_normal_initializer(stddev=1.)
filter = tf.get_variable('f1', [5, 1, 10], initializer=init)
output = conv1d_layer(output, filter, 2, [batch_size, 25, 25], True)
filter = tf.get_variable('f2', [5, 1, 25], initializer=init)
output = conv1d_layer(output, filter, 2, [batch_size, 50, 50], True)
filter = tf.get_variable('f3', [5, 1, 50], initializer=init)
out = conv1d_layer(output, filter, 2, [batch_size, 100, 100], True)
out = tf.reshape(out, [-1, 100 * 100])
vocab_size = 10000
w = tf.get_variable('w', [100*100, vocab_size], initializer=init)
b = tf.Variable(tf.zeros([vocab_size])) # biases
return tf.matmul(output, w) + b
When given some input, I do a forward pass by calling conv1d_layer. Then when I compute the gradient as follows, that's when the error happens.
output = forward(input)
loss = tf.nn.softmax_cross_entropy_with_logits_v2(logits=output, labels=classes)
optimizer = tf.train.AdamOptimizer(lr, beta1=0.5)
grads = optimizer.compute_gradients(loss)
I suspect my filters are wrong (I want to get a final shape of [100, 100, vocab_size] where vocab_size = 10000 as per above. Any help would be highly appreciated.
filter gradient transpose convolution
asked Nov 24 '18 at 17:49
NoorNoor
114
add a comment |
I am having the following error when computing the gradient.
ValueError: Incompatible shapes between op input and calculated input gradient. Forward operation: write_8/conv1d_transpose. Input index: 2. Original input shape: (100, 1, 10, 10, 100). Calculated input gradient shape: (100, 1, 13, 10)
def conv1d_layer(inp, filters, stride, output_shape=None, transpose=False):
if output_shape is not None:
output_shape = tf.constant(output_shape)
if transpose: return tf.contrib.nn.conv1d_transpose(inp,
filters, output_shape, stride, padding='SAME')
return tf.nn.conv1d(inp, filters, stride, 'SAME')
def forward(input, batch_size=100):
output = tf.reshape(output, [-1, 10, 10, 100])
init = tf.truncated_normal_initializer(stddev=1.)
filter = tf.get_variable('f1', [5, 1, 10], initializer=init)
output = conv1d_layer(output, filter, 2, [batch_size, 25, 25], True)
filter = tf.get_variable('f2', [5, 1, 25], initializer=init)
output = conv1d_layer(output, filter, 2, [batch_size, 50, 50], True)
filter = tf.get_variable('f3', [5, 1, 50], initializer=init)
out = conv1d_layer(output, filter, 2, [batch_size, 100, 100], True)
out = tf.reshape(out, [-1, 100 * 100])
vocab_size = 10000
w = tf.get_variable('w', [100*100, vocab_size], initializer=init)
b = tf.Variable(tf.zeros([vocab_size])) # biases
return tf.matmul(output, w) + b
When given some input, I do a forward pass by calling conv1d_layer. Then when I compute the gradient as follows, that's when the error happens.
output = forward(input)
loss = tf.nn.softmax_cross_entropy_with_logits_v2(logits=output, labels=classes)
optimizer = tf.train.AdamOptimizer(lr, beta1=0.5)
grads = optimizer.compute_gradients(loss)
I suspect my filters are wrong (I want to get a final shape of [100, 100, vocab_size] where vocab_size = 10000 as per above. Any help would be highly appreciated.
filter gradient transpose convolution
asked Nov 24 '18 at 17:49
NoorNoor
114
I meant input in the forward method instead of output, and return tf.matmul(out, w) + b
– Noor
Nov 24 '18 at 17:51
add a comment |
I am having the following error when computing the gradient.
ValueError: Incompatible shapes between op input and calculated input gradient. Forward operation: write_8/conv1d_transpose. Input index: 2. Original input shape: (100, 1, 10, 10, 100). Calculated input gradient shape: (100, 1, 13, 10)
def conv1d_layer(inp, filters, stride, output_shape=None, transpose=False):
if output_shape is not None:
output_shape = tf.constant(output_shape)
if transpose: return tf.contrib.nn.conv1d_transpose(inp,
filters, output_shape, stride, padding='SAME')
return tf.nn.conv1d(inp, filters, stride, 'SAME')
def forward(input, batch_size=100):
output = tf.reshape(output, [-1, 10, 10, 100])
init = tf.truncated_normal_initializer(stddev=1.)
filter = tf.get_variable('f1', [5, 1, 10], initializer=init)
output = conv1d_layer(output, filter, 2, [batch_size, 25, 25], True)
filter = tf.get_variable('f2', [5, 1, 25], initializer=init)
output = conv1d_layer(output, filter, 2, [batch_size, 50, 50], True)
filter = tf.get_variable('f3', [5, 1, 50], initializer=init)
out = conv1d_layer(output, filter, 2, [batch_size, 100, 100], True)
out = tf.reshape(out, [-1, 100 * 100])
vocab_size = 10000
w = tf.get_variable('w', [100*100, vocab_size], initializer=init)
b = tf.Variable(tf.zeros([vocab_size])) # biases
return tf.matmul(output, w) + b
When given some input, I do a forward pass by calling conv1d_layer. Then when I compute the gradient as follows, that's when the error happens.
output = forward(input)
loss = tf.nn.softmax_cross_entropy_with_logits_v2(logits=output, labels=classes)
optimizer = tf.train.AdamOptimizer(lr, beta1=0.5)
grads = optimizer.compute_gradients(loss)
I suspect my filters are wrong (I want to get a final shape of [100, 100, vocab_size] where vocab_size = 10000 as per above. Any help would be highly appreciated.
filter gradient transpose convolution
asked Nov 24 '18 at 17:49
NoorNoor
114
I am having the following error when computing the gradient.
ValueError: Incompatible shapes between op input and calculated input gradient. Forward operation: write_8/conv1d_transpose. Input index: 2. Original input shape: (100, 1, 10, 10, 100). Calculated input gradient shape: (100, 1, 13, 10)
def conv1d_layer(inp, filters, stride, output_shape=None, transpose=False):
if output_shape is not None:
output_shape = tf.constant(output_shape)
if transpose: return tf.contrib.nn.conv1d_transpose(inp,
filters, output_shape, stride, padding='SAME')
return tf.nn.conv1d(inp, filters, stride, 'SAME')
def forward(input, batch_size=100):
output = tf.reshape(output, [-1, 10, 10, 100])
init = tf.truncated_normal_initializer(stddev=1.)
filter = tf.get_variable('f1', [5, 1, 10], initializer=init)
output = conv1d_layer(output, filter, 2, [batch_size, 25, 25], True)
filter = tf.get_variable('f2', [5, 1, 25], initializer=init)
output = conv1d_layer(output, filter, 2, [batch_size, 50, 50], True)
filter = tf.get_variable('f3', [5, 1, 50], initializer=init)
out = conv1d_layer(output, filter, 2, [batch_size, 100, 100], True)
out = tf.reshape(out, [-1, 100 * 100])
vocab_size = 10000
w = tf.get_variable('w', [100*100, vocab_size], initializer=init)
b = tf.Variable(tf.zeros([vocab_size])) # biases
return tf.matmul(output, w) + b
When given some input, I do a forward pass by calling conv1d_layer. Then when I compute the gradient as follows, that's when the error happens.
output = forward(input)
loss = tf.nn.softmax_cross_entropy_with_logits_v2(logits=output, labels=classes)
optimizer = tf.train.AdamOptimizer(lr, beta1=0.5)
grads = optimizer.compute_gradients(loss)
I suspect my filters are wrong (I want to get a final shape of [100, 100, vocab_size] where vocab_size = 10000 as per above. Any help would be highly appreciated.
filter gradient transpose convolution
filter gradient transpose convolution
asked Nov 24 '18 at 17:49
NoorNoor
114
asked Nov 24 '18 at 17:49
NoorNoor
114
asked Nov 24 '18 at 17:49
NoorNoor
114
asked Nov 24 '18 at 17:49
NoorNoor
114
asked Nov 24 '18 at 17:49
NoorNoor
114
114
I meant input in the forward method instead of output, and return tf.matmul(out, w) + b
– Noor
Nov 24 '18 at 17:51
add a comment |
I meant input in the forward method instead of output, and return tf.matmul(out, w) + b
– Noor
Nov 24 '18 at 17:51
I meant input in the forward method instead of output, and return tf.matmul(out, w) + b
– Noor
Nov 24 '18 at 17:51
I meant input in the forward method instead of output, and return tf.matmul(out, w) + b
– Noor
Nov 24 '18 at 17:51
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53460868%2fincompatible-shapes-between-op-input-and-calculated-input-gradient-conv1d-trans%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53460868%2fincompatible-shapes-between-op-input-and-calculated-input-gradient-conv1d-trans%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
I meant input in the forward method instead of output, and return tf.matmul(out, w) + b
– Noor
Nov 24 '18 at 17:51