I want to multiply (Hadamard product) a matrix with a trainable tensor of the same size in TensorFlow. I.e. every non-zero element of the matrix is supposed to have a trainable multiplier. How do I do this?
The following also 'trains' 0-elements of the matrix.
weights = tf.get_variable('weights', [len_matrix,len_matrix], initializer=tf.contrib.layers.xavier_initializer())
result = weights * matrix
There is a module named tf.sparse which has optimized operations on sparse matrices, however, from what I found, it does not have a sparse variable implementation.
The only way that comes to my mind is to essentially flatten the sparse matrix without the zeros, then you can use a simple 1D variable. This assumes you know the sparse matrix in advance.
a = np.array([[1.4, 0.0, 0.0], [0.0, -4.5, 0.0]])
ix = np.argwhere(a == 0).T
flat_a = a[ix[0], ix[1]]
weights = tf.get_variable('weights', [len(flat_a)], initializer=tf.contrib.layers.xavier_initializer())
result = weights * flat_a
... # when you want to obtain the full result matrix back
flat_result = sess.run(result)
result = np.zeros(a.shape)
result[ix[0], ix[1]] = flat_result
Collected from the Internet
Please contact [email protected] to delete if infringement.
Comments