I want to create a deep neural network in keras, where each element of the input layer is "encoded" using the same, shared Embedding()-layer, before it is fed into the deeper layers.
Each input would be a number that defines the type of an object, and the network should learn an embedding that encapsulates some internal representation of "what this object is".
So, if the input layer has X dimensions, and the embedding has Y dimensions, the first hidden layer should consist of X*Y neurons (each input neuron embedded).
How can I do this?
from keras.layers import Input, Embedding
first_input = Input(shape = (your_shape_tuple) )
second_input = Input(shape = (your_shape_tuple) )
...
embedding_layer = Embedding(embedding_size)
first_input_encoded = embedding_layer(first_input)
second_input_encoded = embedding_layer(second_input)
...
Rest of the model....
The emnedding_layer will have shared weights. You can do this in form of lists of layers if you have a lot of inputs.
If what you want is transforming a tensor of inputs, the way to do it is :
from keras.layers import Input, Embedding
# If your inputs are all fed in one numpy array :
input_layer = Input(shape = (num_input_indices,) )
# the output of this layer will be a 2D tensor of shape (num_input_indices, embedding_size)
embedded_input = Embedding(embedding_size)(input_layer)
Is this what you were looking for?
Collected from the Internet
Please contact [email protected] to delete if infringement.
Comments