Keras for basic neural networks

Keras is one of the high level APIs in Tensorflow deep learning stack. It is the recommended framework to get started with neural networks, if you do not have special requirements.

Keras Sequential

The Sequential class can be used for simple stuff. A sequential model is appropriate for a plain stack of layers where each layer has exactly one input tensor and one output tensor.

Keras functional API

The Keras Functional API is more flexible. This means, you can for example “branch” multiple output layers. You add layers to tf.keras.layers one by one.

A simple network would not be significantly more difficult with the Keras functional API compared to Sequential. Only more lines of code.

Subclassing tf.keras.Model makes it possible to customize everything.

Keras pre-processing

Pre-processing can be integrated as part of the model training, which usually the best option. It has the following pre-processing layers:

Cool thing is, pre-processing steps can be visualized by tf.keras.utils.plot_model().

Non-trainable features in Keras pre-processing

Non-trainable features or weights are not updated during backpropagation. This is the case with constants you have defined.

Some of the Keras pre-processing layers are non-trainable. They require the subset of the training data instead of just one observation. This means, the state must be set before training. So, a constant value is pre-computed before training.

Full pass vs instance level tansformations .

Another option to pre-compute non-trainable features is to use adapt function of Keras. It should not be applied to large vocabularies (+500 MB). It can be applied to these methods:

• TextVetorization
• StringLookup
• Normalization
• Discretization

Custom layers

The keras.layers.layer can be subclassed to create a custom layer. It needs the number of neurons as a parameter.

Typically random weights are initialized. The output is the return value of the call function. Non-trainable weights can be set to be considered in back-propagation.

Deferring weight creation until the shape of the inputs is known is a best practice. The build function can be used for this.

Use optionally self.add_loss() and self.add_metric().

Custom models

Subclass keras.Model to create custom models.

The call function needs to return the last layer of the model.

Keras code example

import numpy as np
from tensorflow.keras import Sequential, layers, utils

labels = [1, 2]
np_features = np.array([
    [0.435 , 0.335],
    [0.585 , 0.45 ]
])

normalize = layers.Normalization()
normalize.adapt(np_features)

model = Sequential([
    normalize,
    layers.Dense(5),
    layers.Dense(1)
])

model.compile()
history = model.fit(np_features, labels, epochs=3)
model.summary()
utils.plot_model(model)
model.predict(np_features)

Input layer is not explicitly defined, it is inferred from data.

Images in Keras

Keras has these tools for data augmentation

• RandomTranslation
• RandomRotation
• RandomZoom
• RandomCrop
• RandomFlip

Keras has two modes to pad data for CNN. The valid options pads the right and bottom pixels. When choosing same both sides are padded evenly.