TensorFlow provides tensor operations, while Keras provides the high-level layer and model interface used in most TensorFlow neural-network code. Keras conventions differ from some PyTorch conventions, especially for recurrent inputs, layer construction, and loss calls.
Tensor Shape Rules
Keras normally places the batch dimension first at runtime:
| Data | Common shape |
|---|---|
| Feature rows | (batch, features) |
| Images | (batch, height, width, channels) |
| Token IDs | (batch, time) |
| Recurrent features | (batch, time, features) |
Keras recurrent inputs are batch first. They do not place the batch dimension last.
keras.Input Omits the Batch Dimension
When a model input is declared, shape describes one sample:
inputs = keras.Input(shape=(5, 8))This declaration means:
one sample: (5 time steps, 8 features)
runtime batch: (batch, 5 time steps, 8 features)For a fixed batch size, use batch_shape:
inputs = keras.Input(batch_shape=(32, 5, 8))Layer Arguments and Shape Inference
Keras layers often ask for the output size and infer the input size when the layer first receives data:
dense = keras.layers.Dense(units=4)
gru = keras.layers.GRU(units=6)
embedding = keras.layers.Embedding(input_dim=1000, output_dim=64)Read these as:
| Layer | Input rule | Output rule |
|---|---|---|
Dense(4) | Input size is inferred from the final input dimension | Final output dimension becomes 4 |
GRU(6) | Input-feature size is inferred from the sequence tensor | Hidden-state dimension becomes 6 |
Embedding(1000, 64) | Integer IDs range from 0 to 999 | Each ID becomes 64 values |
Dense changes only the final dimension:
(3, 5, 8) -> Dense(4) -> (3, 5, 4)Keras stores a Dense kernel with shape (input features, output units).
Data Types
Common rules:
- neural-network activations and weights normally use floating-point tensors
- token IDs and sparse class targets normally use integer tensors
- the input data type must match the type expected by the layer and loss
token_ids = tf.constant([[1, 2, 3]], dtype=tf.int32)
features = tf.constant([[0.2, 0.5]], dtype=tf.float32)Loss Call Order
Keras loss objects receive targets first and predictions second:
loss_fn = keras.losses.SparseCategoricalCrossentropy(from_logits=True)
predictions = model(x)
loss = loss_fn(targets, predictions)This order is (y_true, y_pred). It is the reverse of the common PyTorch loss call order.
Managed and Manual Training
model.compile() and model.fit() manage the forward pass, loss calculation, gradient calculation, and parameter update:
model.compile(
optimizer=keras.optimizers.Adam(),
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
)
model.fit(x, targets)Use tf.GradientTape when a custom training step is required:
with tf.GradientTape() as tape:
predictions = model(x, training=True)
loss = loss_fn(targets, predictions)
gradients = tape.gradient(loss, model.trainable_weights)
optimizer.apply_gradients(zip(gradients, model.trainable_weights))training=True selects training behaviour for layers such as Dropout. training=False selects inference behaviour.
The official interfaces are listed in the Keras API.