transformer

Classes

Transformer

Implementation of a Transformer model.

Module Contents

class transformer.Transformer(config)

Bases: torch.nn.Module

Implementation of a Transformer model.

Parameters:

  • BaseModel – Base model class that contains the utilities for all models we use in this tutorial.

  • config (emmi.schemas.models.TransformerConfig) – Configuration of the Transformer model.

blocks
forward(x, attn_kwargs)

Forward pass of the Transformer model.

Parameters:

  • x (torch.Tensor) – Input tensor of shape (batch_size, seq_len, hidden_dim).

  • attn_kwargs (dict[str, torch.Tensor]) – Additional arguments for the attention mechanism.

Returns:

Output tensor after processing through the Transformer model.

Return type:

torch.Tensor