emmi_inference.models.modules.mlp

Classes

Mlp	MLP as used in transformers nn.Linear(dim, dim * 4) -> GELU -> nn.Linear(dim * 4, dim).

Module Contents

class emmi_inference.models.modules.mlp.Mlp(dim)

Bases: torch.nn.Module

MLP as used in transformers nn.Linear(dim, dim * 4) -> GELU -> nn.Linear(dim * 4, dim).

Parameters:

dim (int) – Input dimension of the MLP.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

fc1

act

fc2

forward(x)

Parameters:

x (torch.Tensor)

Return type:

torch.Tensor