emmi_inference.models.modules.mlp
=================================

.. py:module:: emmi_inference.models.modules.mlp


Classes
-------

.. autoapisummary::

   emmi_inference.models.modules.mlp.Mlp


Module Contents
---------------

.. py:class:: Mlp(dim)

   Bases: :py:obj:`torch.nn.Module`


   MLP as used in transformers nn.Linear(dim, dim * 4) -> GELU -> nn.Linear(dim * 4, dim).

   :param dim: Input dimension of the MLP.

   Initialize internal Module state, shared by both nn.Module and ScriptModule.


   .. py:attribute:: fc1


   .. py:attribute:: act


   .. py:attribute:: fc2


   .. py:method:: forward(x)