emmi.modules.layers.rope_frequency¶

Classes¶

RopeFrequency

Creates frequencies for rotary embeddings (RoPE) from https://arxiv.org/abs/2104.09864 for variable positions.

Module Contents¶

class emmi.modules.layers.rope_frequency.RopeFrequency(config)¶

Bases: torch.nn.Module

Creates frequencies for rotary embeddings (RoPE) from https://arxiv.org/abs/2104.09864 for variable positions.

Parameters:: config (emmi.schemas.modules.layers.RopeFrequencyConfig) – Configuration for RoPE frequency settings.

hidden_dim¶

input_dim¶

implementation¶

ndim_padding¶

sincos_padding¶

max_wavelength¶

padding¶

assert_positive¶

forward(coords)¶

Parameters:: coords (torch.Tensor)
Return type:: torch.Tensor