layers.rope_frequenty_config

Classes

RopeFrequencyConfig

Configuration for RoPE frequency settings.

Module Contents

class layers.rope_frequenty_config.RopeFrequencyConfig(/, **data)

Bases: pydantic.BaseModel

Configuration for RoPE frequency settings.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:

data (Any)

hidden_dim: int = None

Dimensionality of frequencies (in transformers this should be the head dimension).

input_dim: int = None

Dimensionality of the coordinates (e.g., 2 for 2D coordinates, 3 for 3D coordinates).

max_wavelength: int = None

10000.0

Type:

Theta parameter for the transformer sine/cosine embedding. Default

assert_positive: bool = None

True

Type:

Makes sure that coordinates were rescaled to be positive only. Default

implementation: Literal['real', 'complex'] = None

“real” -> basic implementation using real coordinates (this is slow and only here for backward compatibility). “complex” -> fast implementation of rotation via complex multiplication. Default: “real”.