Hide navigation sidebar

Hide table of contents sidebar

Skip to content

Toggle site navigation sidebar

Emmi AI Core documentation

Toggle table of contents sidebar

Tutorials

Prerequisites
Getting Started: Install & Verify
Training First Model (with Configs)
Training First Model (with Code)
Running First Inference

How-to Guides

Working with the CLI
How to Load a Custom Dataset
How to Use Private Data Source
How to Write Your Own Data Preprocessors
How to Write Your Own Data Collators
How to Configure All Training Options
How to Use Custom Model
How to Write Custom Loss or Metrics
How to Run Inference via the CLI
How to Run Inference via the Code

Explanation

Overview
Toggle navigation of Overview
Disk Caching

Reference

Emmi Data Management
Toggle navigation of Emmi Data Management
- emmi_data_management
  Toggle navigation of emmi_data_management
  - emmi_data_management.checkpoint
    Toggle navigation of emmi_data_management.checkpoint
    - emmi_data_management.checkpoint.iohash
    - emmi_data_management.checkpoint.providers
      Toggle navigation of emmi_data_management.checkpoint.providers
      - emmi_data_management.checkpoint.providers.file_provider
      - emmi_data_management.checkpoint.providers.huggingface_provider
    - emmi_data_management.checkpoint.resolver
    - emmi_data_management.checkpoint.types
  - emmi_data_management.cli
  - emmi_data_management.cli_aws
  - emmi_data_management.cli_checkpoint
  - emmi_data_management.cli_huggingface
  - emmi_data_management.cli_utils
  - emmi_data_management.cli_verification
  - emmi_data_management.credentials
  - emmi_data_management.diskcache
    Toggle navigation of emmi_data_management.diskcache
    - emmi_data_management.diskcache.cache_benchmark
    - emmi_data_management.diskcache.lru_cache
  - emmi_data_management.interfaces
    Toggle navigation of emmi_data_management.interfaces
  - emmi_data_management.logging_config
  - emmi_data_management.logging_progress
  - emmi_data_management.providers
  - emmi_data_management.verification
Emmi Inference Module
Toggle navigation of Emmi Inference Module
- emmi_inference
  Toggle navigation of emmi_inference
  - emmi_inference.api
  - emmi_inference.cli
  - emmi_inference.cli_utils
  - emmi_inference.collator_registry
  - emmi_inference.common
  - emmi_inference.dataset_registry
  - emmi_inference.datasets
    Toggle navigation of emmi_inference.datasets
    - emmi_inference.datasets.shapenet
  - emmi_inference.device
  - emmi_inference.examples
    Toggle navigation of emmi_inference.examples
  - emmi_inference.logging_config
  - emmi_inference.models
    Toggle navigation of emmi_inference.models
    - emmi_inference.models.abupt
    - emmi_inference.models.base_model
    - emmi_inference.models.loader
    - emmi_inference.models.modules
      Toggle navigation of emmi_inference.models.modules
      - emmi_inference.models.modules.attention
        Toggle navigation of emmi_inference.models.modules.attention
        
        emmi_inference.models.modules.attention.anchor_attention
        
        emmi_inference.models.modules.attention.dot_product_attention
        
        emmi_inference.models.modules.attention.perceiver_attention
        
        emmi_inference.models.modules.attention.sharedweights_crossattn_attention
        
        emmi_inference.models.modules.attention.sharedweights_splitattn_attention
      - emmi_inference.models.modules.blocks
        Toggle navigation of emmi_inference.models.modules.blocks
        
        emmi_inference.models.modules.blocks.perceiver_block
        
        emmi_inference.models.modules.blocks.transformer_block
      - emmi_inference.models.modules.continuous_sincos_embed
      - emmi_inference.models.modules.mlp
      - emmi_inference.models.modules.rope
      - emmi_inference.models.modules.rope_frequency
      - emmi_inference.models.modules.supernode_pooling_posonly
    - emmi_inference.models.pipelines
      Toggle navigation of emmi_inference.models.pipelines
    - emmi_inference.models.registry
    - emmi_inference.models.transolver
    - emmi_inference.models.upt
  - emmi_inference.preprocess
  - emmi_inference.runner
  - emmi_inference.s3_spooling_uploader
  - emmi_inference.writer
High level API (emmi)
Toggle navigation of High level API (emmi)
- emmi
  Toggle navigation of emmi
  - emmi.callbacks
    Toggle navigation of emmi.callbacks
    - emmi.callbacks.supernode_pooling_coverage
  - emmi.datasets
    Toggle navigation of emmi.datasets
    - emmi.datasets.ahmedml
    - emmi.datasets.drivaerml
  - emmi.functional
    Toggle navigation of emmi.functional
  - emmi.models
  - emmi.modules
    Toggle navigation of emmi.modules
    - emmi.modules.activations
    - emmi.modules.attention
      Toggle navigation of emmi.modules.attention
    - emmi.modules.blocks
      Toggle navigation of emmi.modules.blocks
    - emmi.modules.decoder
      Toggle navigation of emmi.modules.decoder
      - emmi.modules.decoder.deep_perceiver_decoder
    - emmi.modules.encoder
      Toggle navigation of emmi.modules.encoder
    - emmi.modules.layers
      Toggle navigation of emmi.modules.layers
    - emmi.modules.mlp
      Toggle navigation of emmi.modules.mlp
      - emmi.modules.mlp.mlp
      - emmi.modules.mlp.upactdown_mlp
    - emmi.modules.preprocessors
      Toggle navigation of emmi.modules.preprocessors
      - emmi.modules.preprocessors.normalizers
        Toggle navigation of emmi.modules.preprocessors.normalizers
        
        emmi.modules.preprocessors.normalizers.mean_std_normalization
        
        emmi.modules.preprocessors.normalizers.position_normalizer
        
        emmi.modules.preprocessors.normalizers.shift_and_scale_normalizer
        
        emmi.modules.preprocessors.normalizers.types
      - emmi.modules.preprocessors.numpy_to_torch_tensor
  - emmi.optim
    Toggle navigation of emmi.optim
    - emmi.optim.lion
  - emmi.pipeline
    Toggle navigation of emmi.pipeline
    - emmi.pipeline.batch_processors
      Toggle navigation of emmi.pipeline.batch_processors
    - emmi.pipeline.collators
      Toggle navigation of emmi.pipeline.collators
    - emmi.pipeline.sample_processors
      Toggle navigation of emmi.pipeline.sample_processors
  - emmi.preprocessing
    Toggle navigation of emmi.preprocessing
    - emmi.preprocessing.calculate_statistics_all
    - emmi.preprocessing.calculate_statistics_train
    - emmi.preprocessing.process_drivaer_and_ahmed_volumetric
    - emmi.preprocessing.shapenet_car
      Toggle navigation of emmi.preprocessing.shapenet_car
      - emmi.preprocessing.shapenet_car.preprocess_shapenet_car
  - emmi.schemas
    Toggle navigation of emmi.schemas
    - emmi.schemas.data
      Toggle navigation of emmi.schemas.data
      - emmi.schemas.data.data_specs
    - emmi.schemas.models
  - emmi.setup
  - emmi.types
  - emmi.utils
    Toggle navigation of emmi.utils
Low level API (ksuit)
Toggle navigation of Low level API (ksuit)
- ksuit
  Toggle navigation of ksuit
  - ksuit.callbacks
    Toggle navigation of ksuit.callbacks
    - ksuit.callbacks.base
      Toggle navigation of ksuit.callbacks.base
    - ksuit.callbacks.checkpoint_callbacks
      Toggle navigation of ksuit.callbacks.checkpoint_callbacks
    - ksuit.callbacks.default_callbacks
      Toggle navigation of ksuit.callbacks.default_callbacks
    - ksuit.callbacks.early_stoppers
      Toggle navigation of ksuit.callbacks.early_stoppers
    - ksuit.callbacks.offline_callbacks
      Toggle navigation of ksuit.callbacks.offline_callbacks
      - ksuit.callbacks.offline_callbacks.offline_loss
    - ksuit.callbacks.online_callbacks
      Toggle navigation of ksuit.callbacks.online_callbacks
      - ksuit.callbacks.online_callbacks.best_metric
      - ksuit.callbacks.online_callbacks.update_output
  - ksuit.cli
    Toggle navigation of ksuit.cli
    - ksuit.cli.main_calculate_stats
    - ksuit.cli.main_train
  - ksuit.configs
    Toggle navigation of ksuit.configs
  - ksuit.data
    Toggle navigation of ksuit.data
    - ksuit.data.base
      Toggle navigation of ksuit.data.base
    - ksuit.data.pipeline
      Toggle navigation of ksuit.data.pipeline
    - ksuit.data.preprocessors
      Toggle navigation of ksuit.data.preprocessors
    - ksuit.data.samplers
      Toggle navigation of ksuit.data.samplers
      - ksuit.data.samplers.interleaved_sampler
      - ksuit.data.samplers.internals
    - ksuit.data.wrappers
      Toggle navigation of ksuit.data.wrappers
      - ksuit.data.wrappers.dataset_wrappers
        Toggle navigation of ksuit.data.wrappers.dataset_wrappers
        
        ksuit.data.wrappers.dataset_wrappers.repeat_wrapper
        
        ksuit.data.wrappers.dataset_wrappers.shuffle_wrapper
        
        ksuit.data.wrappers.dataset_wrappers.subset_wrapper
      - ksuit.data.wrappers.property_subset_wrapper
  - ksuit.distributed
    Toggle navigation of ksuit.distributed
    - ksuit.distributed.config
      Toggle navigation of ksuit.distributed.config
      - ksuit.distributed.config.distributed_config
    - ksuit.distributed.functional
      Toggle navigation of ksuit.distributed.functional
      - ksuit.distributed.functional.all_gather_grad_autograd
    - ksuit.distributed.gather
    - ksuit.distributed.run
      Toggle navigation of ksuit.distributed.run
  - ksuit.extractors
    Toggle navigation of ksuit.extractors
    - ksuit.extractors.forward_hook
  - ksuit.factory
    Toggle navigation of ksuit.factory
    - ksuit.factory.base
      Toggle navigation of ksuit.factory.base
      - ksuit.factory.base.factory_base
      - ksuit.factory.base.reflection_utils
    - ksuit.factory.dataset_factory
    - ksuit.factory.optim_factory
    - ksuit.factory.schedule_factory
  - ksuit.initializers
    Toggle navigation of ksuit.initializers
    - ksuit.initializers.base
      Toggle navigation of ksuit.initializers.base
      - ksuit.initializers.base.checkpoint_initializer
      - ksuit.initializers.base.initializer_base
    - ksuit.initializers.previous_run_initializer
    - ksuit.initializers.resume_initializer
  - ksuit.models
    Toggle navigation of ksuit.models
  - ksuit.optimizer
    Toggle navigation of ksuit.optimizer
    - ksuit.optimizer.optimizer_wrapper
    - ksuit.optimizer.param_group_modifiers
      Toggle navigation of ksuit.optimizer.param_group_modifiers
      - ksuit.optimizer.param_group_modifiers.base
        Toggle navigation of ksuit.optimizer.param_group_modifiers.base
        
        ksuit.optimizer.param_group_modifiers.base.param_group_modifier_base
      - ksuit.optimizer.param_group_modifiers.lr_scale_by_name_modifier
      - ksuit.optimizer.param_group_modifiers.weight_decay_by_name_modifier
  - ksuit.providers
    Toggle navigation of ksuit.providers
  - ksuit.runners
    Toggle navigation of ksuit.runners
    - ksuit.runners.hydra_runner
    - ksuit.runners.stats_runner
  - ksuit.schedules
    Toggle navigation of ksuit.schedules
    - ksuit.schedules.base
      Toggle navigation of ksuit.schedules.base
    - ksuit.schedules.constant_schedule
    - ksuit.schedules.cosine
      Toggle navigation of ksuit.schedules.cosine
      - ksuit.schedules.cosine.decreasing
      - ksuit.schedules.cosine.increasing
    - ksuit.schedules.custom_schedule
    - ksuit.schedules.functional
    - ksuit.schedules.linear
      Toggle navigation of ksuit.schedules.linear
      - ksuit.schedules.linear.decreasing
      - ksuit.schedules.linear.increasing
    - ksuit.schedules.linear_warmup_cosine_decay
    - ksuit.schedules.polynomial
      Toggle navigation of ksuit.schedules.polynomial
      - ksuit.schedules.polynomial.decreasing
      - ksuit.schedules.polynomial.increasing
    - ksuit.schedules.step
      Toggle navigation of ksuit.schedules.step
  - ksuit.scripts
    Toggle navigation of ksuit.scripts
    - ksuit.scripts.validate_pytorch_compatibilities
  - ksuit.setup
  - ksuit.trackers
    Toggle navigation of ksuit.trackers
  - ksuit.trainers
    Toggle navigation of ksuit.trainers
    - ksuit.trainers.base
    - ksuit.trainers.types
  - ksuit.utils
    Toggle navigation of ksuit.utils
    - ksuit.utils.bidict
    - ksuit.utils.code
    - ksuit.utils.common
      Toggle navigation of ksuit.utils.common
    - ksuit.utils.data
      Toggle navigation of ksuit.utils.data
      - ksuit.utils.data.data_container
    - ksuit.utils.logging
      Toggle navigation of ksuit.utils.logging
    - ksuit.utils.model
      Toggle navigation of ksuit.utils.model
      - ksuit.utils.model.helpers
      - ksuit.utils.model.model_utils
    - ksuit.utils.platform
      Toggle navigation of ksuit.utils.platform
      - ksuit.utils.platform.system
      - ksuit.utils.platform.worker
    - ksuit.utils.torch
      Toggle navigation of ksuit.utils.torch
      - ksuit.utils.torch.amp
      - ksuit.utils.torch.device
    - ksuit.utils.training
      Toggle navigation of ksuit.utils.training
  - ksuit.writers
    Toggle navigation of ksuit.writers
    - ksuit.writers.checkpoint_writer
    - ksuit.writers.log_writer

Toggle table of contents sidebar

emmi.modules.layers¶

Submodules¶

emmi.modules.layers.continuous_sincos_embed
emmi.modules.layers.drop_path
emmi.modules.layers.layer_scale
emmi.modules.layers.linear_projection
emmi.modules.layers.rope_frequency
emmi.modules.layers.scalar_conditioner
emmi.modules.layers.transformer_batchnorm

emmi.modules.layers.continuous_sincos_embed

emmi.modules.encoder.supernode_pooling_posonly

Copyright © 2025, Emmi AI Developers

Made with Furo

On this page

emmi.modules.layers
- Submodules