activation

eMultiheadAttention(in_rep, num_heads[, ...])	Drop-in replacement for torch.nn.MultiheadAttention that preserves G-equivariance.
AdditivePosMultiheadAttention(embed_dim, ...)	Wrap torch.nn.MultiheadAttention and add positional features to Q/K only.
AdditiveRelMultiheadAttention(embed_dim, ...)	Wrap torch.nn.MultiheadAttention and subtract a relative-position bias from the logits.
eAdditivePosMultiheadAttention(in_rep, ...)	Equivariant additive positional attention with invariant query/key updates.
eAdditiveRelMultiheadAttention(in_rep, ...)	Equivariant relative-bias attention with an equivariant attention backend.
PositionalAttentionBase(*args, **kwargs)	Abstract interface for attention blocks with explicit positional branches.
RoPEMultiheadAttention(embed_dim, num_heads)	Multi-head attention with rotary position embeddings applied to Q and K.
RotaryEmbedding(dim[, base, device, dtype])	Precompute the cosine and sine tables used by rotary embeddings.