TransformerEncoder#

class TransformerEncoder(encoder_layer, num_layers, norm=None, enable_nested_tensor=True, mask_check=True)[source]#

Bases: Module

Stack encoder layers and apply an optional final normalization.

Attributes:#

layers:

Sequential copies of the encoder layer.

norm:

Optional normalization applied after the final layer.

num_layers:

Number of stacked encoder layers.

Initialize the encoder stack.

type encoder_layer:

Module

param encoder_layer:

Base layer to replicate.

type encoder_layer:

torch.nn.Module

type num_layers:

int

param num_layers:

Number of stacked encoder layers.

type num_layers:

int

type norm:

Module | None

param norm:

Final normalization layer.

type norm:

torch.nn.Module, optional

type enable_nested_tensor:

bool

param enable_nested_tensor:

Preserved for API compatibility. Default: True.

type enable_nested_tensor:

bool

type mask_check:

bool

param mask_check:

Preserved for API compatibility. Default: True.

type mask_check:

bool

forward(src, mask=None, src_key_padding_mask=None, is_causal=None, **layer_kwargs)[source]#

Apply the encoder stack to a batch-first source sequence.

Return type:

Tensor

Parameters:

Shape#

  • src: (B, T, D).

  • Returns: encoded source with shape (B, T, D).

Parameters: