pytext.models.representations package

Submodules

pytext.models.representations.bilstm module

class pytext.models.representations.bilstm.BiLSTM(config: pytext.models.representations.bilstm.BiLSTM.Config, embed_dim: int, padding_value: float = 0.0)

Bases: pytext.models.representations.representation_base.RepresentationBase

BiLSTM implements a multi-layer bidirectional LSTM representation layer preceded by a dropout layer.

Parameters:

• config (Config) – Configuration object of type BiLSTM.Config.
• embed_dim (int) – The number of expected features in the input.
• padding_value (float) – Value for the padded elements. Defaults to 0.0.

padding_value

float – Value for the padded elements.

dropout

nn.Dropout – Dropout layer preceding the LSTM.

lstm

nn.LSTM – LSTM layer that operates on the inputs.

representation_dim

int – The calculated dimension of the output features of BiLSTM.

Config

alias of BiLSTM.Config

forward(embedded_tokens: torch.Tensor, seq_lengths: torch.Tensor, states: Optional[Tuple[torch.Tensor, torch.Tensor]] = None) → Tuple[torch.Tensor, Tuple[torch.Tensor, torch.Tensor]]

Given an input batch of sequential data such as word embeddings, produces a bidirectional LSTM representation of the sequential input and new state tensors.

Parameters:

• embedded_tokens (torch.Tensor) – Input tensor of shape (bsize x seq_len x input_dim).
• seq_lengths (torch.Tensor) – List of sequences lengths of each batch element.
• states (Tuple[torch.Tensor, torch.Tensor]) – Tuple of tensors containing the initial hidden state and the cell state of each element in the batch. Each of these tensors have a dimension of (bsize x num_layers * num_directions x nhid). Defaults to None.

Returns:

Bidirectional

LSTM representation of input and the state of the LSTM t = seq_len. Shape of representation is (bsize x seq_len x representation_dim). Shape of each state is (bsize x num_layers * num_directions x nhid).

Return type:

Tuple[torch.Tensor, Tuple[torch.Tensor, torch.Tensor]]

pytext.models.representations.bilstm_doc_attention module

class pytext.models.representations.bilstm_doc_attention.BiLSTMDocAttention(config: pytext.models.representations.bilstm_doc_attention.BiLSTMDocAttention.Config, embed_dim: int)

Bases: pytext.models.representations.representation_base.RepresentationBase

BiLSTMDocAttention implements a multi-layer bidirectional LSTM based representation for documents with or without pooling. The pooling can be max pooling, mean pooling or self attention.

Parameters:

• config (Config) – Configuration object of type BiLSTMDocAttention.Config.
• embed_dim (int) – The number of expected features in the input.

dropout

nn.Dropout – Dropout layer preceding the LSTM.

lstm

nn.Module – Module that implements the LSTM.

attention

nn.Module – Module that implements the attention or pooling.

dense

nn.Module – Module that implements the non-linear projection over attended representation.

representation_dim

int – The calculated dimension of the output features of the BiLSTMDocAttention representation.

Config

alias of BiLSTMDocAttention.Config

forward(embedded_tokens: torch.Tensor, seq_lengths: torch.Tensor, *args, states: Tuple[torch.Tensor, torch.Tensor] = None) → Tuple[torch.Tensor, Tuple[torch.Tensor, torch.Tensor]]

Given an input batch of sequential data such as word embeddings, produces a bidirectional LSTM representation with or without pooling of the sequential input and new state tensors.

Parameters:

• embedded_tokens (torch.Tensor) – Input tensor of shape (bsize x seq_len x input_dim).
• seq_lengths (torch.Tensor) – List of sequences lengths of each batch element.
• states (Tuple[torch.Tensor, torch.Tensor]) – Tuple of tensors containing the initial hidden state and the cell state of each element in the batch. Each of these tensors have a dimension of (bsize x num_layers * num_directions x nhid). Defaults to None.

Returns:

Bidirectional

LSTM representation of input and the state of the LSTM at t = seq_len.

Return type:

Tuple[torch.Tensor, Tuple[torch.Tensor, torch.Tensor]]

pytext.models.representations.bilstm_doc_slot_attention module

class pytext.models.representations.bilstm_doc_slot_attention.BiLSTMDocSlotAttention(config: pytext.models.representations.bilstm_doc_slot_attention.BiLSTMDocSlotAttention.Config, embed_dim: int)

Bases: pytext.models.representations.representation_base.RepresentationBase

BiLSTMDocSlotAttention implements a multi-layer bidirectional LSTM based representation with support for various attention mechanisms.

In default mode, when attention configuration is not provided, it behaves like a multi-layer LSTM encoder and returns the output features from the last layer of the LSTM, for each t. When document_attention configuration is provided, it produces a fixed-sized document representation. When slot_attention configuration is provide, it attends on output of each cell of LSTM module to produce a fixed sized word representation.

Parameters:

• config (Config) – Configuration object of type BiLSTMDocSlotAttention.Config.
• embed_dim (int) – The number of expected features in the input.

dropout

nn.Dropout – Dropout layer preceding the LSTM.

relu

nn.ReLU – An instance of the ReLU layer.

lstm

nn.Module – Module that implements the LSTM.

use_doc_attention

bool – If True, indicates using document attention.

doc_attention

nn.Module – Module that implements document attention.

self.projection_d

nn.Sequential – A sequence of dense layers for projection over document representation.

use_word_attention

bool – If True, indicates using word attention.

word_attention

nn.Module – Module that implements word attention.

self.projection_w

nn.Sequential – A sequence of dense layers for projection over word representation.

representation_dim

int – The calculated dimension of the output features of the BiLSTMDocAttention representation.

Config

alias of BiLSTMDocSlotAttention.Config

forward(embedded_tokens: torch.Tensor, seq_lengths: torch.Tensor, *args, states: torch.Tensor = None) → Tuple[torch.Tensor, torch.Tensor, Tuple[torch.Tensor, torch.Tensor]]

Given an input batch of sequential data such as word embeddings, produces a bidirectional LSTM representation the appropriate attention.

Parameters:

• embedded_tokens (torch.Tensor) – Input tensor of shape (bsize x seq_len x input_dim).
• seq_lengths (torch.Tensor) – List of sequences lengths of each batch element.
• states (Tuple[torch.Tensor, torch.Tensor]) – Tuple of tensors containing the initial hidden state and the cell state of each element in the batch. Each of these tensors have a dimension of (bsize x num_layers * num_directions x nhid). Defaults to None.

Returns:

Tensors containing the document and the word representation of the input.

Return type:

Tuple[torch.Tensor, torch.Tensor, Tuple[torch.Tensor, torch.Tensor]]

pytext.models.representations.bilstm_slot_attn module

class pytext.models.representations.bilstm_slot_attn.BiLSTMSlotAttention(config: pytext.models.representations.bilstm_slot_attn.BiLSTMSlotAttention.Config, embed_dim: int)

Bases: pytext.models.representations.representation_base.RepresentationBase

BiLSTMSlotAttention implements a multi-layer bidirectional LSTM based representation with attention over slots.

Parameters:

• config (Config) – Configuration object of type BiLSTMSlotAttention.Config.
• embed_dim (int) – The number of expected features in the input.

dropout

nn.Dropout – Dropout layer preceding the LSTM.

lstm

nn.Module – Module that implements the LSTM.

attention

nn.Module – Module that implements the attention.

dense

nn.Module – Module that implements the non-linear projection over attended representation.

representation_dim

int – The calculated dimension of the output features of the SlotAttention representation.

Config

alias of BiLSTMSlotAttention.Config

forward(embedded_tokens: torch.Tensor, seq_lengths: torch.Tensor, *args, states: torch.Tensor = None, **kwargs) → torch.Tensor

Given an input batch of sequential data such as word embeddings, produces a bidirectional LSTM representation with or without Slot attention.

Parameters:

• embedded_tokens (torch.Tensor) – Input tensor of shape (bsize x seq_len x input_dim).
• seq_lengths (torch.Tensor) – List of sequences lengths of each batch element.
• states (Tuple[torch.Tensor, torch.Tensor]) – Tuple of tensors containing the initial hidden state and the cell state of each element in the batch. Each of these tensors have a dimension of (bsize x num_layers * num_directions x nhid). Defaults to None.

Returns:

Bidirectional LSTM representation of input with or

without slot attention.

Return type:

torch.Tensor

pytext.models.representations.biseqcnn module

class pytext.models.representations.biseqcnn.BSeqCNNRepresentation(config: pytext.models.representations.biseqcnn.BSeqCNNRepresentation.Config, embed_dim: int)

Bases: pytext.models.representations.representation_base.RepresentationBase

This class is an implementation of the paper https://arxiv.org/pdf/1606.07783. It is a bidirectional CNN model that captures context like RNNs do.

The module expects that input mini-batch is already padded.

TODO: Current implementation has a single layer conv-maxpool operation.

Config

alias of BSeqCNNRepresentation.Config

forward(inputs: torch.Tensor, *args) → torch.Tensor

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

pytext.models.representations.contextual_intent_slot_rep module

class pytext.models.representations.contextual_intent_slot_rep.ContextualIntentSlotRepresentation(config: pytext.models.representations.contextual_intent_slot_rep.ContextualIntentSlotRepresentation.Config, embed_dim: Tuple[int, ...])

Bases: pytext.models.representations.representation_base.RepresentationBase

Representation for a contextual intent slot model

The inputs are two embeddings: word level embedding containing dictionary features, sequence (contexts) level embedding. See following diagram for the representation implementation that combines the two embeddings. Seq_representation is concatenated with word_embeddings.

+-----------+
| word_embed|--------------------------->+   +--------------------+
+-----------+                            |   | doc_representation |
+-----------+   +-------------------+    |-->+--------------------+
| seq_embed |-->| seq_representation|--->+   | word_representation|
+-----------+   +-------------------+        +--------------------+
                                              joint_representation

Config

alias of ContextualIntentSlotRepresentation.Config

forward(word_seq_embed: Tuple[torch.Tensor, torch.Tensor], word_lengths: torch.Tensor, seq_lengths: torch.Tensor, *args) → List[torch.Tensor]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

pytext.models.representations.docnn module

class pytext.models.representations.docnn.DocNNRepresentation(config: pytext.models.representations.docnn.DocNNRepresentation.Config, embed_dim: int)

Bases: pytext.models.representations.representation_base.RepresentationBase

CNN based representation of a document.

Config

alias of DocNNRepresentation.Config

conv_and_pool(x, conv)

forward(embedded_tokens: torch.Tensor, *args) → torch.Tensor

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

pytext.models.representations.jointcnn_rep module

class pytext.models.representations.jointcnn_rep.JointCNNRepresentation(config: pytext.models.representations.jointcnn_rep.JointCNNRepresentation.Config, embed_dim: int)

Bases: pytext.models.representations.representation_base.RepresentationBase

Config

alias of JointCNNRepresentation.Config

forward(embedded_tokens: torch.Tensor, *args) → List[torch.Tensor]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

pytext.models.representations.pair_rep module

class pytext.models.representations.pair_rep.PairRepresentation(config: pytext.models.representations.pair_rep.PairRepresentation.Config, embed_dim: Tuple[int, ...])

Bases: pytext.models.representations.representation_base.RepresentationBase

Wrapper representation for a pair of inputs.

Takes a tuple of inputs: the left sentence, and the right sentence(s). Returns a representation of the pair of sentences, either as a concatenation of the two sentence embeddings or as a “siamese” representation which also includes their difference and elementwise product (arXiv:1705.02364). If more than two inputs are provided, the extra inputs are assumed to be extra “right” sentences, and the output will be the stacked pair representations of the left sentence together with all right sentences. This is more efficient than separately computing all these pair representations, because the left sentence will not need to be re-embedded multiple times.

Config

alias of PairRepresentation.Config

forward(embeddings: Tuple[torch.Tensor, ...], *lengths) → torch.Tensor

Computes the pair representations.

Parameters:

• embeddings – token embeddings of the left sentence, followed by the token embeddings of the right sentence(s).
• lengths – the corresponding sequence lengths.

Returns:

A tensor of shape (num_right_inputs, batch_size, rep_size), with the first dimension squeezed if one.

pytext.models.representations.pooling module

class pytext.models.representations.pooling.BoundaryPool(config: pytext.models.representations.pooling.BoundaryPool.Config, n_input: int)

Bases: pytext.models.module.Module

Config

alias of BoundaryPool.Config

forward(inputs: torch.Tensor, seq_lengths: torch.Tensor = None) → torch.Tensor

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class pytext.models.representations.pooling.MaxPool(config: pytext.config.module_config.ModuleConfig, n_input: int)

Bases: pytext.models.module.Module

Config

alias of pytext.config.component.ComponentMeta.__new__.<locals>.Config

forward(inputs: torch.Tensor, seq_lengths: torch.Tensor = None) → torch.Tensor

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class pytext.models.representations.pooling.MeanPool(config: pytext.config.module_config.ModuleConfig, n_input: int)

Bases: pytext.models.module.Module

Config

alias of pytext.config.component.ComponentMeta.__new__.<locals>.Config

forward(inputs: torch.Tensor, seq_lengths: torch.Tensor) → torch.Tensor

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class pytext.models.representations.pooling.NoPool(config: pytext.config.module_config.ModuleConfig, n_input: int)

Bases: pytext.models.module.Module

Config

alias of pytext.config.component.ComponentMeta.__new__.<locals>.Config

forward(inputs: torch.Tensor, seq_lengths: torch.Tensor = None) → torch.Tensor

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class pytext.models.representations.pooling.SelfAttention(config: pytext.models.representations.pooling.SelfAttention.Config, n_input: int)

Bases: pytext.models.module.Module

Config

alias of SelfAttention.Config

forward(inputs: torch.Tensor, seq_lengths: torch.Tensor = None) → torch.Tensor

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

init_weights(init_range: float = 0.1) → None

pytext.models.representations.pure_doc_attention module

class pytext.models.representations.pure_doc_attention.PureDocAttention(config: pytext.models.representations.pure_doc_attention.PureDocAttention.Config, embed_dim: int)

Bases: pytext.models.representations.representation_base.RepresentationBase

pooling (e.g. max pooling or self attention) followed by optional MLP

Config

alias of PureDocAttention.Config

forward(embedded_tokens: torch.Tensor, seq_lengths: torch.Tensor, *args) → Any

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

pytext.models.representations.representation_base module

class pytext.models.representations.representation_base.RepresentationBase(config)

Bases: pytext.models.module.Module

Config

alias of pytext.config.component.ComponentMeta.__new__.<locals>.Config

forward(*inputs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

get_representation_dim()

pytext.models.representations.seq_rep module

class pytext.models.representations.seq_rep.SeqRepresentation(config: pytext.models.representations.seq_rep.SeqRepresentation.Config, embed_dim: int)

Bases: pytext.models.representations.representation_base.RepresentationBase

Representation for a sequence of sentences Each sentence will be embedded with a DocNN model, then all the sentences are embedded with another DocNN/BiLSTM model

Config

alias of SeqRepresentation.Config

forward(embedded_seqs: torch.Tensor, seq_lengths: torch.Tensor, *args) → torch.Tensor

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

pytext.models.representations.slot_attention module

class pytext.models.representations.slot_attention.SlotAttention(config: pytext.models.representations.slot_attention.SlotAttention.Config, n_input: int, batch_first: bool = True)

Bases: pytext.models.module.Module

Config

alias of SlotAttention.Config

forward(inputs: torch.Tensor) → torch.Tensor

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Module contents