pytext.models.representations package¶
Submodules¶
pytext.models.representations.bilstm module¶
-
class
pytext.models.representations.bilstm.
BiLSTM
(config: pytext.models.representations.bilstm.BiLSTM.Config, embed_dim: int, padding_value: float = 0.0)[source]¶ Bases:
pytext.models.representations.representation_base.RepresentationBase
BiLSTM implements a multi-layer bidirectional LSTM representation layer preceded by a dropout layer.
Parameters: - config (Config) – Configuration object of type BiLSTM.Config.
- embed_dim (int) – The number of expected features in the input.
- padding_value (float) – Value for the padded elements. Defaults to 0.0.
-
padding_value
¶ float – Value for the padded elements.
-
dropout
¶ nn.Dropout – Dropout layer preceding the LSTM.
-
lstm
¶ nn.LSTM – LSTM layer that operates on the inputs.
-
representation_dim
¶ int – The calculated dimension of the output features of BiLSTM.
-
Config
[source]¶ alias of
BiLSTM.Config
-
forward
(embedded_tokens: torch.Tensor, seq_lengths: torch.Tensor, states: Optional[Tuple[torch.Tensor, torch.Tensor]] = None) → Tuple[torch.Tensor, Tuple[torch.Tensor, torch.Tensor]][source]¶ Given an input batch of sequential data such as word embeddings, produces a bidirectional LSTM representation of the sequential input and new state tensors.
Parameters: - embedded_tokens (torch.Tensor) – Input tensor of shape (bsize x seq_len x input_dim).
- seq_lengths (torch.Tensor) – List of sequences lengths of each batch element.
- states (Tuple[torch.Tensor, torch.Tensor]) – Tuple of tensors containing the initial hidden state and the cell state of each element in the batch. Each of these tensors have a dimension of (bsize x num_layers * num_directions x nhid). Defaults to None.
Returns: - Bidirectional
LSTM representation of input and the state of the LSTM t = seq_len. Shape of representation is (bsize x seq_len x representation_dim). Shape of each state is (bsize x num_layers * num_directions x nhid).
Return type: Tuple[torch.Tensor, Tuple[torch.Tensor, torch.Tensor]]
pytext.models.representations.bilstm_doc_attention module¶
-
class
pytext.models.representations.bilstm_doc_attention.
BiLSTMDocAttention
(config: pytext.models.representations.bilstm_doc_attention.BiLSTMDocAttention.Config, embed_dim: int)[source]¶ Bases:
pytext.models.representations.representation_base.RepresentationBase
BiLSTMDocAttention implements a multi-layer bidirectional LSTM based representation for documents with or without pooling. The pooling can be max pooling, mean pooling or self attention.
Parameters: - config (Config) – Configuration object of type BiLSTMDocAttention.Config.
- embed_dim (int) – The number of expected features in the input.
-
dropout
¶ nn.Dropout – Dropout layer preceding the LSTM.
-
lstm
¶ nn.Module – Module that implements the LSTM.
-
attention
¶ nn.Module – Module that implements the attention or pooling.
-
dense
¶ nn.Module – Module that implements the non-linear projection over attended representation.
-
representation_dim
¶ int – The calculated dimension of the output features of the BiLSTMDocAttention representation.
-
Config
[source]¶ alias of
BiLSTMDocAttention.Config
-
forward
(embedded_tokens: torch.Tensor, seq_lengths: torch.Tensor, *args, states: Tuple[torch.Tensor, torch.Tensor] = None) → Tuple[torch.Tensor, Tuple[torch.Tensor, torch.Tensor]][source]¶ Given an input batch of sequential data such as word embeddings, produces a bidirectional LSTM representation with or without pooling of the sequential input and new state tensors.
Parameters: - embedded_tokens (torch.Tensor) – Input tensor of shape (bsize x seq_len x input_dim).
- seq_lengths (torch.Tensor) – List of sequences lengths of each batch element.
- states (Tuple[torch.Tensor, torch.Tensor]) – Tuple of tensors containing the initial hidden state and the cell state of each element in the batch. Each of these tensors have a dimension of (bsize x num_layers * num_directions x nhid). Defaults to None.
Returns: - Bidirectional
LSTM representation of input and the state of the LSTM at t = seq_len.
Return type: Tuple[torch.Tensor, Tuple[torch.Tensor, torch.Tensor]]
pytext.models.representations.bilstm_doc_slot_attention module¶
-
class
pytext.models.representations.bilstm_doc_slot_attention.
BiLSTMDocSlotAttention
(config: pytext.models.representations.bilstm_doc_slot_attention.BiLSTMDocSlotAttention.Config, embed_dim: int)[source]¶ Bases:
pytext.models.representations.representation_base.RepresentationBase
BiLSTMDocSlotAttention implements a multi-layer bidirectional LSTM based representation with support for various attention mechanisms.
In default mode, when attention configuration is not provided, it behaves like a multi-layer LSTM encoder and returns the output features from the last layer of the LSTM, for each t. When document_attention configuration is provided, it produces a fixed-sized document representation. When slot_attention configuration is provide, it attends on output of each cell of LSTM module to produce a fixed sized word representation.
Parameters: - config (Config) – Configuration object of type BiLSTMDocSlotAttention.Config.
- embed_dim (int) – The number of expected features in the input.
-
dropout
¶ nn.Dropout – Dropout layer preceding the LSTM.
-
relu
¶ nn.ReLU – An instance of the ReLU layer.
-
lstm
¶ nn.Module – Module that implements the LSTM.
-
use_doc_attention
¶ bool – If True, indicates using document attention.
-
doc_attention
¶ nn.Module – Module that implements document attention.
-
self.
projection_d
¶ nn.Sequential – A sequence of dense layers for projection over document representation.
-
use_word_attention
¶ bool – If True, indicates using word attention.
-
word_attention
¶ nn.Module – Module that implements word attention.
-
self.
projection_w
¶ nn.Sequential – A sequence of dense layers for projection over word representation.
-
representation_dim
¶ int – The calculated dimension of the output features of the BiLSTMDocAttention representation.
-
Config
[source]¶ alias of
BiLSTMDocSlotAttention.Config
-
forward
(embedded_tokens: torch.Tensor, seq_lengths: torch.Tensor, *args, states: torch.Tensor = None) → Tuple[torch.Tensor, torch.Tensor, Tuple[torch.Tensor, torch.Tensor]][source]¶ Given an input batch of sequential data such as word embeddings, produces a bidirectional LSTM representation the appropriate attention.
Parameters: - embedded_tokens (torch.Tensor) – Input tensor of shape (bsize x seq_len x input_dim).
- seq_lengths (torch.Tensor) – List of sequences lengths of each batch element.
- states (Tuple[torch.Tensor, torch.Tensor]) – Tuple of tensors containing the initial hidden state and the cell state of each element in the batch. Each of these tensors have a dimension of (bsize x num_layers * num_directions x nhid). Defaults to None.
Returns: Tensors containing the document and the word representation of the input.
Return type: Tuple[torch.Tensor, torch.Tensor, Tuple[torch.Tensor, torch.Tensor]]
pytext.models.representations.bilstm_slot_attn module¶
-
class
pytext.models.representations.bilstm_slot_attn.
BiLSTMSlotAttention
(config: pytext.models.representations.bilstm_slot_attn.BiLSTMSlotAttention.Config, embed_dim: int)[source]¶ Bases:
pytext.models.representations.representation_base.RepresentationBase
BiLSTMSlotAttention implements a multi-layer bidirectional LSTM based representation with attention over slots.
Parameters: - config (Config) – Configuration object of type BiLSTMSlotAttention.Config.
- embed_dim (int) – The number of expected features in the input.
-
dropout
¶ nn.Dropout – Dropout layer preceding the LSTM.
-
lstm
¶ nn.Module – Module that implements the LSTM.
-
attention
¶ nn.Module – Module that implements the attention.
-
dense
¶ nn.Module – Module that implements the non-linear projection over attended representation.
-
representation_dim
¶ int – The calculated dimension of the output features of the SlotAttention representation.
-
Config
[source]¶ alias of
BiLSTMSlotAttention.Config
-
forward
(embedded_tokens: torch.Tensor, seq_lengths: torch.Tensor, *args, states: torch.Tensor = None, **kwargs) → torch.Tensor[source]¶ Given an input batch of sequential data such as word embeddings, produces a bidirectional LSTM representation with or without Slot attention.
Parameters: - embedded_tokens (torch.Tensor) – Input tensor of shape (bsize x seq_len x input_dim).
- seq_lengths (torch.Tensor) – List of sequences lengths of each batch element.
- states (Tuple[torch.Tensor, torch.Tensor]) – Tuple of tensors containing the initial hidden state and the cell state of each element in the batch. Each of these tensors have a dimension of (bsize x num_layers * num_directions x nhid). Defaults to None.
Returns: - Bidirectional LSTM representation of input with or
without slot attention.
Return type: torch.Tensor
pytext.models.representations.biseqcnn module¶
-
class
pytext.models.representations.biseqcnn.
BSeqCNNRepresentation
(config: pytext.models.representations.biseqcnn.BSeqCNNRepresentation.Config, embed_dim: int)[source]¶ Bases:
pytext.models.representations.representation_base.RepresentationBase
This class is an implementation of the paper https://arxiv.org/pdf/1606.07783. It is a bidirectional CNN model that captures context like RNNs do.
The module expects that input mini-batch is already padded.
TODO: Current implementation has a single layer conv-maxpool operation.
-
Config
[source]¶ alias of
BSeqCNNRepresentation.Config
-
forward
(inputs: torch.Tensor, *args) → torch.Tensor[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
pytext.models.representations.contextual_intent_slot_rep module¶
-
class
pytext.models.representations.contextual_intent_slot_rep.
ContextualIntentSlotRepresentation
(config: pytext.models.representations.contextual_intent_slot_rep.ContextualIntentSlotRepresentation.Config, embed_dim: Tuple[int, ...])[source]¶ Bases:
pytext.models.representations.representation_base.RepresentationBase
Representation for a contextual intent slot model
The inputs are two embeddings: word level embedding containing dictionary features, sequence (contexts) level embedding. See following diagram for the representation implementation that combines the two embeddings. Seq_representation is concatenated with word_embeddings.
+-----------+ | word_embed|--------------------------->+ +--------------------+ +-----------+ | | doc_representation | +-----------+ +-------------------+ |-->+--------------------+ | seq_embed |-->| seq_representation|--->+ | word_representation| +-----------+ +-------------------+ +--------------------+ joint_representation
-
forward
(word_seq_embed: Tuple[torch.Tensor, torch.Tensor], word_lengths: torch.Tensor, seq_lengths: torch.Tensor, *args) → List[torch.Tensor][source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
pytext.models.representations.docnn module¶
-
class
pytext.models.representations.docnn.
DocNNRepresentation
(config: pytext.models.representations.docnn.DocNNRepresentation.Config, embed_dim: int)[source]¶ Bases:
pytext.models.representations.representation_base.RepresentationBase
CNN based representation of a document.
-
Config
[source]¶ alias of
DocNNRepresentation.Config
-
forward
(embedded_tokens: torch.Tensor, *args) → torch.Tensor[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
pytext.models.representations.jointcnn_rep module¶
-
class
pytext.models.representations.jointcnn_rep.
JointCNNRepresentation
(config: pytext.models.representations.jointcnn_rep.JointCNNRepresentation.Config, embed_dim: int)[source]¶ Bases:
pytext.models.representations.representation_base.RepresentationBase
-
Config
[source]¶ alias of
JointCNNRepresentation.Config
-
forward
(embedded_tokens: torch.Tensor, *args) → List[torch.Tensor][source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
pytext.models.representations.pair_rep module¶
-
class
pytext.models.representations.pair_rep.
PairRepresentation
(config: pytext.models.representations.pair_rep.PairRepresentation.Config, embed_dim: Tuple[int, ...])[source]¶ Bases:
pytext.models.representations.representation_base.RepresentationBase
Wrapper representation for a pair of inputs.
Takes a tuple of inputs: the left sentence, and the right sentence(s). Returns a representation of the pair of sentences, either as a concatenation of the two sentence embeddings or as a “siamese” representation which also includes their difference and elementwise product (arXiv:1705.02364). If more than two inputs are provided, the extra inputs are assumed to be extra “right” sentences, and the output will be the stacked pair representations of the left sentence together with all right sentences. This is more efficient than separately computing all these pair representations, because the left sentence will not need to be re-embedded multiple times.
-
Config
[source]¶ alias of
PairRepresentation.Config
-
forward
(embeddings: Tuple[torch.Tensor, ...], *lengths) → torch.Tensor[source]¶ Computes the pair representations.
Parameters: - embeddings – token embeddings of the left sentence, followed by the token embeddings of the right sentence(s).
- lengths – the corresponding sequence lengths.
Returns: A tensor of shape (num_right_inputs, batch_size, rep_size), with the first dimension squeezed if one.
-
pytext.models.representations.pooling module¶
-
class
pytext.models.representations.pooling.
BoundaryPool
(config: pytext.models.representations.pooling.BoundaryPool.Config, n_input: int)[source]¶ Bases:
pytext.models.module.Module
-
Config
[source]¶ alias of
BoundaryPool.Config
-
forward
(inputs: torch.Tensor, seq_lengths: torch.Tensor = None) → torch.Tensor[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
pytext.models.representations.pooling.
MaxPool
(config: pytext.config.module_config.ModuleConfig, n_input: int)[source]¶ Bases:
pytext.models.module.Module
-
Config
¶ alias of
pytext.config.component.ComponentMeta.__new__.<locals>.Config
-
forward
(inputs: torch.Tensor, seq_lengths: torch.Tensor = None) → torch.Tensor[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
pytext.models.representations.pooling.
MeanPool
(config: pytext.config.module_config.ModuleConfig, n_input: int)[source]¶ Bases:
pytext.models.module.Module
-
Config
¶ alias of
pytext.config.component.ComponentMeta.__new__.<locals>.Config
-
forward
(inputs: torch.Tensor, seq_lengths: torch.Tensor) → torch.Tensor[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
pytext.models.representations.pooling.
NoPool
(config: pytext.config.module_config.ModuleConfig, n_input: int)[source]¶ Bases:
pytext.models.module.Module
-
Config
¶ alias of
pytext.config.component.ComponentMeta.__new__.<locals>.Config
-
forward
(inputs: torch.Tensor, seq_lengths: torch.Tensor = None) → torch.Tensor[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
pytext.models.representations.pooling.
SelfAttention
(config: pytext.models.representations.pooling.SelfAttention.Config, n_input: int)[source]¶ Bases:
pytext.models.module.Module
-
Config
[source]¶ alias of
SelfAttention.Config
-
forward
(inputs: torch.Tensor, seq_lengths: torch.Tensor = None) → torch.Tensor[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
pytext.models.representations.pure_doc_attention module¶
-
class
pytext.models.representations.pure_doc_attention.
PureDocAttention
(config: pytext.models.representations.pure_doc_attention.PureDocAttention.Config, embed_dim: int)[source]¶ Bases:
pytext.models.representations.representation_base.RepresentationBase
pooling (e.g. max pooling or self attention) followed by optional MLP
-
Config
[source]¶ alias of
PureDocAttention.Config
-
forward
(embedded_tokens: torch.Tensor, seq_lengths: torch.Tensor, *args) → Any[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
pytext.models.representations.representation_base module¶
-
class
pytext.models.representations.representation_base.
RepresentationBase
(config)[source]¶ Bases:
pytext.models.module.Module
-
Config
¶ alias of
pytext.config.component.ComponentMeta.__new__.<locals>.Config
-
forward
(*inputs)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
pytext.models.representations.seq_rep module¶
-
class
pytext.models.representations.seq_rep.
SeqRepresentation
(config: pytext.models.representations.seq_rep.SeqRepresentation.Config, embed_dim: int)[source]¶ Bases:
pytext.models.representations.representation_base.RepresentationBase
Representation for a sequence of sentences Each sentence will be embedded with a DocNN model, then all the sentences are embedded with another DocNN/BiLSTM model
-
Config
[source]¶ alias of
SeqRepresentation.Config
-
forward
(embedded_seqs: torch.Tensor, seq_lengths: torch.Tensor, *args) → torch.Tensor[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
pytext.models.representations.slot_attention module¶
-
class
pytext.models.representations.slot_attention.
SlotAttention
(config: pytext.models.representations.slot_attention.SlotAttention.Config, n_input: int, batch_first: bool = True)[source]¶ Bases:
pytext.models.module.Module
-
Config
[source]¶ alias of
SlotAttention.Config
-
forward
(inputs: torch.Tensor) → torch.Tensor[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-