pytext.task package

Submodules

pytext.task.disjoint_multitask module

class pytext.task.disjoint_multitask.DisjointMultitask(trainer: pytext.trainers.trainer.Trainer, data_handler: pytext.data.data_handler.DataHandler, model: pytext.models.model.Model, metric_reporter: pytext.metric_reporters.metric_reporter.MetricReporter, optimizers: List[torch.optim.optimizer.Optimizer], lr_scheduler: List[torch.optim.lr_scheduler._LRScheduler], exporter: Optional[pytext.exporters.exporter.ModelExporter])[source]

Bases: pytext.task.task.TaskBase

Modules which have the same shared_module_key and type share parameters. Only the first instance of such module should be configured in tasks list.

Config[source]

alias of DisjointMultitask.Config

classmethod from_config(task_config, metadata=None, model_state=None)[source]

Create the task from config, and optionally load metadata/model_state This function will create components including DataHandler, Trainer, Optimizer, Scheduler, MetricReporter, Exporter, and wire them up.

Parameters:
  • task_config (Task.Config) – the config of the current task
  • metadata – saved global context of this task, e.g: vocabulary, will be generated by DataHandler if it’s None
  • model_state – saved model parameters, will be loaded into model when given

pytext.task.serialize module

pytext.task.serialize.load(load_path: str)[source]

Load task, will construct the task using the saved config then load metadata and model state.

pytext.task.serialize.save(config: pytext.config.pytext_config.PyTextConfig, model: pytext.models.model.Model, meta: pytext.data.data_handler.CommonMetadata) → None[source]

Save a task, will save the original config, model state and metadata

pytext.task.task module

class pytext.task.task.Task(trainer: pytext.trainers.trainer.Trainer, data_handler: pytext.data.data_handler.DataHandler, model: pytext.models.model.Model, metric_reporter: pytext.metric_reporters.metric_reporter.MetricReporter, optimizers: List[torch.optim.optimizer.Optimizer], lr_scheduler: List[torch.optim.lr_scheduler._LRScheduler], exporter: Optional[pytext.exporters.exporter.ModelExporter])[source]

Bases: pytext.task.task.TaskBase

Config

alias of pytext.config.component.ComponentMeta.__new__.<locals>.Config

class pytext.task.task.TaskBase(trainer: pytext.trainers.trainer.Trainer, data_handler: pytext.data.data_handler.DataHandler, model: pytext.models.model.Model, metric_reporter: pytext.metric_reporters.metric_reporter.MetricReporter, optimizers: List[torch.optim.optimizer.Optimizer], lr_scheduler: List[torch.optim.lr_scheduler._LRScheduler], exporter: Optional[pytext.exporters.exporter.ModelExporter])[source]

Bases: pytext.config.component.Component

Task is the central place to define and wire up components for data processing, model training, metric reporting, etc. Task class has a Config class containing the config of each component in a descriptive way.

Config[source]

alias of TaskBase.Config

export(model, export_path, summary_writer=None, export_onnx_path=None)[source]

Wrapper method to export PyTorch model to Caffe2 model using Exporter.

Parameters:
  • export_path (str) – file path of exported caffe2 model
  • summary_writer – TensorBoard SummaryWriter, used to output the PyTorch model’s execution graph to TensorBoard, default is None.
  • export_onnx_path (str) – file path of exported onnx model
classmethod format_prediction(predictions, scores, context, target_meta)[source]

Format the prediction and score from model output, by default just return them in a dict

classmethod from_config(task_config, metadata=None, model_state=None)[source]

Create the task from config, and optionally load metadata/model_state This function will create components including DataHandler, Trainer, Optimizer, Scheduler, MetricReporter, Exporter, and wire them up.

Parameters:
  • task_config (Task.Config) – the config of the current task
  • metadata – saved global context of this task, e.g: vocabulary, will be generated by DataHandler if it’s None
  • model_state – saved model parameters, will be loaded into model when given
predict(examples)[source]

Generates predictions using PyTorch model. The difference with test() is that this should be used when the the examples do not have any true label/target.

Parameters:examples – json format examples, input names should match the names specified in this task’s features config
test(test_path)[source]

Wrapper method to compute test metrics on holdout blind test dataset.

Parameters:test_path (str) – test data file path
train(train_config, rank=0, world_size=1)[source]

Wrapper method to train the model using Trainer object.

Parameters:
  • train_config (PyTextConfig) – config for training
  • rank (int) – for distributed training only, rank of the gpu, default is 0
  • world_size (int) – for distributed training only, total gpu to use, default is 1
pytext.task.task.create_task(task_config, metadata=None, model_state=None)[source]

Create a task by finding task class in registry and invoking the from_config function of the class, see from_config() for more details

pytext.task.tasks module

class pytext.task.tasks.ContextualIntentSlotTask(trainer: pytext.trainers.trainer.Trainer, data_handler: pytext.data.data_handler.DataHandler, model: pytext.models.model.Model, metric_reporter: pytext.metric_reporters.metric_reporter.MetricReporter, optimizers: List[torch.optim.optimizer.Optimizer], lr_scheduler: List[torch.optim.lr_scheduler._LRScheduler], exporter: Optional[pytext.exporters.exporter.ModelExporter])[source]

Bases: pytext.task.task.Task

Config[source]

alias of ContextualIntentSlotTask.Config

class pytext.task.tasks.DocClassificationTask(trainer: pytext.trainers.trainer.Trainer, data_handler: pytext.data.data_handler.DataHandler, model: pytext.models.model.Model, metric_reporter: pytext.metric_reporters.metric_reporter.MetricReporter, optimizers: List[torch.optim.optimizer.Optimizer], lr_scheduler: List[torch.optim.lr_scheduler._LRScheduler], exporter: Optional[pytext.exporters.exporter.ModelExporter])[source]

Bases: pytext.task.task.Task

Config[source]

alias of DocClassificationTask.Config

classmethod format_prediction(predictions, scores, context, target_meta)[source]

Format the prediction and score from model output, by default just return them in a dict

class pytext.task.tasks.EnsembleTask(trainer: pytext.trainers.trainer.Trainer, data_handler: pytext.data.data_handler.DataHandler, model: pytext.models.model.Model, metric_reporter: pytext.metric_reporters.metric_reporter.MetricReporter, optimizers: List[torch.optim.optimizer.Optimizer], lr_scheduler: List[torch.optim.lr_scheduler._LRScheduler], exporter: Optional[pytext.exporters.exporter.ModelExporter])[source]

Bases: pytext.task.task.Task

Config[source]

alias of EnsembleTask.Config

train_single_model(train_config, model_id, rank=0, world_size=1)[source]
class pytext.task.tasks.JointTextTask(trainer: pytext.trainers.trainer.Trainer, data_handler: pytext.data.data_handler.DataHandler, model: pytext.models.model.Model, metric_reporter: pytext.metric_reporters.metric_reporter.MetricReporter, optimizers: List[torch.optim.optimizer.Optimizer], lr_scheduler: List[torch.optim.lr_scheduler._LRScheduler], exporter: Optional[pytext.exporters.exporter.ModelExporter])[source]

Bases: pytext.task.task.Task

Config[source]

alias of JointTextTask.Config

classmethod format_prediction(predictions, scores, context, target_meta)[source]

Format the prediction and score from model output, by default just return them in a dict

class pytext.task.tasks.KDDocClassificationTask(trainer: pytext.trainers.trainer.Trainer, data_handler: pytext.data.data_handler.DataHandler, model: pytext.models.model.Model, metric_reporter: pytext.metric_reporters.metric_reporter.MetricReporter, optimizers: List[torch.optim.optimizer.Optimizer], lr_scheduler: List[torch.optim.lr_scheduler._LRScheduler], exporter: Optional[pytext.exporters.exporter.ModelExporter])[source]

Bases: pytext.task.task.Task

Config[source]

alias of KDDocClassificationTask.Config

class pytext.task.tasks.LMTask(trainer: pytext.trainers.trainer.Trainer, data_handler: pytext.data.data_handler.DataHandler, model: pytext.models.model.Model, metric_reporter: pytext.metric_reporters.metric_reporter.MetricReporter, optimizers: List[torch.optim.optimizer.Optimizer], lr_scheduler: List[torch.optim.lr_scheduler._LRScheduler], exporter: Optional[pytext.exporters.exporter.ModelExporter])[source]

Bases: pytext.task.task.Task

Config[source]

alias of LMTask.Config

class pytext.task.tasks.PairClassificationTask(trainer: pytext.trainers.trainer.Trainer, data_handler: pytext.data.data_handler.DataHandler, model: pytext.models.model.Model, metric_reporter: pytext.metric_reporters.metric_reporter.MetricReporter, optimizers: List[torch.optim.optimizer.Optimizer], lr_scheduler: List[torch.optim.lr_scheduler._LRScheduler], exporter: Optional[pytext.exporters.exporter.ModelExporter])[source]

Bases: pytext.task.task.Task

Config[source]

alias of PairClassificationTask.Config

class pytext.task.tasks.SemanticParsingTask(trainer: pytext.trainers.trainer.Trainer, data_handler: pytext.data.data_handler.DataHandler, model: pytext.models.model.Model, metric_reporter: pytext.metric_reporters.metric_reporter.MetricReporter, optimizers: List[torch.optim.optimizer.Optimizer], lr_scheduler: List[torch.optim.lr_scheduler._LRScheduler], exporter: Optional[pytext.exporters.exporter.ModelExporter])[source]

Bases: pytext.task.task.Task

Config[source]

alias of SemanticParsingTask.Config

class pytext.task.tasks.SeqNNTask(trainer: pytext.trainers.trainer.Trainer, data_handler: pytext.data.data_handler.DataHandler, model: pytext.models.model.Model, metric_reporter: pytext.metric_reporters.metric_reporter.MetricReporter, optimizers: List[torch.optim.optimizer.Optimizer], lr_scheduler: List[torch.optim.lr_scheduler._LRScheduler], exporter: Optional[pytext.exporters.exporter.ModelExporter])[source]

Bases: pytext.task.task.Task

Config[source]

alias of SeqNNTask.Config

class pytext.task.tasks.WordTaggingTask(trainer: pytext.trainers.trainer.Trainer, data_handler: pytext.data.data_handler.DataHandler, model: pytext.models.model.Model, metric_reporter: pytext.metric_reporters.metric_reporter.MetricReporter, optimizers: List[torch.optim.optimizer.Optimizer], lr_scheduler: List[torch.optim.lr_scheduler._LRScheduler], exporter: Optional[pytext.exporters.exporter.ModelExporter])[source]

Bases: pytext.task.task.Task

Config[source]

alias of WordTaggingTask.Config

classmethod format_prediction(predictions, scores, context, target_meta)[source]

Format the prediction and score from model output, by default just return them in a dict

Module contents

class pytext.task.Task(trainer: pytext.trainers.trainer.Trainer, data_handler: pytext.data.data_handler.DataHandler, model: pytext.models.model.Model, metric_reporter: pytext.metric_reporters.metric_reporter.MetricReporter, optimizers: List[torch.optim.optimizer.Optimizer], lr_scheduler: List[torch.optim.lr_scheduler._LRScheduler], exporter: Optional[pytext.exporters.exporter.ModelExporter])[source]

Bases: pytext.task.task.TaskBase

Config

alias of pytext.config.component.ComponentMeta.__new__.<locals>.Config

class pytext.task.TaskBase(trainer: pytext.trainers.trainer.Trainer, data_handler: pytext.data.data_handler.DataHandler, model: pytext.models.model.Model, metric_reporter: pytext.metric_reporters.metric_reporter.MetricReporter, optimizers: List[torch.optim.optimizer.Optimizer], lr_scheduler: List[torch.optim.lr_scheduler._LRScheduler], exporter: Optional[pytext.exporters.exporter.ModelExporter])[source]

Bases: pytext.config.component.Component

Task is the central place to define and wire up components for data processing, model training, metric reporting, etc. Task class has a Config class containing the config of each component in a descriptive way.

Config[source]

alias of TaskBase.Config

export(model, export_path, summary_writer=None, export_onnx_path=None)[source]

Wrapper method to export PyTorch model to Caffe2 model using Exporter.

Parameters:
  • export_path (str) – file path of exported caffe2 model
  • summary_writer – TensorBoard SummaryWriter, used to output the PyTorch model’s execution graph to TensorBoard, default is None.
  • export_onnx_path (str) – file path of exported onnx model
classmethod format_prediction(predictions, scores, context, target_meta)[source]

Format the prediction and score from model output, by default just return them in a dict

classmethod from_config(task_config, metadata=None, model_state=None)[source]

Create the task from config, and optionally load metadata/model_state This function will create components including DataHandler, Trainer, Optimizer, Scheduler, MetricReporter, Exporter, and wire them up.

Parameters:
  • task_config (Task.Config) – the config of the current task
  • metadata – saved global context of this task, e.g: vocabulary, will be generated by DataHandler if it’s None
  • model_state – saved model parameters, will be loaded into model when given
predict(examples)[source]

Generates predictions using PyTorch model. The difference with test() is that this should be used when the the examples do not have any true label/target.

Parameters:examples – json format examples, input names should match the names specified in this task’s features config
test(test_path)[source]

Wrapper method to compute test metrics on holdout blind test dataset.

Parameters:test_path (str) – test data file path
train(train_config, rank=0, world_size=1)[source]

Wrapper method to train the model using Trainer object.

Parameters:
  • train_config (PyTextConfig) – config for training
  • rank (int) – for distributed training only, rank of the gpu, default is 0
  • world_size (int) – for distributed training only, total gpu to use, default is 1
pytext.task.save(config: pytext.config.pytext_config.PyTextConfig, model: pytext.models.model.Model, meta: pytext.data.data_handler.CommonMetadata) → None[source]

Save a task, will save the original config, model state and metadata

pytext.task.load(load_path: str)[source]

Load task, will construct the task using the saved config then load metadata and model state.

pytext.task.create_task(task_config, metadata=None, model_state=None)[source]

Create a task by finding task class in registry and invoking the from_config function of the class, see from_config() for more details