pytext.trainers package

Submodules

pytext.trainers.ensemble_trainer module

class pytext.trainers.ensemble_trainer.EnsembleTrainer(real_trainer)[source]

Bases: pytext.trainers.trainer.TrainerBase

Trainer for ensemble models

real_trainer

Trainer – the actual trainer to run

Config[source]

alias of EnsembleTrainer.Config

classmethod from_config(config: pytext.trainers.ensemble_trainer.EnsembleTrainer.Config, *args, **kwargs)[source]
train(train_iter, eval_iter, model, *args, **kwargs)[source]

pytext.trainers.hogwild_trainer module

class pytext.trainers.hogwild_trainer.HogwildTrainer(real_trainer_config, num_workers, *args, **kwargs)[source]

Bases: pytext.trainers.trainer.Trainer

Config[source]

alias of HogwildTrainer.Config

classmethod from_config(config: pytext.trainers.hogwild_trainer.HogwildTrainer.Config, *args, **kwargs)[source]
train(train_iter: torchtext.data.iterator.Iterator, eval_iter: torchtext.data.iterator.Iterator, model: pytext.models.model.Model, metric_reporter: pytext.metric_reporters.metric_reporter.MetricReporter, optimizers: List[torch.optim.optimizer.Optimizer], pytext_config: pytext.config.pytext_config.PyTextConfig, scheduler=None, *args, **kwargs) → Tuple[torch.nn.modules.module.Module, Any][source]

Train and eval a model, the model states will be modified. This function iterates epochs specified in config, and for each epoch do:

  1. Train model using training data, aggregate and report training results
  2. Adjust learning rate if scheduler is specified
  3. Evaluate model using evaluation data
  4. Calculate metrics based on evaluation results and select best model
Parameters:
  • train_iter (BatchIterator) – batch iterator of training data
  • eval_iter (BatchIterator) – batch iterator of evaluation data
  • model (Model) – model to be trained
  • metric_reporter (MetricReporter) – compute metric based on training output and report results to console, file.. etc
  • train_config (PyTextConfig) – training config
  • optimizers (List[torch.optim.Optimizer]) – a list of torch optimizers, in most of the case only contains one optimizer
  • scheduler (Optional[torch.optim.lr_scheduler]) – learning rate scheduler, default is None
  • training_result (Optional) – only meaningful for Hogwild training. default is None
  • rank (int) – only used in distributed training, the rank of the current training thread, evaluation will only be done in rank 0
Returns:

the trained model together with the best metric

Return type:

model, best_metric

pytext.trainers.trainer module

class pytext.trainers.trainer.Trainer(config=None)[source]

Bases: pytext.trainers.trainer.TrainerBase

Base Trainer class that provide ways to
1 Train model, compute metrics against eval set and use the metrics for model selection. 2 Test trained model, compute and publish metrics against a blind test set.
random_seed

int – Manual random seed

epochs

int – Training epochs

early_stop_after

int – Stop after how many epochs when the eval metric is not improving

max_clip_norm

Optional[float] – Clip gradient norm if set

report_train_metrics

bool – Whether metrics on training data should be computed and reported.

Config[source]

alias of Trainer.Config

test(test_iter, model, metric_reporter: pytext.metric_reporters.metric_reporter.MetricReporter)[source]
train(train_iter: pytext.data.data_handler.BatchIterator, eval_iter: pytext.data.data_handler.BatchIterator, model: pytext.models.model.Model, metric_reporter: pytext.metric_reporters.metric_reporter.MetricReporter, train_config: pytext.config.pytext_config.PyTextConfig, optimizers: List[torch.optim.optimizer.Optimizer], scheduler=None, rank: int = 0) → Tuple[torch.nn.modules.module.Module, Any][source]

Train and eval a model, the model states will be modified. This function iterates epochs specified in config, and for each epoch do:

  1. Train model using training data, aggregate and report training results
  2. Adjust learning rate if scheduler is specified
  3. Evaluate model using evaluation data
  4. Calculate metrics based on evaluation results and select best model
Parameters:
  • train_iter (BatchIterator) – batch iterator of training data
  • eval_iter (BatchIterator) – batch iterator of evaluation data
  • model (Model) – model to be trained
  • metric_reporter (MetricReporter) – compute metric based on training output and report results to console, file.. etc
  • train_config (PyTextConfig) – training config
  • optimizers (List[torch.optim.Optimizer]) – a list of torch optimizers, in most of the case only contains one optimizer
  • scheduler (Optional[torch.optim.lr_scheduler]) – learning rate scheduler, default is None
  • training_result (Optional) – only meaningful for Hogwild training. default is None
  • rank (int) – only used in distributed training, the rank of the current training thread, evaluation will only be done in rank 0
Returns:

the trained model together with the best metric

Return type:

model, best_metric

class pytext.trainers.trainer.TrainerBase(config=None)[source]

Bases: pytext.config.component.Component

Config

alias of pytext.config.component.ComponentMeta.__new__.<locals>.Config

Module contents

class pytext.trainers.Trainer(config=None)[source]

Bases: pytext.trainers.trainer.TrainerBase

Base Trainer class that provide ways to
1 Train model, compute metrics against eval set and use the metrics for model selection. 2 Test trained model, compute and publish metrics against a blind test set.
random_seed

int – Manual random seed

epochs

int – Training epochs

early_stop_after

int – Stop after how many epochs when the eval metric is not improving

max_clip_norm

Optional[float] – Clip gradient norm if set

report_train_metrics

bool – Whether metrics on training data should be computed and reported.

Config[source]

alias of Trainer.Config

test(test_iter, model, metric_reporter: pytext.metric_reporters.metric_reporter.MetricReporter)[source]
train(train_iter: pytext.data.data_handler.BatchIterator, eval_iter: pytext.data.data_handler.BatchIterator, model: pytext.models.model.Model, metric_reporter: pytext.metric_reporters.metric_reporter.MetricReporter, train_config: pytext.config.pytext_config.PyTextConfig, optimizers: List[torch.optim.optimizer.Optimizer], scheduler=None, rank: int = 0) → Tuple[torch.nn.modules.module.Module, Any][source]

Train and eval a model, the model states will be modified. This function iterates epochs specified in config, and for each epoch do:

  1. Train model using training data, aggregate and report training results
  2. Adjust learning rate if scheduler is specified
  3. Evaluate model using evaluation data
  4. Calculate metrics based on evaluation results and select best model
Parameters:
  • train_iter (BatchIterator) – batch iterator of training data
  • eval_iter (BatchIterator) – batch iterator of evaluation data
  • model (Model) – model to be trained
  • metric_reporter (MetricReporter) – compute metric based on training output and report results to console, file.. etc
  • train_config (PyTextConfig) – training config
  • optimizers (List[torch.optim.Optimizer]) – a list of torch optimizers, in most of the case only contains one optimizer
  • scheduler (Optional[torch.optim.lr_scheduler]) – learning rate scheduler, default is None
  • training_result (Optional) – only meaningful for Hogwild training. default is None
  • rank (int) – only used in distributed training, the rank of the current training thread, evaluation will only be done in rank 0
Returns:

the trained model together with the best metric

Return type:

model, best_metric

class pytext.trainers.EnsembleTrainer(real_trainer)[source]

Bases: pytext.trainers.trainer.TrainerBase

Trainer for ensemble models

real_trainer

Trainer – the actual trainer to run

Config[source]

alias of EnsembleTrainer.Config

classmethod from_config(config: pytext.trainers.ensemble_trainer.EnsembleTrainer.Config, *args, **kwargs)[source]
train(train_iter, eval_iter, model, *args, **kwargs)[source]
class pytext.trainers.HogwildTrainer(real_trainer_config, num_workers, *args, **kwargs)[source]

Bases: pytext.trainers.trainer.Trainer

Config[source]

alias of HogwildTrainer.Config

classmethod from_config(config: pytext.trainers.hogwild_trainer.HogwildTrainer.Config, *args, **kwargs)[source]
train(train_iter: torchtext.data.iterator.Iterator, eval_iter: torchtext.data.iterator.Iterator, model: pytext.models.model.Model, metric_reporter: pytext.metric_reporters.metric_reporter.MetricReporter, optimizers: List[torch.optim.optimizer.Optimizer], pytext_config: pytext.config.pytext_config.PyTextConfig, scheduler=None, *args, **kwargs) → Tuple[torch.nn.modules.module.Module, Any][source]

Train and eval a model, the model states will be modified. This function iterates epochs specified in config, and for each epoch do:

  1. Train model using training data, aggregate and report training results
  2. Adjust learning rate if scheduler is specified
  3. Evaluate model using evaluation data
  4. Calculate metrics based on evaluation results and select best model
Parameters:
  • train_iter (BatchIterator) – batch iterator of training data
  • eval_iter (BatchIterator) – batch iterator of evaluation data
  • model (Model) – model to be trained
  • metric_reporter (MetricReporter) – compute metric based on training output and report results to console, file.. etc
  • train_config (PyTextConfig) – training config
  • optimizers (List[torch.optim.Optimizer]) – a list of torch optimizers, in most of the case only contains one optimizer
  • scheduler (Optional[torch.optim.lr_scheduler]) – learning rate scheduler, default is None
  • training_result (Optional) – only meaningful for Hogwild training. default is None
  • rank (int) – only used in distributed training, the rank of the current training thread, evaluation will only be done in rank 0
Returns:

the trained model together with the best metric

Return type:

model, best_metric