dicee.executer
Executor module for training, retraining and evaluating KGE models.
This module provides the Execute and ContinuousExecute classes for managing the full lifecycle of knowledge graph embedding model training.
Classes
Executor class for training, retraining and evaluating KGE models. |
|
A subclass of Execute Class for retraining |
Module Contents
- class dicee.executer.Execute(args, continuous_training: bool = False)
Executor class for training, retraining and evaluating KGE models.
Handles the complete workflow: 1. Loading & Preprocessing & Serializing input data 2. Training & Validation & Testing 3. Storing all necessary information
- args
Processed input arguments.
- distributed
Whether distributed training is enabled.
- rank
Process rank in distributed training.
- world_size
Total number of processes.
- local_rank
Local GPU rank.
- trainer
Training handler instance.
- trained_model
The trained model after training completes.
- knowledge_graph
The loaded knowledge graph.
- report
Dictionary storing training metrics and results.
- evaluator
Model evaluation handler.
- distributed
- args
- is_continual_training = False
- trainer: dicee.trainer.DICE_Trainer | None = None
- trained_model = None
- knowledge_graph: dicee.knowledge_graph.KG | None = None
- report: Dict
- evaluator: dicee.evaluator.Evaluator | None = None
- start_time: float | None = None
- is_rank_zero() bool
- cleanup()
- setup_executor() None
Set up storage directories for the experiment.
Creates or reuses experiment directories based on configuration. Saves the configuration to a JSON file.
- create_and_store_kg() None
Create knowledge graph and store as memory-mapped file.
Only executed on rank 0 in distributed training. Skips if memmap already exists.
- load_from_memmap() None
Load knowledge graph from memory-mapped file.
- save_trained_model() None
Save a knowledge graph embedding model
Send model to eval mode and cpu.
Store the memory footprint of the model.
Save the model into disk.
Update the stats of KG again ?
Parameter
- rtype:
None
- end(form_of_labelling: str) dict
End training
Store trained model.
Report runtimes.
Eval model if required.
Parameter
- rtype:
A dict containing information about the training and/or evaluation
- write_report() None
Report training related information in a report.json file
- class dicee.executer.ContinuousExecute(args)
Bases:
ExecuteA subclass of Execute Class for retraining
Loading & Preprocessing & Serializing input data.
Training & Validation & Testing
Storing all necessary info
During the continual learning we can only modify * num_epochs * parameter. Trained model stored in the same folder as the seed model for the training. Trained model is noted with the current time.