dicee.executer ============== .. py:module:: dicee.executer .. autoapi-nested-parse:: Executor module for training, retraining and evaluating KGE models. This module provides the Execute and ContinuousExecute classes for managing the full lifecycle of knowledge graph embedding model training. Classes ------- .. autoapisummary:: dicee.executer.Execute dicee.executer.ContinuousExecute Module Contents --------------- .. py:class:: Execute(args, continuous_training: bool = False) Executor class for training, retraining and evaluating KGE models. Handles the complete workflow: 1. Loading & Preprocessing & Serializing input data 2. Training & Validation & Testing 3. Storing all necessary information .. attribute:: args Processed input arguments. .. attribute:: distributed Whether distributed training is enabled. .. attribute:: rank Process rank in distributed training. .. attribute:: world_size Total number of processes. .. attribute:: local_rank Local GPU rank. .. attribute:: trainer Training handler instance. .. attribute:: trained_model The trained model after training completes. .. attribute:: knowledge_graph The loaded knowledge graph. .. attribute:: report Dictionary storing training metrics and results. .. attribute:: evaluator Model evaluation handler. .. py:attribute:: distributed .. py:attribute:: args .. py:attribute:: is_continual_training :value: False .. py:attribute:: trainer :type: Optional[dicee.trainer.DICE_Trainer] :value: None .. py:attribute:: trained_model :value: None .. py:attribute:: knowledge_graph :type: Optional[dicee.knowledge_graph.KG] :value: None .. py:attribute:: report :type: Dict .. py:attribute:: evaluator :type: Optional[dicee.evaluator.Evaluator] :value: None .. py:attribute:: start_time :type: Optional[float] :value: None .. py:method:: is_rank_zero() -> bool .. py:method:: cleanup() .. py:method:: setup_executor() -> None Set up storage directories for the experiment. Creates or reuses experiment directories based on configuration. Saves the configuration to a JSON file. .. py:method:: create_and_store_kg() -> None Create knowledge graph and store as memory-mapped file. Only executed on rank 0 in distributed training. Skips if memmap already exists. .. py:method:: load_from_memmap() -> None Load knowledge graph from memory-mapped file. .. py:method:: save_trained_model() -> None Save a knowledge graph embedding model (1) Send model to eval mode and cpu. (2) Store the memory footprint of the model. (3) Save the model into disk. (4) Update the stats of KG again ? Parameter ---------- :rtype: None .. py:method:: end(form_of_labelling: str) -> dict End training (1) Store trained model. (2) Report runtimes. (3) Eval model if required. Parameter --------- :rtype: A dict containing information about the training and/or evaluation .. py:method:: write_report() -> None Report training related information in a report.json file .. py:method:: start() -> dict Start training # (1) Loading the Data # (2) Create an evaluator object. # (3) Create a trainer object. # (4) Start the training Parameter --------- :rtype: A dict containing information about the training and/or evaluation .. py:class:: ContinuousExecute(args) Bases: :py:obj:`Execute` A subclass of Execute Class for retraining (1) Loading & Preprocessing & Serializing input data. (2) Training & Validation & Testing (3) Storing all necessary info During the continual learning we can only modify *** num_epochs *** parameter. Trained model stored in the same folder as the seed model for the training. Trained model is noted with the current time. .. py:method:: continual_start() -> dict Start Continual Training (1) Initialize training. (2) Start continual training. (3) Save trained model. Parameter --------- :rtype: A dict containing information about the training and/or evaluation