dicee.read_preprocess_save_load_kg

Submodules

• dicee.read_preprocess_save_load_kg.preprocess
• dicee.read_preprocess_save_load_kg.read_from_disk
• dicee.read_preprocess_save_load_kg.save_load_disk
• dicee.read_preprocess_save_load_kg.util

Classes

PreprocessKG	Preprocess the data in memory
LoadSaveToDisk
ReadFromDisk	Read the data from disk into memory

Package Contents

class dicee.read_preprocess_save_load_kg.PreprocessKG(kg)

Preprocess the data in memory

kg

start() → None

Preprocess train, valid and test datasets stored in knowledge graph instance

Parameter

rtype:

None

preprocess_with_byte_pair_encoding()

preprocess_with_byte_pair_encoding_with_padding() → None

Preprocess with byte pair encoding and add padding

preprocess_with_pandas() → None

Preprocess with pandas: add reciprocal triples, construct vocabulary, and index datasets

preprocess_with_polars() → None

Preprocess with polars: add reciprocal triples and create indexed datasets

sequential_vocabulary_construction() → None

1. Read input data into memory

2. Remove triples with a condition

3. Serialize vocabularies in a pandas dataframe where

   => the index is integer and => a single column is string (e.g. URI)

class dicee.read_preprocess_save_load_kg.LoadSaveToDisk(kg)

kg

save()

load()

class dicee.read_preprocess_save_load_kg.ReadFromDisk(kg)

Read the data from disk into memory

kg

start() → None

Read a knowledge graph from disk into memory

Data will be available at the train_set, test_set, valid_set attributes.

Parameter

None

rtype:

None

add_noisy_triples_into_training()