dicee.models.clifford ===================== .. py:module:: dicee.models.clifford Classes ------- .. autoapisummary:: dicee.models.clifford.Keci dicee.models.clifford.CKeci dicee.models.clifford.DeCaL Module Contents --------------- .. py:class:: Keci(args) Bases: :py:obj:`dicee.models.base_model.BaseKGE` Base class for all neural network modules. Your models should also subclass this class. Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:: import torch.nn as nn import torch.nn.functional as F class Model(nn.Module): def __init__(self) -> None: super().__init__() self.conv1 = nn.Conv2d(1, 20, 5) self.conv2 = nn.Conv2d(20, 20, 5) def forward(self, x): x = F.relu(self.conv1(x)) return F.relu(self.conv2(x)) Submodules assigned in this way will be registered, and will also have their parameters converted when you call :meth:`to`, etc. .. note:: As per the example above, an ``__init__()`` call to the parent class must be made before assignment on the child. :ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool .. py:attribute:: name :value: 'Keci' .. py:attribute:: p .. py:attribute:: q .. py:attribute:: r .. py:attribute:: requires_grad_for_interactions :value: True .. py:method:: compute_sigma_pp(hp, rp) Compute sigma_{pp} = \sum_{i=1}^{p-1} \sum_{k=i+1}^p (h_i r_k - h_k r_i) e_i e_k sigma_{pp} captures the interactions between along p bases For instance, let p e_1, e_2, e_3, we compute interactions between e_1 e_2, e_1 e_3 , and e_2 e_3 This can be implemented with a nested two for loops results = [] for i in range(p - 1): for k in range(i + 1, p): results.append(hp[:, :, i] * rp[:, :, k] - hp[:, :, k] * rp[:, :, i]) sigma_pp = torch.stack(results, dim=2) assert sigma_pp.shape == (b, r, int((p * (p - 1)) / 2)) Yet, this computation would be quite inefficient. Instead, we compute interactions along all p, e.g., e1e1, e1e2, e1e3, e2e1, e2e2, e2e3, e3e1, e3e2, e3e3 Then select the triangular matrix without diagonals: e1e2, e1e3, e2e3. .. py:method:: compute_sigma_qq(hq, rq) Compute sigma_{qq} = \sum_{j=1}^{p+q-1} \sum_{k=j+1}^{p+q} (h_j r_k - h_k r_j) e_j e_k sigma_{q} captures the interactions between along q bases For instance, let q e_1, e_2, e_3, we compute interactions between e_1 e_2, e_1 e_3 , and e_2 e_3 This can be implemented with a nested two for loops results = [] for j in range(q - 1): for k in range(j + 1, q): results.append(hq[:, :, j] * rq[:, :, k] - hq[:, :, k] * rq[:, :, j]) sigma_qq = torch.stack(results, dim=2) assert sigma_qq.shape == (b, r, int((q * (q - 1)) / 2)) Yet, this computation would be quite inefficient. Instead, we compute interactions along all p, e.g., e1e1, e1e2, e1e3, e2e1, e2e2, e2e3, e3e1, e3e2, e3e3 Then select the triangular matrix without diagonals: e1e2, e1e3, e2e3. .. py:method:: compute_sigma_pq(*, hp, hq, rp, rq) \sum_{i=1}^{p} \sum_{j=p+1}^{p+q} (h_i r_j - h_j r_i) e_i e_j results = [] sigma_pq = torch.zeros(b, r, p, q) for i in range(p): for j in range(q): sigma_pq[:, :, i, j] = hp[:, :, i] * rq[:, :, j] - hq[:, :, j] * rp[:, :, i] print(sigma_pq.shape) .. py:method:: apply_coefficients(hp, hq, rp, rq) Multiplying a base vector with its scalar coefficient .. py:method:: clifford_multiplication(h0, hp, hq, r0, rp, rq) Compute our CL multiplication h = h_0 + \sum_{i=1}^p h_i e_i + \sum_{j=p+1}^{p+q} h_j e_j r = r_0 + \sum_{i=1}^p r_i e_i + \sum_{j=p+1}^{p+q} r_j e_j ei ^2 = +1 for i =< i =< p ej ^2 = -1 for p < j =< p+q ei ej = -eje1 for i eq j h r = sigma_0 + sigma_p + sigma_q + sigma_{pp} + sigma_{q}+ sigma_{pq} where (1) sigma_0 = h_0 r_0 + \sum_{i=1}^p (h_0 r_i) e_i - \sum_{j=p+1}^{p+q} (h_j r_j) e_j (2) sigma_p = \sum_{i=1}^p (h_0 r_i + h_i r_0) e_i (3) sigma_q = \sum_{j=p+1}^{p+q} (h_0 r_j + h_j r_0) e_j (4) sigma_{pp} = \sum_{i=1}^{p-1} \sum_{k=i+1}^p (h_i r_k - h_k r_i) e_i e_k (5) sigma_{qq} = \sum_{j=1}^{p+q-1} \sum_{k=j+1}^{p+q} (h_j r_k - h_k r_j) e_j e_k (6) sigma_{pq} = \sum_{i=1}^{p} \sum_{j=p+1}^{p+q} (h_i r_j - h_j r_i) e_i e_j .. py:method:: construct_cl_multivector(x: torch.FloatTensor, r: int, p: int, q: int) -> tuple[torch.FloatTensor, torch.FloatTensor, torch.FloatTensor] Construct a batch of multivectors Cl_{p,q}(\mathbb{R}^d) Parameter --------- x: torch.FloatTensor with (n,d) shape :returns: * **a0** (*torch.FloatTensor with (n,r) shape*) * **ap** (*torch.FloatTensor with (n,r,p) shape*) * **aq** (*torch.FloatTensor with (n,r,q) shape*) .. py:method:: forward_k_vs_with_explicit(x: torch.Tensor) .. py:method:: k_vs_all_score(bpe_head_ent_emb, bpe_rel_ent_emb, E) .. py:method:: forward_k_vs_all(x: torch.Tensor) -> torch.FloatTensor Kvsall training (1) Retrieve real-valued embedding vectors for heads and relations \mathbb{R}^d . (2) Construct head entity and relation embeddings according to Cl_{p,q}(\mathbb{R}^d) . (3) Perform Cl multiplication (4) Inner product of (3) and all entity embeddings forward_k_vs_with_explicit and this funcitons are identical Parameter --------- x: torch.LongTensor with (n,2) shape :rtype: torch.FloatTensor with (n, |E|) shape .. py:method:: construct_batch_selected_cl_multivector(x: torch.FloatTensor, r: int, p: int, q: int) -> tuple[torch.FloatTensor, torch.FloatTensor, torch.FloatTensor] Construct a batch of batchs multivectors Cl_{p,q}(\mathbb{R}^d) Parameter --------- x: torch.FloatTensor with (n,k, d) shape :returns: * **a0** (*torch.FloatTensor with (n,k, m) shape*) * **ap** (*torch.FloatTensor with (n,k, m, p) shape*) * **aq** (*torch.FloatTensor with (n,k, m, q) shape*) .. py:method:: forward_k_vs_sample(x: torch.LongTensor, target_entity_idx: torch.LongTensor) -> torch.FloatTensor Parameter --------- x: torch.LongTensor with (n,2) shape target_entity_idx: torch.LongTensor with (n, k ) shape k denotes the selected number of examples. :rtype: torch.FloatTensor with (n, k) shape .. py:method:: score(h, r, t) .. py:method:: forward_triples(x: torch.Tensor) -> torch.FloatTensor Parameter --------- x: torch.LongTensor with (n,3) shape :rtype: torch.FloatTensor with (n) shape .. py:class:: CKeci(args) Bases: :py:obj:`Keci` Without learning dimension scaling .. py:attribute:: name :value: 'CKeci' .. py:attribute:: requires_grad_for_interactions :value: False .. py:class:: DeCaL(args) Bases: :py:obj:`dicee.models.base_model.BaseKGE` Base class for all neural network modules. Your models should also subclass this class. Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:: import torch.nn as nn import torch.nn.functional as F class Model(nn.Module): def __init__(self) -> None: super().__init__() self.conv1 = nn.Conv2d(1, 20, 5) self.conv2 = nn.Conv2d(20, 20, 5) def forward(self, x): x = F.relu(self.conv1(x)) return F.relu(self.conv2(x)) Submodules assigned in this way will be registered, and will also have their parameters converted when you call :meth:`to`, etc. .. note:: As per the example above, an ``__init__()`` call to the parent class must be made before assignment on the child. :ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool .. py:attribute:: name :value: 'DeCaL' .. py:attribute:: entity_embeddings .. py:attribute:: relation_embeddings .. py:attribute:: p .. py:attribute:: q .. py:attribute:: r .. py:attribute:: re .. py:method:: forward_triples(x: torch.Tensor) -> torch.FloatTensor Parameter --------- x: torch.LongTensor with (n, ) shape :rtype: torch.FloatTensor with (n) shape .. py:method:: cl_pqr(a: torch.tensor) -> torch.tensor Input: tensor(batch_size, emb_dim) ---> output: tensor with 1+p+q+r components with size (batch_size, emb_dim/(1+p+q+r)) each. 1) takes a tensor of size (batch_size, emb_dim), split it into 1 + p + q +r components, hence 1+p+q+r must be a divisor of the emb_dim. 2) Return a list of the 1+p+q+r components vectors, each are tensors of size (batch_size, emb_dim/(1+p+q+r)) .. py:method:: compute_sigmas_single(list_h_emb, list_r_emb, list_t_emb) here we compute all the sums with no others vectors interaction taken with the scalar product with t, that is, .. math:: s0 = h_0r_0t_0 s1 = \sum_{i=1}^{p}h_ir_it_0 s2 = \sum_{j=p+1}^{p+q}h_jr_jt_0 s3 = \sum_{i=1}^{q}(h_0r_it_i + h_ir_0t_i) s4 = \sum_{i=p+1}^{p+q}(h_0r_it_i + h_ir_0t_i) s5 = \sum_{i=p+q+1}^{p+q+r}(h_0r_it_i + h_ir_0t_i) and return: .. math:: sigma_0t = \sigma_0 \cdot t_0 = s0 + s1 -s2 s3, s4 and s5 .. py:method:: compute_sigmas_multivect(list_h_emb, list_r_emb) Here we compute and return all the sums with vectors interaction for the same and different bases. For same bases vectors interaction we have .. math:: \sigma_pp = \sum_{i=1}^{p-1}\sum_{i'=i+1}^{p}(h_ir_{i'}-h_{i'}r_i) (models the interactions between e_i and e_i' for 1 <= i, i' <= p) \sigma_qq = \sum_{j=p+1}^{p+q-1}\sum_{j'=j+1}^{p+q}(h_jr_{j'}-h_{j'} (models the interactions between e_j and e_j' for p+1 <= j, j' <= p+q) \sigma_rr = \sum_{k=p+q+1}^{p+q+r-1}\sum_{k'=k+1}^{p}(h_kr_{k'}-h_{k'}r_k) (models the interactions between e_k and e_k' for p+q+1 <= k, k' <= p+q+r) For different base vector interactions, we have .. math:: \sigma_pq = \sum_{i=1}^{p}\sum_{j=p+1}^{p+q}(h_ir_j - h_jr_i) (interactionsn between e_i and e_j for 1<=i <=p and p+1<= j <= p+q) \sigma_pr = \sum_{i=1}^{p}\sum_{k=p+q+1}^{p+q+r}(h_ir_k - h_kr_i) (interactionsn between e_i and e_k for 1<=i <=p and p+q+1<= k <= p+q+r) \sigma_qr = \sum_{j=p+1}^{p+q}\sum_{j=p+q+1}^{p+q+r}(h_jr_k - h_kr_j) (interactionsn between e_j and e_k for p+1 <= j <=p+q and p+q+1<= j <= p+q+r) .. py:method:: forward_k_vs_all(x: torch.Tensor) -> torch.FloatTensor Kvsall training (1) Retrieve real-valued embedding vectors for heads and relations (2) Construct head entity and relation embeddings according to Cl_{p,q, r}(\mathbb{R}^d) . (3) Perform Cl multiplication (4) Inner product of (3) and all entity embeddings forward_k_vs_with_explicit and this funcitons are identical Parameter --------- x: torch.LongTensor with (n, ) shape :rtype: torch.FloatTensor with (n, |E|) shape .. py:method:: apply_coefficients(h0, hp, hq, hk, r0, rp, rq, rk) Multiplying a base vector with its scalar coefficient .. py:method:: construct_cl_multivector(x: torch.FloatTensor, re: int, p: int, q: int, r: int) -> tuple[torch.FloatTensor, torch.FloatTensor, torch.FloatTensor] Construct a batch of multivectors Cl_{p,q,r}(\mathbb{R}^d) Parameter --------- x: torch.FloatTensor with (n,d) shape :returns: * **a0** (*torch.FloatTensor*) * **ap** (*torch.FloatTensor*) * **aq** (*torch.FloatTensor*) * **ar** (*torch.FloatTensor*) .. py:method:: compute_sigma_pp(hp, rp) Compute .. math:: \sigma_{p,p}^* = \sum_{i=1}^{p-1}\sum_{i'=i+1}^{p}(x_iy_{i'}-x_{i'}y_i) \sigma_{pp} captures the interactions between along p bases For instance, let p e_1, e_2, e_3, we compute interactions between e_1 e_2, e_1 e_3 , and e_2 e_3 This can be implemented with a nested two for loops results = [] for i in range(p - 1): for k in range(i + 1, p): results.append(hp[:, :, i] * rp[:, :, k] - hp[:, :, k] * rp[:, :, i]) sigma_pp = torch.stack(results, dim=2) assert sigma_pp.shape == (b, r, int((p * (p - 1)) / 2)) Yet, this computation would be quite inefficient. Instead, we compute interactions along all p, e.g., e1e1, e1e2, e1e3, e2e1, e2e2, e2e3, e3e1, e3e2, e3e3 Then select the triangular matrix without diagonals: e1e2, e1e3, e2e3. .. py:method:: compute_sigma_qq(hq, rq) Compute .. math:: \sigma_{q,q}^* = \sum_{j=p+1}^{p+q-1}\sum_{j'=j+1}^{p+q}(x_jy_{j'}-x_{j'}y_j) Eq. 16 sigma_{q} captures the interactions between along q bases For instance, let q e_1, e_2, e_3, we compute interactions between e_1 e_2, e_1 e_3 , and e_2 e_3 This can be implemented with a nested two for loops results = [] for j in range(q - 1): for k in range(j + 1, q): results.append(hq[:, :, j] * rq[:, :, k] - hq[:, :, k] * rq[:, :, j]) sigma_qq = torch.stack(results, dim=2) assert sigma_qq.shape == (b, r, int((q * (q - 1)) / 2)) Yet, this computation would be quite inefficient. Instead, we compute interactions along all p, e.g., e1e1, e1e2, e1e3, e2e1, e2e2, e2e3, e3e1, e3e2, e3e3 Then select the triangular matrix without diagonals: e1e2, e1e3, e2e3. .. py:method:: compute_sigma_rr(hk, rk) .. math:: \sigma_{r,r}^* = \sum_{k=p+q+1}^{p+q+r-1}\sum_{k'=k+1}^{p}(x_ky_{k'}-x_{k'}y_k) .. py:method:: compute_sigma_pq(*, hp, hq, rp, rq) Compute .. math:: \sum_{i=1}^{p} \sum_{j=p+1}^{p+q} (h_i r_j - h_j r_i) e_i e_j results = [] sigma_pq = torch.zeros(b, r, p, q) for i in range(p): for j in range(q): sigma_pq[:, :, i, j] = hp[:, :, i] * rq[:, :, j] - hq[:, :, j] * rp[:, :, i] print(sigma_pq.shape) .. py:method:: compute_sigma_pr(*, hp, hk, rp, rk) Compute .. math:: \sum_{i=1}^{p} \sum_{j=p+1}^{p+q} (h_i r_j - h_j r_i) e_i e_j results = [] sigma_pq = torch.zeros(b, r, p, q) for i in range(p): for j in range(q): sigma_pq[:, :, i, j] = hp[:, :, i] * rq[:, :, j] - hq[:, :, j] * rp[:, :, i] print(sigma_pq.shape) .. py:method:: compute_sigma_qr(*, hq, hk, rq, rk) .. math:: \sum_{i=1}^{p} \sum_{j=p+1}^{p+q} (h_i r_j - h_j r_i) e_i e_j results = [] sigma_pq = torch.zeros(b, r, p, q) for i in range(p): for j in range(q): sigma_pq[:, :, i, j] = hp[:, :, i] * rq[:, :, j] - hq[:, :, j] * rp[:, :, i] print(sigma_pq.shape)