Skip to content
This repository has been archived by the owner on Nov 7, 2020. It is now read-only.

Latest commit

 

History

History
143 lines (111 loc) · 5.29 KB

README.md

File metadata and controls

143 lines (111 loc) · 5.29 KB
Raw

knowledge-graph-embeddings

Python Implementations of Embedding-based methods for Knowledge Base Completion tasks, mainly inspired by scikit-kge and complex.

List of methods

  • RESCAL [Nickel+. 2011]
  • TransE [Bordes+. 2013]
  • DistMult [Yang+. 2015]
  • HolE [Nicklel+. 2016]
    • This model is equivalent to ComplEx[Hayashi and Shimbo. 2018], and the computation cost of ComplEx is lower than of HolE.
  • ComplEx [Trouillon+. 2016]
  • ANALOGY [Liu+. 2017]
    • This model can be regarded as a hybrid between DistMult and ComplEx.

Run to train and test

For training...

▶  python train.py -h
usage: Link prediction models [-h] [--mode MODE] [--ent ENT] [--rel REL]
                              [--train TRAIN] [--valid VALID]
                              [--method METHOD] [--epoch EPOCH]
                              [--batch BATCH] [--lr LR] [--dim DIM]
                              [--margin MARGIN] [--negative NEGATIVE]
                              [--opt OPT] [--l2_reg L2_REG]
                              [--gradclip GRADCLIP] [--save_step SAVE_STEP]
                              [--cp_ratio CP_RATIO] [--metric METRIC]
                              [--nbest NBEST] [--filtered]
                              [--graphall GRAPHALL] [--log LOG]

optional arguments:
  -h, --help            show this help message and exit
  --mode MODE           training mode ["pairwise", "single"]
  --ent ENT             entity list
  --rel REL             relation list
  --train TRAIN         training data
  --valid VALID         validation data
  --method METHOD       method ["complex", "distmult", "transe", "hole",
                        "rescal", "analogy"]
  --epoch EPOCH         number of epochs
  --batch BATCH         batch size
  --lr LR               learning rate
  --dim DIM             dimension of embeddings
  --margin MARGIN       margin in max-margin loss for pairwise training
  --negative NEGATIVE   number of negative samples for pairwise training
  --opt OPT             optimizer ["sgd", "adagrad"]
  --l2_reg L2_REG       L2 regularization
  --gradclip GRADCLIP   gradient clipping
  --save_step SAVE_STEP
                        epoch step for saving model
  --cp_ratio CP_RATIO   ratio of complex's dimention in ANALOGY
  --metric METRIC       evaluation metrics ["mrr", "hits"]
  --nbest NBEST         n-best for hits metric
  --filtered            use filtered metric
  --graphall GRAPHALL   all graph file for filtered evaluation
  --log LOG             output log dir

For testing...

▶  python test.py -h
usage: Link prediction models [-h] [--ent ENT] [--rel REL] [--data DATA]
                              [--filtered] [--graphall GRAPHALL]
                              [--method METHOD] [--model MODEL]

optional arguments:
  -h, --help           show this help message and exit
  --ent ENT            entity list
  --rel REL            relation list
  --data DATA          test data
  --filtered           use filtered metric
  --graphall GRAPHALL  all graph file for filtered evaluation
  --method METHOD      method ["complex", "distmult", "transe", "hole",
                       "rescal", "analogy"]
  --model MODEL        trained model path

Experiments

WordNet (WN18)

Models MRR (flt) MRR (raw) Hits@1 (flt) Hits@3 (flt) Hits@10 (flt)
ComplEx* 94.1 58.7 93.6 94.5 94.7
ComplEx 94.3 58.2 94.0 94.6 94.8

hyper parameters

  • mode : single
  • epoch : 500
  • batch : 128
  • lr : 0.05
  • dim : 200
  • negative : 5
  • opt : adagrad
  • l2_reg : 0.001
  • gradclip : 5

FreeBase (FB15k)

Models MRR (flt) MRR (raw) Hits@1 (flt) Hits@3 (flt) Hits@10 (flt)
ComplEx* 69.2 24.2 59.9 75.9 84.0
ComplEx 69.5 24.2 59.8 76.9 85.0

hyper parameters

  • mode : single
  • epoch : 500
  • batch : 128
  • lr : 0.05
  • dim : 200
  • negative : 10
  • opt : adagrad
  • l2_reg : 0.0001
  • gradclip : 5

* means the results reported from the original papers

Dependencies

  • numpy
  • scipy

References

  • Bordes, A.; Usunier, N.; Garcia-Duran, A.; Weston, J.; and Yakhnenko, O. 2013. Translating embeddings for modeling multi-relational data. In Advances in Neural Information Processing Systems (NIPS).

  • Liu, H.; Wu, Y.; and Yang, Y. 2017. Analogical inference for multi-relational embeddings. In Proceedings of the 34th International Conference on Machine Learning (ICML).

  • Nickel, M.; Rosasco, L.; and Poggio, T. 2016. Holographic embeddings of knowledge graphs. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI’16.

  • Nickel, M.; Tresp, V.; and Kriegel, H.-P. 2011. A threeway model for collective learning on multi-relational data. In International Conference on Machine Learning (ICML-11), ICML ’11,

  • Trouillon, T.; Welbl, J.; Riedel, S.; Gaussier, E.; and Bouchard, G. 2016. Complex embeddings for simple link prediction. In International Conference on Machine Learning (ICML).

  • Yang, B.; Yih, W.; He, X.; Gao, J.; and Deng, L. 2015. Embedding entities and relations for learning and inference in knowledge bases. International Conference on Learning Representations 2015.