Skip to content

iofu728/PaperRead

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

212 Commits
notes
notes
 
 
paper
paper
 
 
.gitignore
.gitignore
 
 
README.md
README.md
 
 

Repository files navigation

🤧Some Paper Read By @gunjianpan

This is a repository like 'douban' personal home page in ML & System field.

❱❱❱❱❱❱❱ ❱❱❱❱❱❱ ❱❱❱❱❱ ❱❱❱❱ ❱❱❱❱ ❱❱❱ ❱❱ ❱

Categories

NLP

Information Extraction

Read Public Conference Title HighLight Code Other
200614 200429 ACL 2020 CGEXpan Expand entity set using MLM CGExpan -

Relation Extraction

Read Public Conference Title HighLight Code Other
200612 200605 - Relation of Relation RoRs RoE -
200116 200106 AAAI 2020 Improve EL via latent Embedding improve the embedding - -
191216 191128 AAAI 2020 Inducing Relation Induing Realtion using cloze - -
200108 190904 ACL 2019 RE view as Multi-turn MRC Multi-turn like RL, MRC MRC -
200115 190721 NLE 2019 25 years IE survey IE in past 25years - -
191128 190607 ACL 2019 Matching the Blanks How to use BERT in relation link BERTem -
200909 190225 NAACL 2019 RE using explicit context second order - -

NER

Read Public Conference Title HighLight Code Other
200424 200416 ACL 2020 Trigger NER Template Trigger TriggerNER -
200331 191112 AAAI 2020 Attention in NER XOR Limitation in BiLSTM cross -
191113 191110 - TENER improve Transformer in NER - -
191020 190712 AAAI 2019 GRN +Long-term->CNN GRN -
191220 190327 EMNLP 2019 capitalized capitalized in ner and pos - -
200107 170613 CONLL 2017 zero-shot MRC view NER as MRC - -
Nested NER
Read Public Conference Title HighLight Code Other
200923 200710 ACL 2020 Pyramid layer like pyramid - -
200517 200514 ACL 2020 NERviaDP leaned N*N biaffine -
200513 200510 AAAI 2020 Boundary-enhance Boundary Info - -
200605 200501 ACL 2020 Bipartite Flat-Graph NER outmost->inmost - -
200924 200429 ACL 2020 Instance-Based NER KNN insurance-ner -
200606 200428 ACL 2020 Translation Discontinuous Discontinuous discontinuous-ner -
200427 200424 ACL 2020 FLAT Position + Lattice - Chinese
200411 200327 AAAI 2020 RBM subEntity - BiLSTM base
200223 191120 EMNLP 2019 Boundary-aware Boundary + Classify boundary -
200223 191120 BioNLP 2019 ost Recursive NER ost -
200223 191110 - KGNER MAQA + Knowledge - -
191101 191028 ACL 2020 MRCNER QA gen + Nested QA - -
190916 190910 EMNLP 2019 DyGIE++ Bertology dygiepp -
190919 190905 - Second Best CRF Recursive CRF secondbest optime cost
190912 190903 EMNLP 2019 Combining Spans into Entities Neural Net->CRF disco_em19 Two stage
190920 190824 - Query-based NER ->QA - usePriorInfo
190812 190804 ACL 2019 Linearization Nest NER label -> CONLL-like nested Seq2seq
190703 190630 ACL 2019 Merge and label Two-stage mergeLabel threshold
190703 190620 ACL 2019 Multi-Grained NER Two-stage MGNER centerSearch
190707 190405 NAACL 2019 DyGIE Dynamic span graph DyGIE IE Framework
Few-shot/Zero-shot
Read Public Conference Title HighLight Code Other
200906 200824 - Example-based NER Span + Top k - Few-shot
191006 190926 - CRF-VAEs VAE in NER - Unlabeled
191006 190925 CONLL 2019 UnifiedNETagger Multi-Corpus NewBioNer Unlabeled
Medical
Read Public Conference Title HighLight Code Other
200908 181213 ACL 2019 Joint NER and Assertion Detection Multi-task + Conditional pytorch-conditional-model -

Cross-Lingual

Read Public Conference Title HighLight Code Other
200414 200410 - XTREME 40language multi-task benchmark xtreme -
200224 191114 AAAI 2020 MetaLearningCrossLingualNER direct transfer + meta-learning - -
200611 190904 EMNLP 2019 Unicoder three cross-lingual tasks Unicoder -

Graph

Read Public Conference Title HighLight Code Other
200716 200704 ICML 2020 GCNII residual + identity GCNII -

Knowledge Bases

Read Public Conference Title HighLight Code Other
200708 200704 ACL 2020 EmbedKGQA multi-hop, missingLink pred EmbedKGQA -
200628 200502 - BERT-kNN add IR in kNN-LM - -
200628 200501 ACL 2020 E-BERT inject embedding to PLM - -
200628 200429 ACL 2020 BERTRAM infer embed for rare words bertram -
200416 200415 - EAE Top-K Memory, like ERNIE - -
200714 200406 ACL 2020 R-MeN Deep KB Embedding R-MeN -
200402 200210 - REALM retrieve-then-predict - blog
200306 200210 - K-Adapter Plugin Knowledge - -
200308 191113 - KEPLER L_RE + L_MLM WikiData5M -
200630 191031 EMNLP 2019 KnowBERT entity linker - -
200308 191003 - BERT_MK GATs - -
200405 190926 ICLR 2020 KNN-LM KNN improve LM representation knnlm review
191220 190926 ICLR 2020 WKLM add KB in pretrain - -
200112 190917 - K-BERT Integrated KG to sentence K-BERT -
200308 190909 EMNLP 2019 KnowBERT Alternate learning kb -
200308 190905 - LIBERT lexical - -
200308 190815 - SenseBERT supersense bert-sense -
200629 190602 ACL 2019 KT-BERT inject & align embedding KTNET -
200308 190517 ACL 2019 ERNIE TransE ERNIE -

Language Models

Read Public Conference Title HighLight Code Other
200228 200213 AAAI 2020 LRLM Span level + latent to align lrlm -
191017 191005 - Megatron-LM parallelism & little code change megatron-LM blog
200302 190215 - ContextualWordRepresent word embedding - -
200403 180922 EMNLP 2018 CVT windows+NER train unlabeled data cvt -
200228 170502 - NKLM Knowledge + LM - review

Piece

Read Public Conference Title HighLight Code Other
200314 190907 - BBPE UTF-8 -> BPE - -

Bertology

Read Public Conference Title HighLight Code Other
200611 200610 - MCBERT Hard ELECTRA(Multi-choice) MCBERT -
200421 200420 - MPNet split predict and no-predict MPNet -
200704 200228 ICML 2020 UniLMv2 Pseudo-Masked unilm -
191112 191029 - BART Auto-Encode + Auto-Regressive - -
191024 191024 - T5 Decathlon T5 C4
200318 190926 ICLR 2020 REFORMER LSH Mask Attention trax review
191113 190926 ICLR 2020 ELECTRA GAN -> difficult MASK - -
190928 190926 ICLR 2020 ALBert ReduceParams - -
190801 190508 NeurIPS 2019 UniLM three Mask unilm -
Bert Distilled
Read Public Conference Title HighLight Code Other
200706 200414 ACL 2020 MobileBERT task-agnostic low latency mobile-bert review
200409 200404 ACL 2020 TinyMBERT LL RL CE -> BiLSTM - -
200717 200525 - BERT of Theseus theseus to teach students BERT-of-Theseus -
191018 191002 NeurIPS 2019 DistillBert support device transformers -
191228 190926 ICLR 2020 DEFINE reduce embed size + residual - -
191002 190926 - TinyBert new tew-stage - -
Bert Probe
Read Public Conference Title HighLight Code Other
200708 200509 ACL 2020 Intermediate-Task which intermediate task good - -
200627 200623 ACL 2020 Climbing towards NLU metaphysical(meaning/linguistic) - -
200628 200227 - Primer in BERTology What knowledge does BERT have - -
191216 190904 EMNLP 2019 LM as KB? Bert in Relation Extraction LAMA -
200320 190624 ACL 2019 RightForTheWrongReasons bad in some anti-heuristic sample - -
191014 190515 ACL 2019 Bert Rediscover probe bert layer - -
Bert DownStream
Read Public Conference Title HighLight Code Other
200113 191031 - DuoBERT Three-stage Recall-Rank(pairwise) DuoBERT -
191020 190827 EMNLP 2019 Sentence Bert Bert in STS sentence-transformers -
Multi-modality Bert
Read Public Conference Title HighLight Code Other
200924 200917 - GraphCodeBERT AST Edge Pred + Node Align - -
200221 200219 - CodeBERT NL + PL - -
Prompt
Read Public Conference Title HighLight Code Other
210722 210406 NAACL 2021 HowManyDataPointsIsAPromptWorth head vs prompt in Few-shot pet -

Representation

Read Public Conference Title HighLight Code Other
210421 210420 - SimCSE Dropout, Contrastive Learning SimCSE -
Document-level Representation
Read Public Conference Title HighLight Code Other
200418 200415 ACL 2020 Citation-informed Triplet Loss Negative base on citation - -

Tune

Read Public Conference Title HighLight Code Other
200420 200224 - Self-Distillation study gold + self-ensemble - -
200308 190515 ICML 2019 PALs Project Att Bert-n-Pals -
200308 190113 ICML 2019 Adapter-BERT Fine-tune => Adapter adapter-bert -

NLG

Read Public Conference Title HighLight Code Other
200303 190926 ICLR 2020 Nucleus Sampling Top-p Sampling gpt2 review
200302 190926 ICLR 2020 D2GPo negative influence
-> prior Gaussian		 - -
191005 190926 ICLR 2020 BertScore auto evaluation use bert bertScore -
190926 190926 - CTRL controllable generation ctrl -
191010 190812 - UnLikelihood unLikelihood -

Summarization

Read Public Conference Title HighLight Code Other
200313 190708 ACL 2019 ExtractiveSummarization Diff Architecture Extractive Summarization fastNLP homepage
181212 170725 ICML 2017 ConvS2S CNN semantic fairseq notes

NMT

Read Public Conference Title HighLight Code Other
200626 200618 - DeeperEncoder,ShallowDecoder reduce decoder layer - -
200629 191110 ACL 2020 Sandwich improve the FFN & MHA order sandwich -
200113 171107 ICLR 2018 No-autoregressive NMT fertilities to auxiliary parallel nonauto-nmt -

ChatBot

Read Public Conference Title HighLight Code Other
200401 200227 - Meena Evolved Transformer + Billion - blog

QA

MRC

Read Public Conference Title HighLight Code Other
200623 200602 - Pre-Constructed reader-retriever - -
200623 200508 ACL 2002 BoundaryDetectionMRC mixMRC + LAKM - -
200316 191028 - Multiple-Choice MRC Multi-choice + Probing - -
200609 200502 - DensePassageRetrieval Dense Retrieval qa-dpr -
200405 190627 ACL 2019 ORQA Latent Retriever - -

Dialogue

Read Public Conference Title HighLight Code Other
200728 200505 ACL 2020 ConceptFlow multi-hop in dialogue ConceptFlow -
200415 200409 ACL 2020 MuTual Reason Multi-tune Benchmark MuTual -

Information System

Read Public Conference Title HighLight Code Other
200707 200507 ACL 2020 QUARTS Adversarial improve similarity - -

Other

Text Style Transfer

Read Public Conference Title HighLight Code Other
191006 190925 EMNLP 2019 CrossProject Latent Space Project CrossProject -

Parsing

Read Public Conference Title HighLight Code Other
200622 200610 ACL 2020 Perturbed Masking $d(x\{x_i}, x\{x_i, x_j})$ Perturbed-Masking -
191111 191030 EMNLP 2019 Predict DS using sentiment using sentiment generate DP dataSet - -
200421 190129 CONLL 2018 Universal DP pipeline dp stanfordnlp -
200601 170310 ICLR 2017 Biaffine graph-based stanfordnlp review

Chinese

Read Public Conference Title HighLight Code Other
200714 200513 ACL 2020 SpellGCN phonological + semantic SpellGCN -
191013 190906 - NeZha chinese bert - -

Attack

Read Public Conference Title HighLight Code Other
200717 200414 ACL 2020 Weight Poison Attacks Weight Poison RIPPLe -
190911 190829 EMNLP 2019 Universal Adversarial Triggers Adversarial-NLP universal-triggers blog

Common Sense

Read Public Conference Title HighLight Code Other
191218 190904 EMNLP 2019 KagNet Commonsense via KB + Pretrain KagNet -
200107 190604 ACL 2019 quantities Rule-base Quantity distribution-over-quantities -

ML

Architecture

Transformer

Read Public Conference Title HighLight Code Other
200721 200712 ICML 2020 T-Fixup param initial=>warm up tfixup -
200708 200707 ACL 2020 Long-range Memory? Do need LRM? - -
200426 200410 - Longformer Global + slide Win Att longformer -
191229 191127 - SHA-RNN RNN + Att sha-rnn -
200608 191022 ICLR 2020 LiteTransformer Global + Local lite review
191028 191022 ICLR 2020 Depth-adaptive Transformer depth adaptive - -
190929 190926 - LayerNorm Transformer improve warm-up - -
190716 190710 - Large Memory Layer KeyValue search XLM twitter
190718 190624 NeurIPS 2019 Tensorized Transformer decompose + param share - -
190908 190606 - Macaron Net Strange-Macaron - note
190718 190519 ACL 2019 Adaptive Att span adapting span adaptiveSpan -
191118 190228 NAACL 2019 Star Transformer FFN -> Star Arch fastNLP -
190719 180710 ICLR 2019 Universal Transformer Recurrent + ACT UT slider
Transformer in CV
Read Public Conference Title HighLight Code Other
210729 210407 - CaiT LayerScale + Class Attention deit -
Relative position embedding
Read Public Conference Title HighLight Code Other
210424 210323 - RoFormer Complex domain RoFormer RoPE
200702 200630 ICLR 2021 TUPE reset + untie TUPE -
191229 190926 ICLR 2020 Complex Order Complex WE + PE complex-order explainable
190425 190306 ACL 2019 Transformer-XL Recurrent + Relative PE Transformer-XL reject once
191229 180306 NAACL 2018 Self-Att with Relative PE Relative PE - Transformer Writer

Attention

Read Public Conference Title HighLight Code Other
200613 200513 ACL 2020 h - 1 heads learn heads weightAsexpects MAE -
190819 190607 ACL 2019 Analysis Multi-Head every head role heads -
190619 190226 NAACL 2019 Attention not explanation weighted correction + robust AttentionExplanation -

Normalization

Read Public Conference Title HighLight Code Other
200319 200317 - BN in Transformer high var in NLP, PN - -

Model Compression

Read Public Conference Title HighLight Code Other
210819 171103 ICLR 2018 Code Learning Code-Based code_learning -
210819 180324 AAAI 2019 Binarization AutoEncoder Binarization lossless -
210819 191002 EMNLP 2020 finding DistilledNonlinearCompression Matrix Decompose + KD - -
210819 200801 ACL 2020 Adaptive Compression Embedding Adpative Code-based Embedding Compression - -
211220 210119 AAAI 2021 Compressed Attn Decoder only have two parallel attention - -
210810 201023 NeurIPS 2020 Movement Pruning 1-st pruning when fine-tune nn_pruning blog

Strategy

Metric Learning

Read Public Conference Title HighLight Code Other
190829 190525 CVPR 2020 Circle Loss Unified Npair,Softmax - -
190829 190525 NeurIPS 2019 Constellation Loss Multiclass n + Triple constellation_loss -
191007 170406 CVPR 2017 Quadruplet Loss two margin - -
190827 160601 CVPR 2016 Multi-class N-pair Loss multi negative sample - -
190827 150617 CVPR 2015 Triplet Loss Triple - -

Interpretability

Read Public Conference Title HighLight Code Other
200307 200208 - Memorization-Generalization measure degree of generation - Chiyuan Zhang

Multi-task

Read Public Conference Title HighLight Code Other
200610 190717 AAAI 2019 SNR Flexible connection - -
200330 181006 ECCV 2018 Dynamic Task Prioritization Focal Loss Priori - -
191216 170214 ECCV 2016 Learning without forgetting Multi-task by no old data LwF -

Regularization

Read Public Conference Title HighLight Code Other
200727 200220 ICML 2020 Flood give content loss - -
191228 190926 ICLR 2020 Mixout origin + dropout - -

Optimization

Read Public Conference Title HighLight Code Other
200227 141222 ICLR 2015 Adam momentum +squared gradient Adam -

Negative Sample

Read Public Conference Title HighLight Code Other
200621 200613 - BYOL self-supervised (remove negative) - -
200119 191211 - NegativeSampleInVAEs using VAEs to solve OOD - -
200117 181216 ICDE 2019 NSCaching dynamical probability sampler replace GAN NSCashing -
200117 180923 AAAI 2018 Incorporating GAN in Entity Linking using GAN to generate high quality NS - -
200118 180105 AAAI 2018 VSE-ens dynamical sampler to reduce sample time VSE-ens -
200119 140621 AAAI 2014 Translating2Hyperplanes solve the multi-label problem Knowledge -

Data

Unbalance Classify

Read Public Conference Title HighLight Code Other
191111 191107 - Dice Loss in NLP like F1 FP==FN - -
191011 190116 ICLR 2019 Class-Balance Loss effective number sample googleResearch -

Noisy Labels

Read Public Conference Title HighLight Code Other
191029 181030 NeurIPS 2018 Co-teaching Memorize effect Co-teaching -

Extreme Classification

Read Public Conference Title HighLight Code Other
200104 190926 ICLR 2020 AdversarialSoftmax negative sample + DT AdversarialSoftMax -

Tabular Learning

Read Public Conference Title HighLight Code Other
191006 190926 - TabNet NN method googleResearch interpretable

Applied Data Scrience

Read Public Conference Title HighLight Code Other
200101 190725 KDD 2019 Job Mobility Prediction Data Encoder + evaluate for job - video

Table&Chart

Read Public Conference Title HighLight Code Other
200311 190725 AAAI 2019 Table2Analysis LM + Beam search to get Table Analysis - -
200110 190725 AAAI 2019 TableSense detect Table Boundaries from spreadsheet - -

Demo

Read Public Conference Title HighLight Code Other
200421 200316 - Stanza RawText Toolkit - homepage
191218 191015 ICMLA 2019 ComprehendMedical Medical NER + RE API AWS -
191011 191009 - Transformers Unified API Transformers -

CV

Video Prediction

Read Public Conference Title HighLight Code Other
191005 190926 ICLR 2020 CrevNet information preserving - -

System

Distribution

Read Public Conference Title HighLight Code Other
200310 141101 USENIX ATX 2014 Raft Consensus algorithm - web
200304 101101 SIGOPS 2010 VM-FT Fault-Tolerant replicating - -
200222 040301 OSDI 2004 MapReduce Map + Reduce For distribution cal MapReduce -
200225 041020 SOSP 2003 GFS distribution file system - -

SpeedUp

Read Public Conference Title HighLight Code Other
210720 210422 NAACL 2021 LightSeq SpeedUp Transformer LightSeq -

License

Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)

Copyright (c) 2019-present, gunjianpan(iofu728)

About

📒Record some paper read notes

Resources

Readme
Activity

Stars

20 stars

Watchers

4 watching

Forks

2 forks
Report repository

Releases

No releases published

Packages

No packages published