JDDCfeel free

JDDCJD Dialog Challenge, 2018430201810184

	feel free
	4
	0.744771
	17
	0.254222

./preliminary

./final

[email protected]

TFIDFTF-IDF (Term Frequency-Inverse Document Frequency) TF-IDFNLPTF-IDFtop 10

TF-IDFTF

IDF

1323Q1A1Q2A2Q3+A3

tfidf1tri-grams2top10A

1

1BM252word2vec3LSItfidf

BM25TFIDF0.3seq2seqTransformerseq2seq+attention+dropout+beam search0.56~0.6transformer+beam search0.7

contextcontextcontextQn-1Qn+AnQA10%Q2Q3+A3QAQAAA[3, 200]QA

transformer+beam searchTransformer state-of-the-art

2Transformer3

EncoderDecoderEncoder6Multi-Head AttentionFeed Forward Neural Networkresidual connectionlayer normalizationsub-layer512Decoder6Multi-Head AttentionFeed Forward Neural NetworkMasked Multi-Head Attentionresidual connectionlayer normalizationAttentionTransformerinput embeddingoutput embeddingpositional encoding

Transformer CNNsRNNsattentionTransformerTensor2Tensor (https://github.com/tensorflow/tensor2tensor) Tensor2TensorTransformer450QAbeam searchsize460.01

Q1A1Q2A2Q3A3QA11

QAQ1A1Q2A2Q3A3

TFIDFA3top10top10A3

10deltaBleuDQNbleuDQN10010top10deltaBleu2

3

tfidf_baselinebaseline

contextcontextcontext100

15015150contextA3

context

contextAcontextcontext

AcontextB12Bcontext43B

B****CcontextB

1context

A	context
B	context
C	context

contextCQ2Q3+A3

1seq2seq2transformer3BM25TFIDF

0.30.6

seq2seqSequence to Sequence Learning with Neural Networks0.5seq2seq50.56seq2seq

seq2seqTransformer2017WMTseq2seq

seq2seqTransformerTransformerseq2seqQATransformerAttention Is All You Needbase modelQA1200.61QA450Attention Is All You Needbig modelTransformer0.69Transformer20checkpoints5checkpointsaverage0.73beam size0.74

---

1baseline

2context

3

4

5

6checkpointaverage

========================================

End-to-EndSeq2SeqfancyEnd-to-End Pipeline (NLU-DM-NLG)

NLPNLPNER

** **

1. http://www.wildml.com/2016/07/deep-learning-for-chatbots-2-retrieval-based-model-tensorflow/

2. GALLEY M, BROCKETT C, SORDONI A. deltaBLEU: A Discriminative Metric for Generation Tasks with Intrinsically Diverse Targets[J]. arXiv:1506.06863 [cs], 2015.

3. VASWANI A, SHAZEER N, PARMAR N. Attention Is All You Need[J]. arXiv:1706.03762 [cs], 2017.

4. VASWANI A, BENGIO S, BREVDO E. Tensor2Tensor for Neural Machine Translation[J]. arXiv:1803.07416 [cs, stat], 2018.

5. BORDES A, BOUREAU Y-L, WESTON J. Learning End-to-End Goal-Oriented Dialog[J]. arXiv:1605.07683 [cs], 2016.

6. https://github.com/IBM/pytorch-seq2seq

7. https://github.com/tensorflow/tensor2tensor

8. SUTSKEVER I, VINYALS O, LE Q V. Sequence to Sequence Learning with Neural Networks[J]. arXiv:1409.3215 [cs], 2014.

9. QIU M, LI F-L, WANG S. AliMe Chat: A Sequence to Sequence and Rerank based Chatbot Engine[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Vancouver, Canada: Association for Computational Linguistics, 2017: 498503.