Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

没有weights/encoder,weights/decoder,weights/attention文件 #1

Open
zysNLP opened this issue Nov 27, 2018 · 33 comments
Open

没有weights/encoder,weights/decoder,weights/attention文件 #1

zysNLP opened this issue Nov 27, 2018 · 33 comments

Comments

@zysNLP
Copy link

zysNLP commented Nov 27, 2018

您好,我在运行过程中发现缺少这三个文件,作者是不是忘记导出这些参数了?谢谢!

@liangstein
Copy link
Owner

权重文件可以用自己的数据集训练。

@zysNLP
Copy link
Author

zysNLP commented Nov 29, 2018

请问是运行哪个文件能训练出权重文件呢?我试了三个python文件好像都不行

@liangstein
Copy link
Owner

有个train的py文件

@zysNLP
Copy link
Author

zysNLP commented Nov 29, 2018

是下面这个train_chatbot.py的最后三行吗?
with open("weights/encoder","wb") as f:
pickle.dump([ele.cpu() for ele in params_encoder],f,protocol=pickle.HIGHEST_PROTOCOL)
with open("weights/decoder","wb") as f:
pickle.dump([ele.cpu() for ele in params_decoder],f,protocol=pickle.HIGHEST_PROTOCOL)
with open("weights/attention", "wb") as f:
pickle.dump([ele.cpu() for ele in params_attention], f, protocol=pickle.HIGHEST_PROTOCOL)

@liangstein
Copy link
Owner

这是把训练好的权重写进硬盘,要开始训练需要有pytorch的环境,然后直接python train_chatbot.py就开始了。

@zysNLP
Copy link
Author

zysNLP commented Nov 29, 2018

运行python train_chatbot.py以后没什么问题,结果:
....
tensor(5.2833, device='cuda:0', grad_fn=)
tensor(5.1880, device='cuda:0', grad_fn=)
tensor(5.3055, device='cuda:0', grad_fn=)
Traceback (most recent call last):
File "train_chatbot.py", line 104, in
with open("weights/encoder","wb") as f:
FileNotFoundError: [Errno 2] No such file or directory: 'weights/encoder'

这意思是让我实现建立一个'weights/encoder'吗,enconder文件后缀用什么

@zysNLP
Copy link
Author

zysNLP commented Nov 29, 2018

原来需要在目录中事先建立一个'weights‘文件就好了。。。建议您在github里添上这个空文件夹吧

@zysNLP
Copy link
Author

zysNLP commented Nov 29, 2018

最后想问下A_no_repeat和Q_no_repeat这两个文件怎么打开呀?需要用什么工具么

@liangstein
Copy link
Owner

用pickle 加载
pickle.load("filename")

@zysNLP
Copy link
Author

zysNLP commented Nov 29, 2018

这跑到什么时候停止啊,现在已经是tensor(1.5055, device='cuda:0', grad_fn=),电脑太烫了

@zysNLP
Copy link
Author

zysNLP commented Nov 29, 2018

老哥能给说下这个到什么时候收敛么,再跑下去我怕电脑热炸了....

@liangstein
Copy link
Owner

loss降到1e-4以下,看losses这个文本文件

@zysNLP
Copy link
Author

zysNLP commented Nov 29, 2018

请问下有什么办法能把loss迅速降到1e-4以下?按照您的算法,跑完这个需要几天时间吧?而且电脑一直特别烫的问题怎么解决。我32G,1080Ti的电脑,跑您这个程序,感觉对电脑有点担忧啊,您跑得时候也这么烫么?

@liangstein
Copy link
Owner

我大概用了两天,1080ti要快些。我用1080平均75度。

@zysNLP
Copy link
Author

zysNLP commented Nov 29, 2018

主板温度到了99度,GTX1080Ti温度87度,package id 0--68度也不知道是什么?主板这么高的温度我怕烤化了。。。

@liangstein
Copy link
Owner

那是你电脑机箱有问题。:)

@zysNLP
Copy link
Author

zysNLP commented Nov 29, 2018

是没安装散热器么

@liangstein
Copy link
Owner

算了 我把model上传吧。
https://www.dropbox.com/s/i8lleyc4h4shx7n/weights.tar?dl=0

@zysNLP
Copy link
Author

zysNLP commented Nov 29, 2018

打不开这个网站,老哥把weights单独发我一份吧,[email protected]

@liangstein
Copy link
Owner

那我没办法了 几百兆我邮箱发不了,打不开就想办法打开。

@zysNLP
Copy link
Author

zysNLP commented Nov 29, 2018

实在打不开,哥你加我qq,然后压缩一下用qq发给我吧,648634000。多谢!!!

@liangstein
Copy link
Owner

@zysNLP
Copy link
Author

zysNLP commented Nov 29, 2018

OK,多谢!

@zysNLP
Copy link
Author

zysNLP commented Nov 29, 2018

最后,如果我改变了训练语料库,得到了比小黄鸡语料更多的语料,是不是还要重新训练得到新的'weights‘?...我让师傅看看机箱吧,感谢!

@liangstein
Copy link
Owner

要的.
不用。

@DavidLanz
Copy link

請問我想使用自己的語料,不是小黃鴨的,請問原始的小黃鴨資料格式為何呢? 大感謝!

@liangstein
Copy link
Owner

需要分词,我是按照字分的,一个字对应一个one-hot编码。
输入格式是[batch_size, time_step, word_dimension]

@DavidLanz
Copy link

請問您 train_chatbot.py裡,word_index_dict與index_word_dict這兩個變量是怎麼產生的呢?

@liangstein liangstein reopened this Feb 18, 2019
@liangstein
Copy link
Owner

这是通过count每个字出现的次数统计得到的。

@DavidLanz
Copy link

謝謝您,請問是在訓練集每句話 {問句+答句} 的全部的統計結果嗎?

比方說「鸡」,他的key是17,即: word_index_dict['鸡'] => 17,是重複了17次對嗎,不知道理解有沒有錯誤?

for k,v in enumerate(word_index_dict, start=0):
print(("k=%s, v=%s") % (k,v))

@liangstein
Copy link
Owner

是在整个训练集上的字数统计,包含了问和答。

你的理解是对的,我是这么做的,你也可以分词,然后统计词出现的频率。

@DavidLanz
Copy link

我使用了jieba分詞,無法變成您的單"字",以下是我的程式,方便參考您的作法嗎?

import os, codecs
import jieba
from collections import Counter

def get_words(txt):
seg_list = jieba.cut(txt)
c = Counter()
for x in seg_list:
if len(x)>1 and x != '\r\n':
c[x] += 1
print('常用词频度统计结果')
for (k,v) in c.most_common(100):
print('%s%s %s %d' % (' '(5-len(k)), k, ''*int(v/3), v))

if name == 'main':
with codecs.open('courps.txt', 'r', encoding='utf-8') as f:
txt = f.read()
get_words(txt)

@liangstein
Copy link
Owner

这样做应该可以得到词的统计。
我是没有用分词,直接按字统计出现次数的。如果要变成字,去掉jieba.cut就可以了。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants