Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

grad_parameters函数输出结果w1的值好像有点异常 #3

Open
yyl2016000 opened this issue Jun 27, 2019 · 2 comments
Open

grad_parameters函数输出结果w1的值好像有点异常 #3

yyl2016000 opened this issue Jun 27, 2019 · 2 comments

Comments

@yyl2016000
Copy link

在ANN_pre_1.ipynb文件的中In [27]这个块里。执行之后得出来的关于w1的梯度值都为0,这样的话不是无论训练多少次后w1参数的值都不会变化么?但是B站的视频中在执行这一块代码块的时候w1的值好像是正常的,所以一开始我以为是我的代码问题,但是来github上看了以后发现这里的代码有着一样的问题,请问原因是什么呢?好像因为这一步的关系导致训练的效率很低,而且多次训练后最终的准确率也只能在百分之八十五左右。希望能得到解答,十分感谢。
批注 2019-06-27 211016

@YQGong
Copy link
Owner

YQGong commented Sep 1, 2019

你可以运行一下这一句:
grad_parameters(train_img[2],train_lab[2],init_parameters())['w1'].max()
就会发现其实w1的值不全是0,只是它的边边角角位置都为0罢了。

至于为什么它边角都是零,需要看它的计算过程:
grad_w1=-2*np.outer(l0_out,act1)
其中的l0_out是图片+b0,其中图片的边边角角都是黑色的,因此都是0,而b0的初始化是全为0的,因此l0_out的边边角角也都是0,用外积算出来的结果边边角角也都是0。

@saaavsaaa
Copy link

准确程度这里受学习率影响很大,可以试试0.01左右调调,我第三个 epoch 就 0.9 了

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants