Deep learning II - I Practical aspects of deep learning - Vanishing/Exploring gradients 梯度消失/爆炸
Vanishing/Exploring gradients 梯度消失/爆炸 如图所示，由于深度神经网络深度较大，当每一层的 w w 都小于1或者都大于1时，最终的输出值会成指数级较小或上升，造成梯度消失或者爆炸，从而使得梯度下降变得困难 1. 为了不让 z z 太大，当 n n 越大时， w w 的值应该越小 2. 针对 ReLU R e L U ，初始化 w[l]=np.random.randn
相关文章
- 1. Deep learning II - I Practical aspects of deep learning - Normalizing inputs 输入归一化
- 2. Deep learning II - I Practical aspects of deep learning - Understanding dropout 理解随机失活正则化
- 3. Deep learning II - I Practical aspects of deep learning - Dropout regularization 随机失活正则化
- 4. 007 Practical aspects of Deep Learning
- 5. Deep learning II - I Practical aspects of deep learning - 正则化如何较少过拟合
- 6. Deep learning II - I Practical aspects of deep learning - other regularization methods 其他正则化方法
- 7. [coursera/ImprovingDL/week1]Practical aspects of Deep Learning(summary&question)
- 8. 【吴恩达深度学习笔记】2.1 Practical aspects of Deep Learning
- 9. 梯度爆炸（Exploding Gradients）
- 10. Deep learning II - I Practical aspects of deep learning - Regularizing your neural network 神经网络范数正则化
- 更多相关文章...
- • Lua 文件 I/O - Lua 教程
- • HTTP 消息结构 - HTTP 教程
- • 算法总结-广度优先算法
- • 算法总结-深度优先算法
相关标签/搜索
每日一句
-
每一个你不满意的现在，都有一个你没有努力的曾经。
欢迎关注本站公众号,获取更多信息
相关文章
- 1. Deep learning II - I Practical aspects of deep learning - Normalizing inputs 输入归一化
- 2. Deep learning II - I Practical aspects of deep learning - Understanding dropout 理解随机失活正则化
- 3. Deep learning II - I Practical aspects of deep learning - Dropout regularization 随机失活正则化
- 4. 007 Practical aspects of Deep Learning
- 5. Deep learning II - I Practical aspects of deep learning - 正则化如何较少过拟合
- 6. Deep learning II - I Practical aspects of deep learning - other regularization methods 其他正则化方法
- 7. [coursera/ImprovingDL/week1]Practical aspects of Deep Learning(summary&question)
- 8. 【吴恩达深度学习笔记】2.1 Practical aspects of Deep Learning
- 9. 梯度爆炸（Exploding Gradients）
- 10. Deep learning II - I Practical aspects of deep learning - Regularizing your neural network 神经网络范数正则化