Study deep arning since November 30, 2020.This study note was last updated on February 21, 2021.2.21

Part1 Introduction to deep learning

  • Deep Learning Notes (1) Introduction to deep learning

Part2 Basic neural network

  • Deep learning Notes (2) Logistic Regression
  • Deep learning Notes (3) Calculation graph and its derivative operation method
  • Notes on deep learning (iv) Vectorization Vectorization
  • Deep learning notes (5)
  • Ng deep learning programming exercises: Logistic regression with Neural Network Thinking

Part3 Shallow neural network

  • Deep learning notes (6) Representation and output of shallow neural networks
  • Deep learning notes (7) Activation functions of shallow neural networks
  • Deep learning notes (8) Neural network back propagation gradient descent algorithm
  • Random initialization parameters and parameter VS hyperparameter
  • Deep learning programming exercise: Planar Data classification with One hidden Layer

Part4 Deep neural network

  • Deep learning notes (10) Representation and forward propagation of deep neural networks
  • Check the dimensions of matrix (Determine the essence of matrix dimensions)
  • Why Deep Neural Networks and not shallow ones?
  • Building deep neural network blocks and forward and back propagation process
  • [Deep learning] Manual construction of DNN model of deep neural network

Part5: improving deep neural networks: hyperparametric debugging, regularization, and optimization

  • Deep learning Notes (14) Data set and deviation variance
  • Deep Learning Framework and TensorFlow
  • Deep Learning Notes (16) Regularization (L2 Dropout Data Amplification)
  • Deep learning notes (17) Normalized input
  • Deep learning Notes (18) Gradient disappearance/explosion, initialization weights, and gradient checking
  • Mini-batch Gradient descent method and exponential weighted average
  • Momentum Gradient Descent and RSMprop Adam Optimization Algorithm
  • Deep learning notes (21) Learning rate decay and local optimization problems
  • Deep learning notes (22) Hyperparameter debugging processing
  • Deep learning Notes (23) Batch Norm
  • Softmax regression
  • DNN’s Softmax classification realizes gesture image recognition

Part6 Structured machine learning project

  • Deep Learning Notes (25) Machine learning Strategies for Structured Machine learning Projects 1
  • Deep Learning Notes (26) Machine learning Strategies for structured machine learning projects 2
  • Notes on deep Learning (27) Transfer learning and multi-task learning
  • End – to – end deep learning

Part7 Convolutional neural network

  • Deep Learning Notes (29) Convolution operations and edge detection in convolutional Neural networks
  • Deep learning notes (30) CNN Padding and convolution step length Stride
  • Deep Learning Notes (31) Three-dimensional convolution and convolutional Neural Networks
  • Deep learning Notes (32) CNN Pooling layer
  • Deep Learning Notes (33) Convolutional Neural Network recognition of handwritten numbers
  • Classical convolutional Neural Networks: Lenet-5 AlexNet VGG-16
  • Deep Learning Notes (35) Residual Neural Network ResNet
  • Deep Learning Notes (36) 1×1 convolution (Network of Networks) and Google Inception Networks
  • Notes on deep Learning (37) Transfer learning and data amplification
  • Deep Learning notes (38) Object detection and YOLO algorithm
  • Convolutional neural network realizes gesture image recognition

  • So, a little summary! It is the end of the winter vacation in 2021, that is, on February 21, 2021.I began to learn the first lesson on November 30, 2020.I wrote down my first deep learning notes. By the time of this note, target detection has been finished, which is what I need to know in my undergraduate study, probably computer vision. This does not mean that my deep learning will stop here. I will continue to learn the knowledge of deep learning system, including the circulating neural network RNN that may be used in NLP later. According to the current schedule, I should take these theoretical knowledge with me to apply and practice extensively, including the demo prepared by Mr. Enda Ng and the target detection by deep learning in the comprehensive design (child monitoring system).

  • In fact, I know that to complete the comprehensive design project, I can read the model code carefully, refer to the data and debug, but I put the practice section behind, for me, the higher priority should be to systematically learn the theory of deep learning (because I am really curious about this field). It includes the second and third courses (neural network improvement strategy and machine learning strategy) which seem to be the most boring, but in fact, they can greatly cultivate my way of thinking in the direction of deep learning.

  • Again, according to my own schedule, I will take the theoretical knowledge I have learned to practice a lot and implement these theories. I may have a deeper insight into these theoretical knowledge when I look back later. My passion for machine learning goes beyond this, and I look forward to having a real understanding of this field in the future!

  • Finally, I would like to thank Mr. Ng and myself for Never stopping