在神經網絡中實現反向傳播

發布于 2024-3-27 15:40

瀏覽

0收藏

建立神經網絡時，需要采取幾個步驟。其中兩個最重要的步驟是實現正向和反向傳播。這兩個詞聽起來真的很沉重，并且總是讓初學者感到恐懼。但實際上，如果將這些技術分解為各自的步驟，則可以正確理解它們。在本文中，我們將專注于反向傳播及其每個步驟的直觀知識。

什么是反向傳播?

這只是實現神經網絡的一項簡單技術，允許我們計算參數的梯度，以執行梯度下降并使成本函數最小化。許多學者將反向傳播描述為神經網絡中數學上最密集的部分。不過請放輕松，因為在本文中我們將完全解密反向傳播的每個部分。

在神經網絡中實現反向傳播-AI.x社區

實施反向傳播

假設一個簡單的兩層神經網絡-一個隱藏層和一個輸出層。我們可以如下執行反向傳播初始化要用于神經網絡的權重和偏差：這涉及隨機初始化神經網絡的權重和偏差。這些參數的梯度將從反向傳播中獲得，并用于更新梯度下降。


#Import Numpy library
import numpy as np

#set seed for reproducability 
np.random.seed(100)
#We will first initialize the weights and bias needed and store them in a dictionary called W_B
def initialize(num_f, num_h, num_out):
    
    '''
    Description: This function randomly initializes the weights and biases of each layer of the neural network
    
    Input Arguments:
    num_f - number of training features
    num_h -the number of nodes in the hidden layers
    num_out - the number of nodes in the output 
    
    Output: 
    
    W_B - A dictionary of the initialized parameters.
    
    '''
    
    #randomly initialize weights and biases, and proceed to store in a dictionary
    W_B = {
        'W1': np.random.randn(num_h, num_f),
        'b1': np.zeros((num_h, 1)),
        'W2': np.random.randn(num_out, num_h),
        'b2': np.zeros((num_out, 1))
    }
    return W_B

執行前向傳播：這涉及到計算隱藏層和輸出層的線性和激活輸出。

對于隱藏層：我們將使用如下所示的relu激活功能：


#We will now proceed to create functions for each of our activation functions

def relu (Z):
    
    '''
    Description: This function performs the relu activation function on a given number or matrix. 
    
    Input Arguments:
    Z - matrix or integer
    
    Output: 
    
   relu_Z -  matrix or integer with relu performed on it
    
    '''
    relu_Z = np.maximum(Z,0)
    
    return relu_Z

對于輸出層：

我們將使用S型激活函數，如下所示：


def sigmoid (Z):
    
    '''
    Description: This function performs the sigmoid activation function on a given number or matrix. 
    
    Input Arguments:
    Z - matrix or integer
    
    Output: 
    
   sigmoid_Z -  matrix or integer with sigmoid performed on it
    
    '''
    sigmoid_Z = 1 / (1 + (np.exp(-Z)))
    
    return sigmoid_Z

執行前向傳播：


#We will now proceed to perform forward propagation

def forward_propagation(X, W_B):    
    '''
    Description: This function performs the forward propagation in a vectorized form 
    
    Input Arguments:
    X - input training examples
    W_B - initialized weights and biases
    
    Output: 
    
   forward_results - A dictionary containing the linear and activation outputs
    
    '''
    
    #Calculate the linear Z for the hidden layer
    Z1 = np.dot(X, W_B['W1'].T)  + W_B['b1']
    
    #Calculate the activation ouput for the hidden layer
    A = relu(Z1)
    
    #Calculate the linear Z for the output layer
    Z2 = np.dot(A, W_B['W2'].T) + W_B['b2']
    
    #Calculate the activation ouput for the ouptu layer
    Y_pred = sigmoid(Z2) 
    
    #Save all ina dictionary 
    forward_results = {"Z1": Z1,
                      "A": A,
                      "Z2": Z2,
                      "Y_pred": Y_pred}
    
    return forward_results

執行向后傳播：相對于與梯度下降相關的參數，計算成本的梯度。在這種情況下，為dLdZ2，dLdW2，dLdb2，dLdZ1，dLdW1和dLdb1。這些參數將與學習率結合起來執行梯度下降。我們將為許多訓練樣本(no_examples)實現反向傳播的矢量化版本。

分步指南如下：

從傳遞中獲取結果，如下所示：

forward_results = forward_propagation(X, W_B)
Z1 = forward_results['Z1']
A = forward_results['A']
Z2 = forward_results['Z2']
Y_pred = forward_results['Y_pred']

獲得訓練樣本的數量，如下所示：

no_examples = X.shape[1]

計算函數的損失：

L = (1/no_examples) * np.sum(-Y_true * np.log(Y_pred) - (1 - Y_true) * np.log(1 - Y_pred))

計算每個參數的梯度，如下所示：

dLdZ2= Y_pred - Y_true
dLdW2 = (1/no_examples) * np.dot(dLdZ2, A.T)
dLdb2 = (1/no_examples) * np.sum(dLdZ2, axis=1, keepdims=True)
dLdZ1 = np.multiply(np.dot(W_B['W2'].T, dLdZ2), (1 - np.power(A, 2)))
dLdW1 = (1/no_examples) * np.dot(dLdZ1, X.T)
dLdb1 = (1/no_examples) * np.sum(dLdZ1, axis=1, keepdims=True)

將梯度下降所需的計算梯度存儲在字典中：

gradients = {"dLdW1": dLdW1,
             "dLdb1": dLdb1,
             "dLdW2": dLdW2,
             "dLdb2": dLdb2}

返回損耗和存儲的梯度：

return gradients, L

這是完整的向后傳播功能：


def backward_propagation(X, W_B, Y_true):
    '''Description: This function performs the backward propagation in a vectorized form 
    
    Input Arguments:
    X - input training examples
    W_B - initialized weights and biases
    Y_True - the true target values of the training examples
    
    Output: 
    
    gradients - the calculated gradients of each parameter
    L - the loss function
    
    '''
    
    # Obtain the forward results from the forward propagation 
    
    forward_results = forward_propagation(X, W_B)
    Z1 = forward_results['Z1']
    A = forward_results['A']
    Z2 = forward_results['Z2']
    Y_pred = forward_results['Y_pred']
    
    #Obtain the number of training samples    
    no_examples = X.shape[1]
    
    # Calculate loss 
    L = (1/no_examples) * np.sum(-Y_true * np.log(Y_pred) - (1 - Y_true) * np.log(1 - Y_pred))
    
    #Calculate the gradients of each parameter needed for gradient descent 
    dLdZ2= Y_pred - Y_true
    dLdW2 = (1/no_examples) * np.dot(dLdZ2, A.T)
    dLdb2 = (1/no_examples) * np.sum(dLdZ2, axis=1, keepdims=True)
    dLdZ1 = np.multiply(np.dot(W_B['W2'].T, dLdZ2), (1 - np.power(A, 2)))
    dLdW1 = (1/no_examples) * np.dot(dLdZ1, X.T)
    dLdb1 = (1/no_examples) * np.sum(dLdZ1, axis=1, keepdims=True)
    
    #Store gradients for gradient descent in a dictionary 
    gradients = {"dLdW1": dLdW1,
             "dLdb1": dLdb1,
             "dLdW2": dLdW2,
             "dLdb2": dLdb2}
    
    return gradients, L

許多人總是認為反向傳播很困難，但是正如本文中介紹的情形，事實并非如此。必須掌握每個步驟，才能掌握整個反向傳播技術。另外，有必要掌握線性代數和微積分等數學知識，以了解如何計算每個函數的各個梯度。使用這些工具，反向傳播應該是小菜一碟!實際上，反向傳播通常由使用的深度學習框架來處理。但是，了解這種技術的內在作用是值得的，因為它有時可以幫助我們理解神經網絡為何訓練得不好。

本文轉載 ??小白遇見AI?? ，作者：小煩

原文鏈接：??https://mp.weixin.qq.com/s/vx2lqz5o8JchPr226lC9cA??

標簽

神經網絡

反向傳播

已于2024-3-27 16:09:56修改

贊

回復