양지우 8주차 과제 제출합니다. #114

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open

didwldn3032 wants to merge 1 commit into Bitamin9:main from didwldn3032:main

8주차(1102)/2조/양지우/Building_a_Recurrent_Neural_Network_Step_by_Step.ipynb

Large diffs are not rendered by default.

8주차(1102)/2조/양지우/images/LSTM.png

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

8주차(1102)/2조/양지우/images/LSTM_cell_backward_rev3a_5.png

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

8주차(1102)/2조/양지우/images/LSTM_cell_backward_rev3a_c2.png

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

8주차(1102)/2조/양지우/images/LSTM_figure4_v3a.png

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

8주차(1102)/2조/양지우/images/LSTM_rnn.png

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

8주차(1102)/2조/양지우/images/initial_state.png

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

8주차(1102)/2조/양지우/images/rnn.png

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

8주차(1102)/2조/양지우/images/rnn_backward_overview_3a_1.png

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

8주차(1102)/2조/양지우/images/rnn_cell_backprop.png

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

8주차(1102)/2조/양지우/images/rnn_cell_backward_3a_4.png

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

8주차(1102)/2조/양지우/images/rnn_cell_backward_3a_c.png

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

8주차(1102)/2조/양지우/images/rnn_forward_sequence_figure3_v3a.png

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

8주차(1102)/2조/양지우/images/rnn_step_forward.png

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

8주차(1102)/2조/양지우/images/rnn_step_forward_figure2_v3a.png

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

8주차(1102)/2조/양지우/rnn_utils.py

-Original file line number
+Diff line change
@@ -0,0 +1,110 @@
+    import numpy as np
+    def softmax(x):
+        e_x = np.exp(x - np.max(x))
+        return e_x / e_x.sum(axis=0)
+    def sigmoid(x):
+        return 1 / (1 + np.exp(-x))
+    def initialize_adam(parameters) :
+        """
+        Initializes v and s as two python dictionaries with:
+                    - keys: "dW1", "db1", ..., "dWL", "dbL"
+                    - values: numpy arrays of zeros of the same shape as the corresponding gradients/parameters.
+        Arguments:
+        parameters -- python dictionary containing your parameters.
+                        parameters["W" + str(l)] = Wl
+                        parameters["b" + str(l)] = bl
+        Returns:
+        v -- python dictionary that will contain the exponentially weighted average of the gradient.
+                        v["dW" + str(l)] = ...
+                        v["db" + str(l)] = ...
+        s -- python dictionary that will contain the exponentially weighted average of the squared gradient.
+                        s["dW" + str(l)] = ...
+                        s["db" + str(l)] = ...
+        """
+        L = len(parameters) // 2 # number of layers in the neural networks
+        v = {}
+        s = {}
+        # Initialize v, s. Input: "parameters". Outputs: "v, s".
+        for l in range(L):
+        ### START CODE HERE ### (approx. 4 lines)
+            v["dW" + str(l+1)] = np.zeros(parameters["W" + str(l+1)].shape)
+            v["db" + str(l+1)] = np.zeros(parameters["b" + str(l+1)].shape)
+            s["dW" + str(l+1)] = np.zeros(parameters["W" + str(l+1)].shape)
+            s["db" + str(l+1)] = np.zeros(parameters["b" + str(l+1)].shape)
+        ### END CODE HERE ###
+        return v, s
+    def update_parameters_with_adam(parameters, grads, v, s, t, learning_rate = 0.01,
+                                    beta1 = 0.9, beta2 = 0.999,  epsilon = 1e-8):
+        """
+        Update parameters using Adam
+        Arguments:
+        parameters -- python dictionary containing your parameters:
+                        parameters['W' + str(l)] = Wl
+                        parameters['b' + str(l)] = bl
+        grads -- python dictionary containing your gradients for each parameters:
+                        grads['dW' + str(l)] = dWl
+                        grads['db' + str(l)] = dbl
+        v -- Adam variable, moving average of the first gradient, python dictionary
+        s -- Adam variable, moving average of the squared gradient, python dictionary
+        learning_rate -- the learning rate, scalar.
+        beta1 -- Exponential decay hyperparameter for the first moment estimates
+        beta2 -- Exponential decay hyperparameter for the second moment estimates
+        epsilon -- hyperparameter preventing division by zero in Adam updates
+        Returns:
+        parameters -- python dictionary containing your updated parameters
+        v -- Adam variable, moving average of the first gradient, python dictionary
+        s -- Adam variable, moving average of the squared gradient, python dictionary
+        """
+        L = len(parameters) // 2                 # number of layers in the neural networks
+        v_corrected = {}                         # Initializing first moment estimate, python dictionary
+        s_corrected = {}                         # Initializing second moment estimate, python dictionary
+        # Perform Adam update on all parameters
+        for l in range(L):
+            # Moving average of the gradients. Inputs: "v, grads, beta1". Output: "v".
+            ### START CODE HERE ### (approx. 2 lines)
+            v["dW" + str(l+1)] = beta1 * v["dW" + str(l+1)] + (1 - beta1) * grads["dW" + str(l+1)]
+            v["db" + str(l+1)] = beta1 * v["db" + str(l+1)] + (1 - beta1) * grads["db" + str(l+1)]
+            ### END CODE HERE ###
+            # Compute bias-corrected first moment estimate. Inputs: "v, beta1, t". Output: "v_corrected".
+            ### START CODE HERE ### (approx. 2 lines)
+            v_corrected["dW" + str(l+1)] = v["dW" + str(l+1)] / (1 - beta1**t)
+            v_corrected["db" + str(l+1)] = v["db" + str(l+1)] / (1 - beta1**t)
+            ### END CODE HERE ###
+            # Moving average of the squared gradients. Inputs: "s, grads, beta2". Output: "s".
+            ### START CODE HERE ### (approx. 2 lines)
+            s["dW" + str(l+1)] = beta2 * s["dW" + str(l+1)] + (1 - beta2) * (grads["dW" + str(l+1)] ** 2)
+            s["db" + str(l+1)] = beta2 * s["db" + str(l+1)] + (1 - beta2) * (grads["db" + str(l+1)] ** 2)
+            ### END CODE HERE ###
+            # Compute bias-corrected second raw moment estimate. Inputs: "s, beta2, t". Output: "s_corrected".
+            ### START CODE HERE ### (approx. 2 lines)
+            s_corrected["dW" + str(l+1)] = s["dW" + str(l+1)] / (1 - beta2 ** t)
+            s_corrected["db" + str(l+1)] = s["db" + str(l+1)] / (1 - beta2 ** t)
+            ### END CODE HERE ###
+            # Update parameters. Inputs: "parameters, learning_rate, v_corrected, s_corrected, epsilon". Output: "parameters".
+            ### START CODE HERE ### (approx. 2 lines)
+            parameters["W" + str(l+1)] = parameters["W" + str(l+1)] - learning_rate * v_corrected["dW" + str(l+1)] / np.sqrt(s_corrected["dW" + str(l+1)] + epsilon)
+            parameters["b" + str(l+1)] = parameters["b" + str(l+1)] - learning_rate * v_corrected["db" + str(l+1)] / np.sqrt(s_corrected["db" + str(l+1)] + epsilon)
+            ### END CODE HERE ###
+        return parameters, v, s