Machine Learning/CS231n
-
[CS231n] 9. Convolutional Neural Networks: Layer Patterns, Case studiesMachine Learning/CS231n 2022. 5. 13. 22:11
ConvNet Architectures We have seen that Convolutional Networks are commonly made up of only three layer types: CONV, POOL and FC. We will also explicitly write the ReLU activation function as a layer, which applies elementwise non-linearity. Layer Patterns The most common form of a ConvNet architecture stacks a few CONV-ReLU layers, follows them with POOL layers, and repeats this pattern until t..
-
[CS231n] 8. Convolutional Neural Networks: Architectures, Pooling LayersMachine Learning/CS231n 2022. 5. 12. 11:52
layers, spatial arrangement, layer patterns, layer sizing patterns, AlexNet/ZFnet/VGGNet case studies, computational considerations Convolutional Neural Networks (CNNs / ConvNets) CNNs are very similar to ordinary NN. → made up of neurons that have learnable weights and biases. Each neuron receives some inputs, performs a dot product and optionally follows it with a non-linearity. still have sco..
-
[CS231n] 7. Neural Networks Part 3 : Learning and EvaluationMachine Learning/CS231n 2022. 5. 5. 16:34
gradient checks, sanity checks, babysitting the learning process, momentum(+nesterov), second-order methods, Adagrad/RMSprop, hyperparameter optimization, model ensembles Learning This section is devoted to the dynamics, the process of learning the parameters and finding good hyperparameters Gradient Checks Use the centered formula finite difference approximation when evaluating the numerical gr..
-
[CS231n] 6. Neural Networks Part2 : Setting up the DataMachine Learning/CS231n 2022. 5. 3. 01:23
preprocessing, weight initialization, batch normalization, regularization (L2/dropout) 1) Data Preprocessing We will assume matrix X is of size [N x D] (N is the number of data, D is their dimensionality) 1.1) Mean subtraction most common form of preprocessing Subtracting the mean across every individual feature in the data It has the geometric interpretation of centering the cloud of data aroun..
-
[CS231n] 5. Neural Networks Part 1: Setting up the ArchitectureMachine Learning/CS231n 2022. 4. 29. 02:22
keywords : model of a biological neuron, activation functions, neural net architecture, representational power 1) Biological motivation and connections basic computational unit of the brain is a neuron Model the firing rate of the neuron with an activation function f, which represents the frequency of the spikes along the axon. Historically, a common choice of activation function is the sigmoid ..
-
[CS231n] 4. BackpropagationMachine Learning/CS231n 2022. 4. 27. 16:13
Keywords : chain rule interpretation, real-valued circuits, patterns in gradient flow 1) Introduction Intuitive understanding of backpropagation A way of computing gradients of expressions through recursive application of chain rule. Understanding of this process is critical to understand, and effectively develop, design and debug neural networks. Think of the training data as given and fixed, a..