.. _sec_mlp_scratch:
Thực hiện các Perceptrons đa lớp từ đầu
=======================================
Bây giờ chúng ta đã đặc trưng các nhận thức đa lớp (MLPs) về mặt toán
học, chúng ta hãy cố gắng thực hiện một chính mình. Để so sánh với các
kết quả trước đây của chúng tôi đạt được với hồi quy softmax
(:numref:`sec_softmax_scratch`), chúng tôi sẽ tiếp tục làm việc với
tập dữ liệu phân loại hình ảnh Fashion-MNIST
(:numref:`sec_fashion_mnist`).
.. raw:: html
.. raw:: html
.. code:: python
from mxnet import gluon, np, npx
from d2l import mxnet as d2l
npx.set_np()
batch_size = 256
train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size)
.. raw:: html
.. raw:: html
.. code:: python
import torch
from torch import nn
from d2l import torch as d2l
batch_size = 256
train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size)
.. raw:: html
.. raw:: html
.. code:: python
import tensorflow as tf
from d2l import tensorflow as d2l
batch_size = 256
train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size)
.. raw:: html
.. raw:: html
.. raw:: html
.. code:: python
num_inputs, num_outputs, num_hiddens = 784, 10, 256
W1 = np.random.normal(scale=0.01, size=(num_inputs, num_hiddens))
b1 = np.zeros(num_hiddens)
W2 = np.random.normal(scale=0.01, size=(num_hiddens, num_outputs))
b2 = np.zeros(num_outputs)
params = [W1, b1, W2, b2]
for param in params:
param.attach_grad()
.. raw:: html
.. raw:: html
.. code:: python
num_inputs, num_outputs, num_hiddens = 784, 10, 256
W1 = nn.Parameter(torch.randn(
num_inputs, num_hiddens, requires_grad=True) * 0.01)
b1 = nn.Parameter(torch.zeros(num_hiddens, requires_grad=True))
W2 = nn.Parameter(torch.randn(
num_hiddens, num_outputs, requires_grad=True) * 0.01)
b2 = nn.Parameter(torch.zeros(num_outputs, requires_grad=True))
params = [W1, b1, W2, b2]
.. raw:: html
.. raw:: html
.. code:: python
num_inputs, num_outputs, num_hiddens = 784, 10, 256
W1 = tf.Variable(tf.random.normal(
shape=(num_inputs, num_hiddens), mean=0, stddev=0.01))
b1 = tf.Variable(tf.zeros(num_hiddens))
W2 = tf.Variable(tf.random.normal(
shape=(num_hiddens, num_outputs), mean=0, stddev=0.01))
b2 = tf.Variable(tf.random.normal([num_outputs], stddev=.01))
params = [W1, b1, W2, b2]
.. raw:: html
.. raw:: html
.. raw:: html
.. code:: python
def relu(X):
return np.maximum(X, 0)
.. raw:: html
.. raw:: html
.. code:: python
def relu(X):
a = torch.zeros_like(X)
return torch.max(X, a)
.. raw:: html
.. raw:: html
.. code:: python
def relu(X):
return tf.math.maximum(X, 0)
.. raw:: html
.. raw:: html
.. raw:: html
.. code:: python
def net(X):
X = X.reshape((-1, num_inputs))
H = relu(np.dot(X, W1) + b1)
return np.dot(H, W2) + b2
.. raw:: html
.. raw:: html
.. code:: python
def net(X):
X = X.reshape((-1, num_inputs))
H = relu(X@W1 + b1) # Here '@' stands for matrix multiplication
return (H@W2 + b2)
.. raw:: html
.. raw:: html
.. code:: python
def net(X):
X = tf.reshape(X, (-1, num_inputs))
H = relu(tf.matmul(X, W1) + b1)
return tf.matmul(H, W2) + b2
.. raw:: html
.. raw:: html
.. raw:: html
.. code:: python
loss = gluon.loss.SoftmaxCrossEntropyLoss()
.. raw:: html
.. raw:: html
.. code:: python
loss = nn.CrossEntropyLoss(reduction='none')
.. raw:: html
.. raw:: html
.. code:: python
def loss(y_hat, y):
return tf.losses.sparse_categorical_crossentropy(
y, y_hat, from_logits=True)
.. raw:: html
.. raw:: html
.. raw:: html
.. code:: python
num_epochs, lr = 10, 0.1
d2l.train_ch3(net, train_iter, test_iter, loss, num_epochs,
lambda batch_size: d2l.sgd(params, lr, batch_size))
.. figure:: output_mlp-scratch_106d07_63_0.svg
.. raw:: html
.. raw:: html
.. code:: python
num_epochs, lr = 10, 0.1
updater = torch.optim.SGD(params, lr=lr)
d2l.train_ch3(net, train_iter, test_iter, loss, num_epochs, updater)
.. figure:: output_mlp-scratch_106d07_66_0.svg
.. raw:: html
.. raw:: html
.. code:: python
num_epochs, lr = 10, 0.1
updater = d2l.Updater([W1, W2, b1, b2], lr)
d2l.train_ch3(net, train_iter, test_iter, loss, num_epochs, updater)
.. figure:: output_mlp-scratch_106d07_69_0.svg
.. raw:: html
.. raw:: html
.. raw:: html
.. code:: python
d2l.predict_ch3(net, test_iter)
.. figure:: output_mlp-scratch_106d07_75_0.svg
.. raw:: html
.. raw:: html
.. code:: python
d2l.predict_ch3(net, test_iter)
.. figure:: output_mlp-scratch_106d07_78_0.svg
.. raw:: html
.. raw:: html
.. code:: python
d2l.predict_ch3(net, test_iter)
.. figure:: output_mlp-scratch_106d07_81_0.svg
.. raw:: html
.. raw:: html
.. raw:: html
`Discussions `__
.. raw:: html
.. raw:: html
`Discussions `__
.. raw:: html
.. raw:: html
`Discussions `__
.. raw:: html
.. raw:: html