diff --git a/tutorials/application/source_en/cv/fgsm.md b/tutorials/application/source_en/cv/fgsm.md index 94201ba7d92428377a9fc61069c2aa4d09633898..78dd27e2173401bfa1ec1af42593492f04684417 100644 --- a/tutorials/application/source_en/cv/fgsm.md +++ b/tutorials/application/source_en/cv/fgsm.md @@ -1,4 +1,4 @@ -# Model Adversarial Attack +# FGSM Network Adversarial Attack @@ -24,7 +24,7 @@ Attacks on models can be classified from the following aspects: - **White-box attack**: Attackers have all knowledge and access permissions on a model, including the model structure, weight, input, and output. Attackers can interact with the model system in the process of generating adversarial attack data. Due to that attackers completely master information about models, they can design specific attack algorithms based on the features of the attacked model. - - **Black-box attack**: Contrary to white-box attacks, black-box attacks only obtain limited information about models. Attackers know nothing about the structure or weight of the model and know only part of the input and output. + - **Black-box attack**: Contrary to white-box attacks, attacks only obtain limited information about models. Attackers know nothing about the structure or weight of the model and know only part of the input and output. 2. Attackers' purposes: @@ -44,7 +44,7 @@ FGSM is a simple and efficient method for generating adversarial examples. Diffe Then, the preceding gradient is added to the original input to increase the loss value. As a result, the classification effect of the reconstructed input examples deteriorates, and the attack is successful. -Another requirement of adversarial examples is that the difference between the generated example and the original example must be as small as possible. The **sign** function can be used to make the image modification as even as possible. +Another requirement of adversarial examples is that the difference between the generated example and the original example must be as small as possible. The sign function can be used to make the image modification as even as possible. The generated adversarial perturbation may be expressed by using the following formula: @@ -70,41 +70,90 @@ In this case, MNIST is used to train LeNet with the qualified accuracy, and then Use the following sample code to download and decompress a dataset to a specified location. ```python -from mindvision.dataset import Mnist - -# Download and process the MNIST dataset. -download_train = Mnist(path="./mnist", split="train", shuffle=True, download=True) -download_eval = Mnist(path="./mnist", split="test", download=True) - -dataset_train = download_train.run() -dataset_eval = download_eval.run() +import mindspore.dataset.vision as transforms +from mindspore.dataset.vision import Inter +from mindspore.dataset import MnistDataset + +dataset_train = MnistDataset(dataset_dir="./mnist_dataset", usage="train", shuffle=True) +dataset_eval = MnistDataset(dataset_dir="./mnist_dataset", usage="test", shuffle=True) + +trans_transform = [ + transforms.Resize(size=32, interpolation=Inter.LINEAR), + transforms.Rescale(1.0 / 255.0, 0.0), + transforms.Rescale(1 / 0.3081, -1 * 0.1307 / 0.3081), + transforms.HWC2CHW(), +] + +dataset_train = dataset_train.map(operations=trans_transform, input_columns=["image"]) +dataset_train = dataset_train.map(operations=lambda x: x.astype("int32"), input_columns=["label"]) +dataset_train = dataset_train.batch(batch_size=32, drop_remainder=True) + +dataset_eval = dataset_eval.map(operations=trans_transform, input_columns=["image"]) +dataset_eval = dataset_eval.map(operations=lambda x: x.astype("int32"), input_columns=["label"]) +dataset_eval = dataset_eval.batch(batch_size=32, drop_remainder=True) ``` The directory structure of the downloaded dataset files is as follows: ```text -./mnist -├── test -│ ├── t10k-images-idx3-ubyte -│ └── t10k-labels-idx1-ubyte -└── train +. +└── mnist_dataset + ├── t10k-images-idx3-ubyte + ├── t10k-labels-idx1-ubyte ├── train-images-idx3-ubyte └── train-labels-idx1-ubyte ``` ## Training LeNet -In the experiment, LeNet is used to complete image classification. You need to define the network and use the MNIST dataset for training. +In the experiment, LeNet is used to complete image classification as a demo model. You need to define the network and use the MNIST dataset for training. -Define LeNet. +Define LeNet: ```python -from mindvision.classification.models import lenet - -network = lenet(num_classes=10, pretrained=False) +from mindspore import nn +from mindspore.common.initializer import Normal + + +class LeNet5(nn.Cell): + """LeNet5""" + def __init__(self, num_classes=10, num_channel=1, include_top=True): + super(LeNet5, self).__init__() + self.include_top = include_top + + self.conv1 = nn.Conv2d(num_channel, 6, 5, pad_mode='valid') + self.conv2 = nn.Conv2d(6, 16, 5, pad_mode='valid') + self.relu = nn.ReLU() + self.max_pool2d = nn.MaxPool2d(kernel_size=2, stride=2) + + if self.include_top: + self.flatten = nn.Flatten() + self.fc1 = nn.Dense(16 * 5 * 5, 120, weight_init=Normal(0.02)) + self.fc2 = nn.Dense(120, 84, weight_init=Normal(0.02)) + self.fc3 = nn.Dense(84, num_classes, weight_init=Normal(0.02)) + + def construct(self, x): + """ + LeNet5 construct. + """ + x = self.conv1(x) + x = self.relu(x) + x = self.max_pool2d(x) + x = self.conv2(x) + x = self.relu(x) + x = self.max_pool2d(x) + if self.include_top: + x = self.flatten(x) + x = self.relu(self.fc1(x)) + x = self.relu(self.fc2(x)) + x = self.fc3(x) + return x + + +network = LeNet5() ``` -Define an optimizer and a loss function. +Define an optimizer and a loss function: ```python import mindspore.nn as nn @@ -113,36 +162,30 @@ net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean') net_opt = nn.Momentum(network.trainable_params(), learning_rate=0.01, momentum=0.9) ``` -Define network parameters. +Define network parameters: ```python -from mindspore.train.callback import ModelCheckpoint, CheckpointConfig +import mindspore as ms -config_ck = CheckpointConfig(save_checkpoint_steps=1875, keep_checkpoint_max=10) -ckpoint = ModelCheckpoint(prefix="checkpoint_lenet", config=config_ck) +config_ck = ms.CheckpointConfig(save_checkpoint_steps=1875, keep_checkpoint_max=10) +ckpoint = ms.ModelCheckpoint(prefix="checkpoint_lenet", config=config_ck) ``` -Train LeNet. +Train LeNet: ```python -from mindspore.train import Model -from mindvision.engine.callback import LossMonitor +from mindspore import LossMonitor -model = Model(network, loss_fn=net_loss, optimizer=net_opt, metrics={'accuracy'}) -model.train(5, dataset_train, callbacks=[ckpoint, LossMonitor(0.01, 1875)]) +model = ms.Model(network, loss_fn=net_loss, optimizer=net_opt, metrics={'accuracy'}) +model.train(5, dataset_train, callbacks=[ckpoint, LossMonitor(1875)]) ``` -```python - Epoch:[ 0/ 5], step:[ 1875/ 1875], loss:[0.094/0.094], time:2345.947 ms, lr:0.01000 - Epoch time: 3680.524 ms, per step time: 1.963 ms, avg loss: 0.094 - Epoch:[ 1/ 5], step:[ 1875/ 1875], loss:[0.323/0.323], time:1250.154 ms, lr:0.01000 - Epoch time: 1250.843 ms, per step time: 0.667 ms, avg loss: 0.323 - Epoch:[ 2/ 5], step:[ 1875/ 1875], loss:[0.014/0.014], time:1408.945 ms, lr:0.01000 - Epoch time: 1409.764 ms, per step time: 0.752 ms, avg loss: 0.014 - Epoch:[ 3/ 5], step:[ 1875/ 1875], loss:[0.198/0.198], time:1332.741 ms, lr:0.01000 - Epoch time: 1333.454 ms, per step time: 0.711 ms, avg loss: 0.198 - Epoch:[ 4/ 5], step:[ 1875/ 1875], loss:[0.001/0.001], time:1436.917 ms, lr:0.01000 - Epoch time: 1437.587 ms, per step time: 0.767 ms, avg loss: 0.001 +```text +epoch: 1 step: 1875, loss is 0.022706642746925354 +epoch: 2 step: 1875, loss is 0.11357966810464859 +epoch: 3 step: 1875, loss is 0.011764582246541977 +epoch: 4 step: 1875, loss is 0.014187423512339592 +epoch: 5 step: 1875, loss is 0.0022521568462252617 ``` Test the current network. You can see that LeNet has reached a high accuracy. @@ -156,15 +199,17 @@ print("{}".format(acc)) {'accuracy': 0.9888822115384616} ``` -Load the trained LeNet model. +Load the trained LeNet model: ```python -import mindspore as ms - param_dict = ms.load_checkpoint("checkpoint_lenet-5_1875.ckpt") ms.load_param_into_net(network, param_dict) ``` +```text +[] +``` + ### Implementing FGSM After the accurate LeNet is obtained, the FGSM attack is used to load noise in the image and perform the test again. @@ -172,84 +217,56 @@ After the accurate LeNet is obtained, the FGSM attack is used to load noise in t Compute a backward gradient using the loss function: ```python -class WithLossCell(nn.Cell): - """Package a network and a loss function.""" - - def __init__(self, network, loss_fn): - super(WithLossCell, self).__init__() - self._network = network - self._loss_fn = loss_fn - - def construct(self, data, label): - out = self._network(data) - return self._loss_fn(out, label) - - -class GradWrapWithLoss(nn.Cell): - """Compute the backward gradient based on the loss.""" - - def __init__(self, network): - super(GradWrapWithLoss, self).__init__() - self._network = network - - def construct(self, inputs, labels): - gout = ms.grad(self._network)(inputs, labels) - return gout +def forward_fn(inputs, targets): + out = network(inputs) + loss = net_loss(out, targets) + return loss ``` Then, implement the FGSM attack according to formula (2): ```python import numpy as np -import mindspore as ms -class FastGradientSignMethod: - """Implement the FGSM attack.""" - - def __init__(self, network, eps=0.07, loss_fn=None): - # Initialize the variables. - self._network = network - self._eps = eps - with_loss_cell = WithLossCell(self._network, loss_fn) - self._grad_all = GradWrapWithLoss(with_loss_cell) - self._grad_all.set_train() - - - def _gradient(self, inputs, labels): - # Compute the gradient. - out_grad = self._grad_all(inputs, labels) - gradient = out_grad.asnumpy() - gradient = np.sign(gradient) - return gradient - - def generate(self, inputs, labels): - # Implement FGSM. - inputs_tensor = ms.Tensor(inputs) - labels_tensor = ms.Tensor(labels) - gradient = self._gradient(inputs_tensor, labels_tensor) - # Generate perturbations. - perturbation = self._eps*gradient - # Generate the perturbed image. - adv_x = inputs + perturbation - return adv_x - - def batch_generate(self, inputs, labels, batch_size=32): - # Process the dataset. - arr_x = inputs - arr_y = labels - len_x = len(inputs) - batches = int(len_x / batch_size) - res = [] - for i in range(batches): - x_batch = arr_x[i*batch_size: (i + 1)*batch_size] - y_batch = arr_y[i*batch_size: (i + 1)*batch_size] - adv_x = self.generate(x_batch, y_batch) - res.append(adv_x) - adv_x = np.concatenate(res, axis=0) - return adv_x + +def gradient_func(inputs, labels): + _grad_all = ops.composite.GradOperation(get_all=True, sens_param=False) + # Obtain the gradient + out_grad = _grad_all(forward_fn)(inputs, labels)[0] + gradient = out_grad.asnumpy() + gradient = np.sign(gradient) + return gradient + + +def generate(inputs, labels, eps): + # Implement FGSM + inputs_tensor = ms.Tensor(inputs) + labels_tensor = ms.Tensor(labels) + gradient = gradient_func(inputs_tensor, labels_tensor) + # Generate perturbation + perturbation = eps * gradient + # Generate the images after perturbation + adv_x = inputs + perturbation + return adv_x + + +def batch_generate(inputs, labels, eps, batch_size): + # Process the dataset + arr_x = inputs + arr_y = labels + len_x = len(inputs) + batches = int(len_x / batch_size) + res = [] + for i in range(batches): + x_batch = arr_x[i * batch_size: (i + 1) * batch_size] + y_batch = arr_y[i * batch_size: (i + 1) * batch_size] + adv_x = generate(x_batch, y_batch, eps=eps) + res.append(adv_x) + adv_x = np.concatenate(res, axis=0) + return adv_x ``` -Process images in the test set in the MNIST dataset again. +Reprocess the images from the test set in the MINIST dataset: ```python images = [] @@ -275,17 +292,16 @@ true_labels = np.concatenate(test_labels) ## Running Attack -It can be seen from the FGSM attack formula that, the larger the attack coefficient $\varepsilon$, the greater the change to the gradient. If the value of $\varepsilon$ is **0**, the attack effect is not reflected. +It can be seen from the FGSM attack formula that, the larger the attack coefficient $\varepsilon$, the greater the change to the gradient. If the value of $\varepsilon$ is 0, the attack effect is not reflected. $$\eta = \varepsilon sign(\nabla_x J(\theta)) \tag{3}$$ -Observe the attack effect when $\varepsilon$ is **0**. +Observe the attack effect when $\varepsilon$ is 0: ```python import mindspore.ops as ops -fgsm = FastGradientSignMethod(network, eps=0.0, loss_fn=net_loss) -advs = fgsm.batch_generate(test_images, true_labels, batch_size=32) +advs = batch_generate(test_images, true_labels, batch_size=32, eps=0.0) adv_predicts = model.predict(ms.Tensor(advs)).asnumpy() adv_predicts = np.argmax(adv_predicts, axis=1) @@ -293,15 +309,14 @@ accuracy = np.mean(np.equal(adv_predicts, true_labels)) print(accuracy) ``` -```python +```text 0.9888822115384616 ``` -Set **$\varepsilon$** to **0.5** and try to run the attack. +Set $\varepsilon$ to 0.5 and try to run the attack. ```python -fgsm = FastGradientSignMethod(network, eps=0.5, loss_fn=net_loss) -advs = fgsm.batch_generate(test_images, true_labels, batch_size=32) +advs = batch_generate(test_images, true_labels, batch_size=32, eps=0.5) adv_predicts = model.predict(ms.Tensor(advs)).asnumpy() adv_predicts = np.argmax(adv_predicts, axis=1) @@ -309,7 +324,7 @@ accuracy = np.mean(np.equal(adv_predicts, true_labels)) print(accuracy) ``` -```python +```text 0.36828926282051283 ``` @@ -319,17 +334,17 @@ The following shows the actual form of the attacked image. It can be seen that t ```python import matplotlib.pyplot as plt -%matplotlib inline +% matplotlib inline adv_examples = np.transpose(advs[:10], [0, 2, 3, 1]) ori_examples = np.transpose(test_images[:10], [0, 2, 3, 1]) plt.figure(figsize=(10, 3), dpi=120) for i in range(10): - plt.subplot(3, 10, i+1) + plt.subplot(3, 10, i + 1) plt.axis("off") plt.imshow(np.squeeze(ori_examples[i])) - plt.subplot(3, 10, i+11) + plt.subplot(3, 10, i + 11) plt.axis("off") plt.imshow(np.squeeze(adv_examples[i])) plt.show()