## [1. Introduction](https://codelabs.developers.google.com/codelabs/tfjs-training-classfication/index.html?hl=zh-cn#0) In this tutorial, we’ll build a TensorFlow.js model that uses a convolutional neural network to recognize handwritten digits. First we’ll train the classifier by letting it “look at” thousands of images of handwritten digits together with their labels. Then we’ll use test data the model has never seen to check how accurate the classifier really is. This is a classification task, because we’re training the model to assign a class (the digit in the image) to each input image. We’ll train the model by feeding many examples of inputs and their correct outputs. This is called [supervised learning](https://developers.google.com/machine-learning/intro-to-ml/supervised?hl=zh-cn). ***## *****The model you’ll build** You’ll create a web page that trains a model right in the browser using TensorFlow.js. Once we give it black‑and‑white images of a specific size, the model will classify which digit appears in each image. The steps: - Load the data. - Define the model architecture. - Train the model and watch how it performs while training. - Evaluate the trained model by making some predictions. ## **What you’ll learn** - TensorFlow.js syntax for creating a convolutional model using the TensorFlow.js Layers API. - How to set up a classification task in TensorFlow.js. - How to use the tfjs-vis library to monitor training in the browser. ## [2. Set things up](https://codelabs.developers.google.com/codelabs/tfjs-training-classfication/index.html?hl=zh-cn#1) ## Create the HTML page and add JavaScript ![](https://codelabs.developers.google.com/static/codelabs/tfjs-training-classfication/img/96914ff65fc3b74c.png?hl=zh-cn) **Copy the following code into an HTML file named `index.html`.** **Note: I’ve tweaked the original tutorial code a bit — removed `type="module"` after `src="data.js"` and `src="script.js"` — so the program can run directly in the browser without throwing CORS errors.** **`index.html`** ```html TensorFlow.js Tutorial

TensorFlow.js Tutorial

``` ## **Create JavaScript files for data and code** - In the same folder as the HTML file above, create a file named **`data.js`**, and copy the contents from [this link](https://storage.googleapis.com/tfjs-tutorials/mnist_data.js) into it. - In the same folder, create a file named **`script.js`**, and put the following code in it: ```js console.log('Hello TensorFlow'); ``` ## **Start testing** Now that you have the HTML and JavaScript files, you can test them. Open `index.html` in a browser, then open the Devtools console. If everything is wired up correctly, two global variables should exist: `tf`, a reference to the TensorFlow.js library, and `tfvis`, a reference to the [tfjs-vis library](https://github.com/tensorflow/tfjs-vis). You should also see a `Hello TensorFlow` message. If you do, you’re ready for the next step. ## [3. Load the data](https://codelabs.developers.google.com/codelabs/tfjs-training-classfication/index.html?hl=zh-cn#2) In this tutorial you’ll train a model to recognize digits in images like the ones below. These images are 28x28‑pixel grayscale images from a dataset called [MNIST](http://yann.lecun.com/exdb/mnist/). ![mnist 4](https://codelabs.developers.google.com/static/codelabs/tfjs-training-classfication/img/c01d85004462fd48.png?hl=zh-cn) ![mnist 3](https://codelabs.developers.google.com/static/codelabs/tfjs-training-classfication/img/58d4cf23ee7202be.png?hl=zh-cn) ![mnist 8](https://codelabs.developers.google.com/static/codelabs/tfjs-training-classfication/img/8998bab63049c12a.png?hl=zh-cn) We’ve provided code to load these images from a special [sprite file](https://storage.googleapis.com/learnjs-data/model-builder/mnist_images.png) (about 10 MB) we created for you, so we can focus on the training part. You can always study the [`data.js`](https://raw.githubusercontent.com/tensorflow/tfjs-examples/master/mnist-core/data.js) file to see how the data is loaded. After you finish this tutorial, you can build your own data‑loading logic. **Note:** In the sample `data.js` file, I removed `import * as tf from '@tensorflow/tfjs';`, and removed the `export` keyword from `export class MnistData {` so the program runs normally in the browser. In `script.js` I also removed `import {MnistData} from './data.js';`. Browser scripts can be called directly; using `module` may give you errors like these, hence the changes: ```text - SyntaxError: Cannot use import statement outside a module - Uncaught TypeError: Failed to resolve module specifier "@tensorflow/tfjs/dist/index.js". Relative references must start with either "/", "./", or "../" ``` **Another note:** Every time you refresh the example, it has to download that 10‑something‑MB data file from Google and slice it up. To avoid that, you can download the `mnist_images.png` file and the `mnist_labels_uint8` file mentioned in the example to your local machine, and run a simple local server (I used `http-server` via `npm i -g http-server`). Then change the image and label paths in `data.js` to local paths: ```js const MNIST_IMAGES_SPRITE_PATH = './mnist_images.png'; // 'https://storage.googleapis.com/learnjs-data/model-builder/mnist_images.png'; const MNIST_LABELS_PATH = './mnist_labels_uint8'; // 'https://storage.googleapis.com/learnjs-data/model-builder/mnist_labels_uint8'; ``` Similarly, when I use a local server, I also download `tf.min.js` and `tfjs-vis.umd.min.js` locally and change their paths to speed up debugging. ```html ``` The provided code exposes a `MnistData` class that has two public methods: - `nextTrainBatch(batchSize)`: returns a random batch of images and labels from the training set. - `nextTestBatch(batchSize)`: returns a batch of images and labels from the test set. The `MnistData` class also handles two important things for you: **shuffling** the data and **normalizing** it. There are 65,000 images in total. We can use up to 55,000 images to train the model, and keep 10,000 images to test how well the model performs afterward. All of this runs in the browser! If you were doing this in Node.js, you could load images directly from the file system and use native image‑processing libraries to read pixel data. Let’s load the data and check that it’s loaded correctly. ![](https://codelabs.developers.google.com/static/codelabs/tfjs-training-classfication/img/96914ff65fc3b74c.png?hl=zh-cn) **Add the following code to your `script.js` file.** ```js async function showExamples(data) { // Create a container in the visor const surface = tfvis.visor().surface({ name: 'Input Data Examples', tab: 'Input Data' }); // Get the examples const examples = data.nextTestBatch(20); const numExamples = examples.xs.shape[0]; // Create a canvas element to render each example for (let i = 0; i < numExamples; i++) { // Reshape the image to 28x28 px const imageTensor = tf.tidy(() => { return examples.xs .slice([i, 0], [1, examples.xs.shape[1]]) .reshape([28, 28, 1]); }); const canvas = document.createElement('canvas'); canvas.width = 28; canvas.height = 28; canvas.style = 'margin: 4px;'; await tf.browser.toPixels(imageTensor, canvas); surface.drawArea.appendChild(canvas); imageTensor.dispose(); } } async function run() { const data = new MnistData(); await data.load(); await showExamples(data); } document.addEventListener('DOMContentLoaded', run); ``` Refresh the page. After a few seconds, a panel should pop up on the left with a bunch of images. ![](https://lxunzzzdnokdqhipbmdf.supabase.co/storage/v1/object/public/media/wp-backup/2022/11/6dff857738b54eed-1024x291.png) ## [4. Frame the task properly](https://codelabs.developers.google.com/codelabs/tfjs-training-classfication/index.html?hl=zh-cn#3) Here’s what our input data looks like: ![](https://lxunzzzdnokdqhipbmdf.supabase.co/storage/v1/object/public/media/wp-backup/2022/11/6dff857738b54eed-1669712548911-3-1024x291.png) Our goal is to train a model that takes **a single image** and learns to predict **a score for each of the 10 possible classes** (digits 0–9). Each image is 28x28 pixels with 1 color channel, because it’s grayscale. So each image has shape `[28, 28, 1]`. Keep in mind that we’re mapping one thing to ten (one input image to ten probabilities), and that each input example has this shape. This matters for the next part. ## [5. Define the model architecture](https://codelabs.developers.google.com/codelabs/tfjs-training-classfication/index.html?hl=zh-cn#4) Now we’ll write code to describe the model architecture. “Model architecture” is just a fancy way of saying “the function the model will run when it’s doing its thing”, or “the algorithm our model will use to compute answers”. In machine learning, we define the architecture (or algorithm), and let the training process learn the parameters of that algorithm. ![](https://codelabs.developers.google.com/static/codelabs/tfjs-training-classfication/img/96914ff65fc3b74c.png?hl=zh-cn) **Add the following function to your** **`script.js`** **file to define the model architecture** ```js function getModel() { const model = tf.sequential(); const IMAGE_WIDTH = 28; const IMAGE_HEIGHT = 28; const IMAGE_CHANNELS = 1; // In the first layer of our convolutional neural network we have // to specify the input shape. Then we specify some parameters for // the convolution operation that takes place in this layer. model.add(tf.layers.conv2d({ inputShape: [IMAGE_WIDTH, IMAGE_HEIGHT, IMAGE_CHANNELS], kernelSize: 5, filters: 8, strides: 1, activation: 'relu', kernelInitializer: 'varianceScaling' })); // The MaxPooling layer acts as a sort of downsampling using max values // in a region instead of averaging. model.add(tf.layers.maxPooling2d({poolSize: [2, 2], strides: [2, 2]})); // Repeat another conv2d + maxPooling stack. // Note that we have more filters in the convolution. model.add(tf.layers.conv2d({ kernelSize: 5, filters: 16, strides: 1, activation: 'relu', kernelInitializer: 'varianceScaling' })); model.add(tf.layers.maxPooling2d({poolSize: [2, 2], strides: [2, 2]})); // Now we flatten the output from the 2D filters into a 1D vector to prepare // it for input into our last layer. This is common practice when feeding // higher dimensional data to a final classification output layer. model.add(tf.layers.flatten()); // Our last layer is a dense layer which has 10 output units, one for each // output class (i.e. 0, 1, 2, 3, 4, 5, 6, 7, 8, 9). const NUM_OUTPUT_CLASSES = 10; model.add(tf.layers.dense({ units: NUM_OUTPUT_CLASSES, kernelInitializer: 'varianceScaling', activation: 'softmax' })); // Choose an optimizer, loss function and accuracy metric, // then compile and return the model const optimizer = tf.train.adam(); model.compile({ optimizer: optimizer, loss: 'categoricalCrossentropy', metrics: ['accuracy'], }); return model; } ``` Let’s unpack this a bit. ## Convolution ```js model.add(tf.layers.conv2d({ inputShape: [IMAGE_WIDTH, IMAGE_HEIGHT, IMAGE_CHANNELS], kernelSize: 5, filters: 8, strides: 1, activation: 'relu', kernelInitializer: 'varianceScaling' })); ``` We’re using a sequential model here. Instead of dense layers, we’re using a `conv2d` layer. We won’t go through every detail of how [convolutions](https://developers.google.com/machine-learning/glossary/?hl=zh-cn#convolutional_layer) work, but here are a couple of good resources if you want to see the basic mechanics: - [Image Kernels explained visually](http://setosa.io/ev/image-kernels/) - [Convolutional Neural Networks for Visual Recognition](http://cs231n.github.io/convolutional-networks/) Now let’s look at each parameter in the `conv2d` config: - `inputShape`: the shape of the data flowing into the first layer. Here, our MNIST examples are 28x28‑pixel black‑and‑white images. The convention for image data is `[row, column, depth]`, so we need `[28, 28, 1]` here. 28 rows, 28 columns, and depth 1, since there’s only one color channel. Notice we don’t specify batch size. Layers are designed to be batch‑size agnostic, so at inference time you can feed tensors with any batch size. - `kernelSize`: the size of the sliding convolution filter window applied to the input. We use `5`, meaning a 5x5 square convolution window. - [`filters`](https://developers.google.com/machine-learning/glossary/?hl=zh-cn#convolutional_filter): how many filters of size `kernelSize` to apply to the input. Here we apply 8 filters. - [`strides`](https://developers.google.com/machine-learning/glossary/?hl=zh-cn#stride): the step size of the sliding window, i.e. how many pixels the filter moves each time. We use stride 1, so the filter slides one pixel at a time. - `activation`: the [activation function](https://developers.google.com/machine-learning/glossary/?hl=zh-cn#activation_function) applied after the convolution. Here we use [ReLU](https://developers.google.com/machine-learning/glossary/?hl=zh-cn#ReLU), which is very common in ML models. - `kernelInitializer`: how to randomly initialize model weights — this matters a lot for training dynamics. We won’t get into initializers here, but `VarianceScaling` is usually a solid [initializer choice](https://js.tensorflow.org/api/latest/?hl=zh-cn#Initializers). You *can* build an image classifier using only dense layers, but convolution layers have proven very effective for a lot of image‑based tasks. ## Flattening the representation ```js model.add(tf.layers.flatten()); ``` Images are high‑dimensional data, and convolution operations usually increase the size of the intermediate representation. Before sending the data into the final classification layer, we need to flatten it into one long array. Dense layers (which we use for the final layer) expect a `tensor1d`, so this step is common in many classification problems. **Note:** the flatten layer has no weights. It just unrolls its input into a long array. ## **Computing the final probability distribution** ```js const NUM_OUTPUT_CLASSES = 10; model.add(tf.layers.dense({ units: NUM_OUTPUT_CLASSES, kernelInitializer: 'varianceScaling', activation: 'softmax' })); ``` We use a dense layer with softmax activation to compute a probability distribution over the 10 possible classes. The class with the highest score is the predicted digit. Remember, we’re mapping one input to ten outputs (one image to ten probabilities). That’s why we have 10 units in the output layer. Softmax is almost always what you want on the last layer of a classification model. ## Choose optimizer and loss function ```js const optimizer = tf.train.adam(); model.compile({ optimizer: optimizer, loss: 'categoricalCrossentropy', metrics: ['accuracy'], }); ``` We… [... content truncated ...]