## [1. Introduction](https://codelabs.developers.google.com/codelabs/tfjs-training-classfication/index.html?hl=zh-cn#0) In this tutorial, we're going to build a TensorFlow.js model that uses a convolutional neural network to recognize handwritten digits. The plan: train a classifier by letting it “look at” thousands of digit images along with their labels, and then see how well it does using some test data it’s never seen before. This is a classification problem—basically, we’re teaching the model to assign a category (the digit in the picture) to each input image. We’ll train it by showing lots of examples and the correct answers. This is what folks in the biz call [supervised learning](https://developers.google.com/machine-learning/intro-to-ml/supervised?hl=zh-cn). ### **What You’re Actually Building** You’ll make a web page that trains a model right in your browser using TensorFlow.js. Once you give it a black-and-white image of a certain size, the model will try to guess which digit is in the picture. The steps look like this: - Load the data. - Define the model architecture. - Train the model and watch how it's doing. - Evaluate the trained model by making some predictions. ## **What You’ll Learn** - How to use TensorFlow.js syntax, especially the Layers API, to build a convolutional model. - How to set up a classification task in TensorFlow.js. - How to watch your model training in real time with the tfjs-vis library. ## [2. Getting Set Up](https://codelabs.developers.google.com/codelabs/tfjs-training-classfication/index.html?hl=zh-cn#1) ## Create an HTML Page and Add JavaScript ![](https://codelabs.developers.google.com/static/codelabs/tfjs-training-classfication/img/96914ff65fc3b74c.png?hl=zh-cn) **Copy the following code into a file called `index.html`.** **Heads up: I tweaked some things from the original tutorial—removed `type="module"` from the src attributes for "data.js" and "script.js". This makes everything run in the browser without running into CORS errors.** **`index.html`** ``` TensorFlow.js Tutorial

TensorFlow.js Tutorial

``` ## **Create JavaScript Files for Data and Code** - In the same folder as your HTML file, make a file called **`data.js`** and copy [the code from this link](https://storage.googleapis.com/tfjs-tutorials/mnist_data.js) into it. - In the same folder, create a file called **`script.js`** and put the following code in it: ``` console.log('Hello TensorFlow'); ``` ## **Ready to Test** Now that you’ve got your HTML and JavaScript files, it’s time to take them for a spin. Open the `index.html` file in your browser, then pop open the Devtools console. If everything’s set up right, you should get two global variables: `tf` (that’s TensorFlow.js) and `tfvis` (the [tfjs-vis library](https://github.com/tensorflow/tfjs-vis)). You should also see a nice, reassuring line: `Hello TensorFlow`. If you do, you’re good to go. ## [3. Loading the Data](https://codelabs.developers.google.com/codelabs/tfjs-training-classfication/index.html?hl=zh-cn#2) We’re going to train a model to recognize the digits in images like these: ![mnist 4](https://codelabs.developers.google.com/static/codelabs/tfjs-training-classfication/img/c01d85004462fd48.png?hl=zh-cn) ![mnist 3](https://codelabs.developers.google.com/static/codelabs/tfjs-training-classfication/img/58d4cf23ee7202be.png?hl=zh-cn) ![mnist 8](https://codelabs.developers.google.com/static/codelabs/tfjs-training-classfication/img/8998bab63049c12a.png?hl=zh-cn) These are 28x28 grayscale images from the classic [MNIST](http://yann.lecun.com/exdb/mnist/) dataset. To keep things simple, there’s code that loads these images from a special [sprite file](https://storage.googleapis.com/learnjs-data/model-builder/mnist_images.png) (~10MB), so we can focus on the fun part: the training. Curious how the data loading works? Check out the [`data.js`](https://raw.githubusercontent.com/tensorflow/tfjs-examples/master/mnist-core/data.js) file whenever you want. After this tutorial, you should try making your own data loader. **Note:** In the example code’s `data.js`, I removed `import * as tf from '@tensorflow/tfjs';` and took out `export` in `export class MnistData{`, so the script works directly in the browser. In `script.js`, I deleted `import {MnistData} from './data.js';`. If you use the `module` method, you might get errors like: ``` - SyntaxError: Cannot use import statement outside a module - Uncaught TypeError: Failed to resolve module specifier "@tensorflow/tfjs/dist/index.js". Relative references must start with either "/", "./", or "../" ``` **Heads up:** Every time you refresh, it has to download the ~10MB data file from Google. To avoid this, you can download the `mnist_images.png` and `mnist_labels_uint8` files to your own machine, run a simple local server (I use http-server: `npm i -g http-server`), and update the paths in `data.js` to point to your local files: ``` const MNIST_IMAGES_SPRITE_PATH = './mnist_images.png'; // 'https://storage.googleapis.com/learnjs-data/model-builder/mnist_images.png'; const MNIST_LABELS_PATH = './mnist_labels_uint8'; // 'https://storage.googleapis.com/learnjs-data/model-builder/mnist_labels_uint8'; ``` Same goes for `tf.min.js` and `tfjs-vis.umd.min.js`—downloading them locally and updating the paths makes debugging way faster. ``` ``` The code gives you a `MnistData` class with two handy methods: - `nextTrainBatch(batchSize)`: gets a random batch of training images and their labels. - `nextTestBatch(batchSize)`: gets a batch from the test set. MnistData also does the important jobs of **shuffling** and **normalizing** the data. There are 65,000 images in total: up to 55,000 for training, and 10,000 set aside for testing after your model’s trained. All of this runs in your browser—no need for a server. If you were doing this in Node.js, you’d just load images straight from disk and use native image libraries to grab the pixel data. Let’s load the data and see if it works. ![](https://codelabs.developers.google.com/static/codelabs/tfjs-training-classfication/img/96914ff65fc3b74c.png?hl=zh-cn) **Add the following code to your `script.js` file.** ``` async function showExamples(data) { // Create a container in the visor const surface = tfvis.visor().surface({ name: 'Input Data Examples', tab: 'Input Data' }); // Get the examples const examples = data.nextTestBatch(20); const numExamples = examples.xs.shape[0]; // Create a canvas element to render each example for (let i = 0; i < numExamples; i++) { // Reshape the image to 28x28 px const imageTensor = examples.xs .slice([i, 0], [1, examples.xs.shape[1]]) .reshape([28, 28, 1]); const canvas = document.createElement('canvas'); canvas.width = 28; canvas.height = 28; canvas.style = 'margin: 4px;'; await tf.browser.toPixels(imageTensor, canvas); surface.drawArea.appendChild(canvas); imageTensor.dispose(); } } async function run() { const data = new MnistData(); await data.load(); await showExamples(data); } document.addEventListener('DOMContentLoaded', run); ``` Refresh the page, and after a few seconds, a panel will pop up on the left with a bunch of images. ![](https://lxunzzzdnokdqhipbmdf.supabase.co/storage/v1/object/public/media/wp-backup/2022/11/6dff857738b54eed-1024x291.png) ## [4. Planning the Task](https://codelabs.developers.google.com/codelabs/tfjs-training-classfication/index.html?hl=zh-cn#3) Here’s what the input data looks like: ![](https://lxunzzzdnokdqhipbmdf.supabase.co/storage/v1/object/public/media/wp-backup/2022/11/6dff857738b54eed-1669712548911-3-1024x291.png) Our goal is to train a model that takes **one image** and predicts **a score for each of the ten possible classes** (digits 0 through 9). Each image is 28x28 pixels, with just one color channel (black-and-white). That means every image has the shape `[28, 28, 1]`. Heads up: we need to map “one image” to “ten possibilities,” and the shape of each example matters for what comes next. ## [5. Defining the Model Architecture](https://codelabs.developers.google.com/codelabs/tfjs-training-classfication/index.html?hl=zh-cn#4) Time to write some code to lay out the model. In plain English, a model architecture is “the function your model runs under the hood,” or “the process it’ll use to compute answers.” In ML, we define the architecture (a.k.a. the algorithm), then let training figure out the best parameters. ![](https://codelabs.developers.google.com/static/codelabs/tfjs-training-classfication/img/96914ff65fc3b74c.png?hl=zh-cn) **Add the following function to your** **`script.js`** **file to define the model architecture** ``` function getModel() { const model = tf.sequential(); const IMAGE_WIDTH = 28; const IMAGE_HEIGHT = 28; const IMAGE_CHANNELS = 1; // In the first layer of our convolutional neural network we have // to specify the input shape. Then we specify some parameters for // the convolution operation that takes place in this layer. model.add(tf.layers.conv2d({ inputShape: [IMAGE_WIDTH, IMAGE_HEIGHT, IMAGE_CHANNELS], kernelSize: 5, filters: 8, strides: 1, activation: 'relu', kernelInitializer: 'varianceScaling' })); // The MaxPooling layer acts as a sort of downsampling using max values // in a region instead of averaging. model.add(tf.layers.maxPooling2d({ poolSize: [2, 2], strides: [2, 2] })); // Repeat another conv2d + maxPooling stack. // Note that we have more filters in the convolution. model.add(tf.layers.conv2d({ kernelSize: 5, filters: 16, strides: 1, activation: 'relu', kernelInitializer: 'varianceScaling' })); model.add(tf.layers.maxPooling2d({ poolSize: [2, 2], strides: [2, 2] })); // Now we flatten the output from the 2D filters into a 1D vector to prepare // it for input into our last layer. This is common practice when feeding // higher dimensional data to a final classification output layer. model.add(tf.layers.flatten()); // Our last layer is a dense layer which has 10 output units, one for each // output class (i.e. 0, 1, 2, 3, 4, 5, 6, 7, 8, 9). const NUM_OUTPUT_CLASSES = 10; model.add(tf.layers.dense({ units: NUM_OUTPUT_CLASSES, kernelInitializer: 'varianceScaling', activation: 'softmax' })); // Choose an optimizer, loss function and accuracy metric, // then compile and return the model const optimizer = tf.train.adam(); model.compile({ optimizer: optimizer, loss: 'categoricalCrossentropy', metrics: ['accuracy'], }); return model; } ``` Let’s poke around a little more. ## Convolution ``` model.add(tf.layers.conv2d({ inputShape: [IMAGE_WIDTH, IMAGE_HEIGHT, IMAGE_CHANNELS], kernelSize: 5, filters: 8, strides: 1, activation: 'relu', kernelInitializer: 'varianceScaling' })); ``` We’re using a sequential model here. Instead of a dense layer, we go with `conv2d`. Going deep on how [convolution](https://developers.google.com/machine-learning/glossary/?hl=zh-cn#convolutional_layer) works is a whole other rabbit hole, but if you’re curious, check these out: - [A visual intro to image kernels](http://setosa.io/ev/image-kernels/) - [Convolutional Neural Networks for Visual Recognition](http://cs231n.github.io/convolutional-networks/) Here’s what each part of the `conv2d` config does: - `inputShape`: the shape of the data going into the first layer. Here, MNIST images are 28x28 pixels, black-and-white, so `[28, 28, 1]`. Standard format is `[row, column, depth]`: 28 rows, 28 columns, depth 1 (just one color channel). We don’t specify batch size in the shape—layers are batch-agnostic, so you can feed any number of images at once. - `kernelSize`: the size of the sliding convolution filter window. We use `5`, meaning a 5x5 square filter. - [`filters`](https://developers.google.com/machine-learning/glossary/?hl=zh-cn#convolutional_filter): number of different filters (of size `kernelSize`) to apply. Here, we use 8. - [`strides`](https://developers.google.com/machine-learning/glossary/?hl=zh-cn#stride): how far the window moves each time. Here, stride is 1, so the filter slides over by 1 pixel at a time. - `activation`: the [activation function](https://developers.google.com/machine-learning/glossary/?hl=zh-cn#activation_function) applied after convolution. Here, it’s [ReLU](https://developers.google.com/machine-learning/glossary/?hl=zh-cn#ReLU), which is super common. - `kernelInitializer`: how to randomly initialize the weights—a surprisingly important detail. We’re using `VarianceScaling`, which is usually a good bet ([more on initializers](https://js.tensorflow.org/api/latest/?hl=zh-cn#Initializers)). You *could* build an image classifier just with dense layers, but convolution layers have proven way more effective for image tasks. ## Flattening ``` model.add(tf.layers.flatten()); ``` Images are high-dimensional, and convolutions tend to expand the data even more. Before passing it to the final classification layer, we flatten everything into a long array. Dense layers (which we use last) expect a flat `tensor1d`, so this step is super common. **Note:** The flatten layer doesn’t have any weights. It just unrolls the input into a long array. ## **Calculating the Final Probability Distribution** ``` const NUM_OUTPUT_CLASSES = 10; model.add(tf.layers.dense({ units: NUM_OUTPUT_CLASSES, kernelInitializer: 'varianceScaling', activation: 'softmax' })); ``` Now we use a dense layer with softmax activation to get a probability distribution over the 10 classes. The class with the highest score is the prediction. Remember, we want to map one image to ten probabilities—hence, 10 units in the output layer. Softmax is almost always the go-to activation function for the last layer in a classification problem. ## Picking Optimizer and Loss Function ``` const optimizer = tf.train.adam(); model.compile({ optimizer: optimizer, loss: 'categoricalCrossentropy', metrics: ['accuracy'], }); ``` We [... content truncated ...]