(Comments)
Updates: If you use the latest TensorFlow 2.0, read this post instead for native support of TensorBoard in any Jupyter notebook - How to run TensorBoard in Jupyter Notebook
Whether you just get started with deep learning, or you are experienced and want a quick experiment, Google Colab is a great free tool to fit the niche. Check out my previous tutorial on a short intro to it if you haven't already.
You might also be aware of TensorBoard, an excellent tool for visualizing while you are training your model.
What's more interesting, if you are building your deep learning model in Keras API with TensorFlow backend, it comes with basic TensorBoard visualizations.
We are going train a Keras model on Colab and visualize it while training with TensorBoard.
But there is one thing we need to address first. Your Google Colab virtual machine is running on a local network located in a Google's server room, while your local machine could be anywhere else in the world.
How to access the TensorBoard page from our local machine?
We are going to use a free service named ngrok to tunnel the connection to your local machine.
Here is a graph to show how it works.
You might have experience build a network tunnel before on Linux, lots of configuration and installation are involved. However, I find it extremely easy to set up a reliable tunnel connection with ngrok.
You can open this Colab notebook while reading this tutorial to enjoy the best learning experience.
Ngrok executable can be directly downloaded to your Colab notebook, run those two lines of code.
!wget https://bin.equinox.io/c/4VmDzA7iaHb/ngrok-stable-linux-amd64.zip
!unzip ngrok-stable-linux-amd64.zip
After this, the executable file ngrok will be extracted to the current directory.
Next, let's fire up the TensorBoard in the background like this:
LOG_DIR = './log'
get_ipython().system_raw(
'tensorboard --logdir {} --host 0.0.0.0 --port 6006 &'
.format(LOG_DIR)
)
It is assuming the TensorBoard log path is "./log", where we are going to tell Keras to log files.
Then, we can run ngrok to tunnel TensorBoard port 6006 to the outside world. This command also runs in the background.
get_ipython().system_raw('./ngrok http 6006 &')
One last step, we get the public URL where we can access the colab TensorBoard web page.
! curl -s http://localhost:4040/api/tunnels | python3 -c \
"import sys, json; print(json.load(sys.stdin)['tunnels'][0]['public_url'])"
This will output a URL you can click on, but wait! We haven't trained our model, so you not get any information from TensorBoard yet.
In this section, we will start training a Keras model and ask Keras to output TensorBoard log files to the ./log directory.
Keras output TensorBoard log files by callbacks, which allows you to visualize dynamic graphs of your training and test metrics, as well as activation histograms for the different layers in your model. You can choose whether to visualize individual components and even how frequently you want Keras to activation and weight histograms. The following code will allow you to visualize all available components of our Keras model. The model itself is a simple two layers of 3x3 convolutional followed by 2 Dense layers to classify MNIST handwritten digits dataset.
from keras.callbacks import TensorBoard
tbCallBack = TensorBoard(log_dir='./log', histogram_freq=1,
write_graph=True,
write_grads=True,
batch_size=batch_size,
write_images=True)
model.fit(x_train, y_train,
batch_size=batch_size,
epochs=epochs,
verbose=1,
validation_data=(x_test, y_test),
callbacks=[tbCallBack])
While this model is training, you can open the previous ngrok link, it should show the TensorBoard page.
If you are new to TensorBoard, in the following section, I will show you some ideas what each component means and when they can be useful.
While the model is training, TensorBoard will automatically update the view periodically.
In this tab, you can see four graphs, acc, loss, acc_val, and loss_val, representing training accuracy, training loss, validation accuracy and validation loss. During an ideal training process, we are expecting the accuracies to increase, and losses decrease over time. However, if you see the validation accuracy start to going downhill after training for some specific epochs. Your model might get overfit on the training sets. I have a quick Two Simple Recipes for Over Fitted Model. While there are other reasons why a model gets overfit, one apparent reason is the training data is too small, and the solution is either find or generate more data.
Our model's losses and accuracies match our expectation.
This tab displays the distribution of activation weights, bias, gradients histograms for different layers in your model. One way this graph can be useful is by visualizing the value of gradients we can tell if a model is suffered from vanishing/exploding gradients, give my previous post a read How to deal with Vanishing/Exploding gradients in Keras.
For other tabs, graphs tab shows the computation graph, more useful if you are building a custom layer or loss function. Image tab visualizes the model weights as images.
You learned how to run TensorBoard on a Google Colab notebook and access it on your local machine by leveraging the free ngrok tunneling service.
One bonus, alternatively you can run the following code in Colab to use localtunnel instead of ngrok.
LOG_DIR = './log'
get_ipython().system_raw(
'tensorboard --logdir {} --host 0.0.0.0 --port 6006 &'
.format(LOG_DIR)
)
# Install
! npm install -g localtunnel
# Tunnel port 6006 (TensorBoard assumed running)
get_ipython().system_raw('lt --port 6006 >> url.txt 2>&1 &')
# Get url
! cat url.txt
To develop more understanding of configuring TensorBoard with Keras, refer to the official document, you can disable some of the unwanted features to speed up the training since generating logs for TensorBoard takes a substantial amount of time.
Also, if you are building a model directly on TensorFlow but not so familiar with TensorBoard yet, you can check out this Hands-on TensorBoard (TensorFlow Dev Summit 2017) on YouTube.
Share on Twitter Share on Facebook
Comments