TensorFlow 101

2018

Apr 30

Y = MX + B

It is time to talk about the latest buzzword in the tech world: Machine Learning! If you are not using machine learning or AI in your products then you are absolutely doing something wrong ;) -- at least that is what you will walk away believing if you read all the articles and the discussion around them.

But in all seriousness, if you haven't looked at TensorFlow, you should give it a spin. I remember when I first wrote a neural network back in the days, it took me a good chunk of a semester and I needed to understand the nitty-gritty details of propagation (forward and backward), activation functions, convolutions, etc. -- uggh!! TensorFlow lets you create a multi-layer deep neural network literally with a handful of lines of code.

We, at Amol Solutions, have been using TensorFlow in a couple of products areas that allow us to model assets in very unique ways. It allows us to understand what "normal" behavior is for an asset and when it deviates from such a state. While working with the team, someone commented on how it took them a considerable amount of time to filter through the online tutorials just to understand what a minimal TensorFlow program might look like.

So, we decided to create YATT ;) (yet another TensorFlow tutorial). The idea is to try and create the bare minimum functional program that does some sort of "learning". And what could be simpler than linear regression. You will need a working installation of TensorFlow. It is worth setting it up locally to try out the various models on a small data set. You can follow the instructions here to set it up. For any real world work, you would need to play either in the cloud (e.g Google Compute Engine) or get a system with a decent GPU. I would recommend the GTX 1060 6GB as a starter GPU or the GTX 1080 TI if that fits in your budget and if you can buy it anywhere :).

Once you have TensorFlow running, you can cut and paste the following program into the Jupyter Notebook to run. So, here goes:

We have 10 values in our "x_set" which are the square footages of houses for sale. So, 2.517 = 2517 sqft.
We have the 10 corresponding values in the "y_set" which are the selling prices of houses. So, 369.900 = $369,900.
We want to find a line that best fits these set of points.

Why? Because that is what "learning" is. Basically:

We're letting TensorFlow learn the relationship between the X and Y values. In our case, the size of the house and the selling price.
In our case this can be represented by a line. i.e. we're learning the line that best fits this set of data points. See: Linear Equation.
Once you learn this line, you can "predict" the selling prices of the houses if you know the size or vice versa.

So how do we get TensorFlow to do this? Rather simply:

We define the linear model: y = mx + b and give it to TensorFlow.
Then, we define what is called a "loss" function. You can think of this as a mathematical function that represents the error in the prediction. And if we can minimize this function, then we've found the best fit. A common one used in statistical regression is Mean Squares. See: Loss Function, Mean Squared Error.
And finally, we define an "optimizer" which is just a way for TensorFlow to iteratively try and converge to the the least error. A very common one is called Gradient Descent which comes built in TensorFlow along with a bunch of other ones. See: Gradient Descent.
After this, we just let TensorFlow iterate through the dataset multiple times. It starts with random values for M and B and then tweaks them using the gradient descent algorithm till it finds the best fit.
That's it!

The program below prints out the final values it finds for M, B and loss. It also plots the original data points along with the line that represents the "learned" values. Give it a run and have fun. You can tweak the X and Y values to see how well the framework is able to find the best fit line. You can try the alternate variations of the "y_set" to provide more correlated values and see how many epochs it takes for the framework to "learn" it back. You can change the "epochs" to limit the number of iterations and you can see how lower numbers will produce more loss and the fit will not be as good.

Obviously, using TensorFlow for linear regression is overkill and there are much simpler ways to do that but the idea here was to introduce the framework in a simplistic way.

Cheers!!

# housekeeping: imports to use tensorflow and matplotlib to show the graph

%matplotlib inline

import tensorflow as tf

import matplotlib.pyplot as plt

import random

# equation of a line: y = mx + b, where m=slope and b=y-intercept (https://en.wikipedia.org/wiki/Linear_equation)

m = tf.Variable([0.0], dtype=tf.float32)

b = tf.Variable([0.0], dtype=tf.float32)

x = tf.placeholder(tf.float32)

linear_model = m * x + b

# define the loss function: mean squared (https://en.wikipedia.org/wiki/Mean_squared_error)

y = tf.placeholder(tf.float32)

squared_deltas = tf.square(linear_model - y)

loss = tf.reduce_mean(squared_deltas)

# define the optimizer: gradient descent (https://en.wikipedia.org/wiki/Gradient_descent)

optimizer = tf.train.GradientDescentOptimizer(0.01)

train = optimizer.minimize(loss)

# our dataset: x=square footage of houses in 1000s of sqft, y=price of the houses in 1000s of dollars

x_set = [2.517, 1.678, 2.614, 2.205, 2.052, 2.022, 2.432, 2.276, 2.413, 1.941]

y_set = [369.900, 341.000, 368.500, 270.000, 350.900, 317.900, 354.999, 315.999, 280.000, 299.500]

# y_set = [80*x for x in x_set]

# y_set = [100*x+random.randint(-10,10) for x in x_set]

# y_set = [85*x+10 for x in x_set]

# number of training runs

epochs = 2000

# launch tensorflow in a session

with tf.Session() as sess:

    sess.run(tf.global_variables_initializer())

    # let tensorflow minimize the loss and find values of m and b that best fit the dataset

    for i in range(epochs):

        sess.run(train, {x: x_set, y: y_set})

    m_value, b_value, loss = sess.run([m, b, loss], {x: x_set, y: y_set})

# lets print out the final equation with the best values that tensorflow optimized

print("y = {}x + {}".format(repr(m_value[0]), repr(b_value[0])))

print("Loss: ", repr(loss))

# just plot the x and y sets along with the best fit line that tensorflow calculated

y_learned = x_set * m_value + b_value

plt.scatter(x_set, y_set)

plt.plot(x_set, y_learned, 'r' )

plt.show()

Report abuse