Build a super fast computer for deep learning for less than $1,000: Continue the exploration of deep learning and cheap hardware!

[[192381]]

Yes, you can run TensorFlow on a $39 Raspberry Pi, and you can run it on an Amazon EC2 node with a GPU for $1 per hour. Yes, these options are probably more realistic than building your own machine. But if you’re anything like me, you absolutely want to build your own blazingly fast deep learning computer.

OK, a thousand bucks is a lot for a DIY project. But once you get the machine working, you can build hundreds of deep learning applications, from robots with augmented brains to art (at least that's how I justify spending the money). At worst, this machine easily beats that $2800 Macbook Pro. It uses a little more power, but everything else is better than the Macbook. Plus, this machine is easily upgradeable, which keeps it years ahead of the laptop.

I haven't built a computer since the 1980s. I'm also quite worried about spending hundreds of dollars and ending up with a bunch of parts that I can't put together myself (or if I do, they may not work properly). But I'm telling you, building a computer is possible! Plus, it's a lot of fun. *** You can have a general-purpose computer that can do predictions and is 20 times faster than a laptop.

Here is a shopping list and some detailed suggestions:

◇Motherboard

There are different form factors. Since I didn't want to use multiple GPUs, the smallest standard motherboard size was mini-ITX, which was enough for this project. My only requirement was to have a PCIe slot for the GPU and two DDR4 memory slots. I bought the ASUS Mini ITX DDR4 LGA 1151 B150I PRO GAMING/WIFI/AURA motherboard, which sells for $125 on Amazon. It also comes with a WiFi antenna. This is very convenient for using this computer in the basement.

◇Chassis

The case is not very important and is very cheap. Since the main customers of DIY computers are gamers, the shapes and colors of the cases are also varied. The size of the case should match the motherboard, so my case needs to support mini-ITX. I bought the Thermaltake Core V1 Mini ITX Cube case, which sells for $50 on Amazon.

◇Memory

I can't believe how cheap RAM is now. You need to buy DDR4 RAM that works with your motherboard (almost all you'll find online) and the prices are pretty much the same for everyone. I bought two 8GB sticks of Corsair Vengeance for $129.

I spent an extra $5 on this LED-lit RAM module because a review on Amazon said it was "a great choice for those who can't fit enough LEDs into their PC cases." If you're building a PC in your basement like I was and don't care about the aesthetics of the case, you'll have a hard time finding parts inside it.

◇CPU

I looked at some CPU comparisons online and thought that a slower CPU would be fine, since I don't have any tasks that require a fast CPU (except training neural networks, which I will use the GPU for). But I also can't stand a computer with a CPU that is three generations behind the rest of the computer. *** I bought an Intel I5-6600 CPU for $214.

Most CPUs you can find on Amazon are suitable. I didn't spend the extra $20 to buy an i5-6600K. It's the same as the i5-6600 except it can be overclocked. But trading system reliability for a 10% speed increase is crazy for me. I admit, though, that I regretted it a little after I started thinking about building my own computer. So who knows? Building a computer will change your lifestyle. Maybe you will regret less if you buy an overclockable CPU. But on the other hand, maybe eliminating unnecessary options will protect you from killing yourself.

◇Hard disk

I also can't believe how cheap hard drives are now. I bought a 1TB SATA hard drive for $50. SSDs are faster, but much more expensive. And deep learning programs are generally not I/O-intensive, because they usually pull a batch of data into memory and then calculate it for a long time. If you have a lot of file transfer tasks, or just want to ensure that your application runs much faster than your friend's Macbook, then I would buy an SSD, such as this Samsung 850 EVO 250GB 2.5-Inch SATA III Internal SSD, which costs $98 for 250G.

These drives made me realize how much Apple rips people off. For an extra 250GB of hard drive capacity on a Macbook Pro, you're paying $200!

◇Graphics card/GPU

Choosing a graphics card is the most important and difficult question. For almost all machine learning applications, you need an NVIDIA graphics card because only NVIDIA has the most important CUDA framework and CuDNN library, which all machine learning frameworks (including TensorFlow) rely on.

As a non-GPU expert, I find the terminology to be confusing, but here are the very basics.

Perhaps the most important metric for deep learning is the amount of memory on your graphics card. If TensorFlow cannot fit the model and the current batch of training data into the GPU's memory, it will fail and fall back to using the CPU, rendering the GPU useless.

Another important consideration is the graphics card architecture. Nvidia's latest architectures are called "Kepler," "Maxwell," and "Pascal." The main difference between these architectures is speed. For example, according to this benchmark, a Pascal Titan X is twice as fast as a Maxwell Titan X.

Most papers on machine learning use TITAN X graphics cards. They are good, but expensive, costing more than $1,000, even for older models. Most people who do machine learning and don’t have much of a budget use Nvidia’s GTX 900 series (Maxwell) or GTX 1000 series (Pascal).

To figure out a graphics card's architecture, you can look at Nvidia's extremely confusing naming system: 9XX cards use Maxwell, while 10XX use Pascal. But a 980 with a higher clock speed and more video memory might still be faster than a 1060.

You may have to learn a little bit about the different architectures of Nvidia GPUs. But the bottom line is that whether it is a 9XX or 10XX graphics card, it will be an order of magnitude faster than your laptop. Don't be confused by these options, if you haven't used a GPU yet, using it will be much faster than your current system.

I bought a GeForce GTX 1060 with 3GB of video memory for $195. It's nearly 20 times faster than my Macbook. But it still occasionally runs out of video memory for some applications. So I should probably spend the extra $60 and buy a GeForce GTX 1060 with 6GB of video memory.

◇Power supply

I bought a 650W power supply for $85. My experience is that power supply failures are very annoying and difficult to fix, so it's not worth saving the money. On the other hand, my system never has a peak load of more than 250 watts.

◇Radiator

I remember the first startup I worked for put their servers in a closet. Even with the closet door, the fans were ridiculously noisy. Those days seem to be long gone. But a 650W power supply makes your machine consume 10 times the energy of a laptop, and energy has to go somewhere. It may not be necessary, but I bought a super cooler for $35, the Master Hyper 212 EVO. It keeps the CPU running cool and the system super quiet.

◇Overview

To get it working properly, you'll also need a monitor, mouse, and keyboard. These are easy to find (I have them all over my basement). The total price is $883, so if you want to spend $1,000, you can still upgrade a few things.

Installation

I was pretty excited when all the packages arrived, then I found out that assembly was easy but unreliable. It took me about an hour to assemble the computer following the instructions for each component. The *** would not light up. I had to replace the motherboard, which was pretty frustrating but it was OK.

The second time I assembled it, I placed all the parts on a cardboard box and connected them to make sure everything worked properly.

Basically, if you plug everything in and it seems to fit properly, you should be fine.

Figure 1 A semi-finished computer on a table with only minimal components connected for testing

Figure 2: The computer looks scary after installing the super large radiator

Figure 3 Looking from above, the hard drive has been installed

Start your computer

Installing the latest version of Ubuntu will make your life much easier, as the new version supports almost all deep learning software. You can save an image to a USB drive and follow its simple step-by-step instructions to install it. In the 1990s, I had to fight with various drivers to install the Linux desktop, but now the installation process has become super easy.

The new Ubuntu operating system is also great. I now regularly use this computer that I built myself. With a lot of memory, a relatively fast CPU, and a lightweight operating system, it is the fastest computer in my house.

Install CUDA and OpenCV

and TensorFlow

In order to use your cool deep learning machine, you need to install CUDA and CudNN. The latest CUDA version is 8.0 and CudNN is 5.1. In the abstract, CUDA is an API and compiler that allows other programs to use the GPU for general applications. CudNN is a library designed to make neural networks run faster on the GPU. You need to download both software from the NVIDIA website.

OpenCV is an open source library that is used by many applications for image processing. The latest version 3.1 is not compatible with the latest CUDA version. You can make them compatible by setting the CUDA_GENERATION flag to Kepler, Maxwell or Pascal (depending on the GPU model you bought). Here is the sequence of commands to download OpenCV and install and set it up:

 git clone https://github.com/opencv/opencv.git \ 
 
 && cd opencv \ 
 
 && mkdir build \ 
 
 && cd build \ 
 
 && cmake .. \ 
 
 && make -j3 \ 
 
 && make install

***, installing TensorFlow has become the easiest task these days. Just follow the instructions on this website.

To see if GPU support is successful, you can run this TensorFlow test program, or execute the following command:

python -m tensorflow.models.image.mnist.convolutional

This should start running a model without errors.

Now comes the fun part!

Now that you've spent nearly $1,000 and countless hours setting up the machine and installing software, it's time to recoup your investment! GPUs make programs run incredibly fast, so you need some applications that can take advantage of that speed. Luckily there are a lot of fun things to do.

Identify nearby objects in real time

Install a cheap USB camera or a Raspberry Pi board with a camera outside your house. You can easily make a Raspberry Pi video streaming app using the RPi camera module I introduced in my previous $100 TensorFlow robot.

YOLO

The YOLO package can perform real-time object recognition while shooting. Using a Macbook, I found that object recognition took 3 to 4 seconds. But using a GPU, I can get a real-time result with very high accuracy.

It is easy to use the YOLO module to run on TensorFlow as described in this YOLO_tensorflow project. You can also install "Darknet", which is a different deep learning framework. YOLO was originally designed for it.

 git clone https://github.com/pjreddie/darknet 
 
 cd darknet 
 
 make

Once Darknet is installed, you can use the following command to let it process images:

 ./darknet detect cfg/yolo.cfg yolo.weights data/dog.jpg

Since the Raspberry Pi camera just puts the file on a web server, you can directly let Darknet connect to that file to do object recognition in the real-time stream. The following video shows me doing object recognition in the garage while a traffic jam is happening outside:

For your Raspberry Pi robot

An enhanced brain

In my previous article, TensorFlow Robot for $100, I found that it is possible to run and apply deep learning on hardware worth only $30. Unfortunately, the most disappointing thing for me was that it took several seconds for the robot to complete object recognition, so it was impossible for the robot to make real-time decisions. Now if the robot uses this new GPU machine, it can not only do real-time object recognition, but also process about 12-20 frames per second.

If you follow my instructions on GitHub, you can build a robot that gets a video stream from a camera that is easy to process.

My friend Shawn Lewis taught me most of the skills I needed for this article, so I took my two Raspberry Pi robots to his office. He and many other roboticists have a dream of having robots that can bring them beer. Here is a video of these robots fighting for beer on the floor.

If you look closely at the computer monitor at the beginning of the video, you can see that Shawn's computer is actually using its GeForce 980 graphics card to do real-time object recognition for the videos shot by the two robots. He claims that his graphics card can handle four video inputs at the same time without overflowing the memory.

Make some art

One of the fun things you can do with neural networks is to replicate what Google's Deep Dream does. While you can do it without a GPU, it will take a long time. Essentially, it's all about modifying the input image to find the image that gets the best response from the neuron, so it's a very computationally intensive task. There are many ways to do this modification, and the results are usually very weird and cool.

A good tutorial that can be used directly is the Deep Dream code released by Google.

You will need to install the Jupyter notebook server (you always need this!) and Caffe.

Then follow Google's tutorial to insert your friend's face. With this new machine, the new image comes out in minutes instead of hours. The modified pictures are fun to play with.

Figure 4. My neighbors Chris Van Dyke and Shruti Gandhi standing in my garage, with the photo scribbled by my Deep Dream program.

[[192386]]

Figure 5 My friend Barney Pell and his chess birthday cake

Figure 6 This is a picture of the machine I ran the Deep Dream program on. It looks like the whole screen is full of dogs (probably because the ImageNet data used to train this model is full of dog pictures. The ImageNet dataset includes more than 120 different dog species)

If you want to do something even crazier, here is Neural Style implemented in TensorFlow based on Deep Dream. The result is even more amazing, you can see some of the results in this mind-blowing blog post.