What if you want to run TuriCreate but don’t have a MAC or Linux experience? This cloud-based application lets you run Python without having to install it. Deep learning is easy to play with a high-performance GPU without spending a cent.
Pain points
How to Recognize Images with Python and Deep Neural Networks? The article received many comments from readers after it was published. Everyone is interested in the unattainable deep neural network image recognition from the past, and everyone wants to try out the easy-to-use TuriCreate framework.
Some readers are happy to try it.
Some readers had a problem:
I wrote in how to Find Approximate Images with Python and Deep Neural Networks. TuriCreate currently supports a limited number of operating systems, including the following options:
If you’re running Windows 7 or below, TuriCreate isn’t currently supported.
There are two solutions:
First, upgrade to Windows 10 and use WSL.
Second, install Linux on a virtual machine.
Are these two solutions good?
Not good.
They are all solutions without solutions.
Because both require users to be exposed to the new Linux system.
Linux is really a good thing for IT professionals.
First of all, it’s free. Therefore, the comprehensive use cost of hardware and software can be minimized.
Second, it is flexible. You can customize everything from your system kernel to your applications. Unlike Windows or macOS, Spanish and text-to-speech are installed by default, regardless of whether you use it or not.
Third, it is strong. Linux, though free, was created with UNIX as its reference and is perfectly capable of running on large servers that shut down once a year.
But the advantages of Linux are not necessarily such a good thing for the “liberal arts students” who are the main readers of my column.
Because Linux has a steep learning curve.
So if you’re adamant about running TuriCreate on your computer, but don’t want to learn Linux, you might want to buy a Macbook.
But who says you have to run your code on your own machine?
The cloud
You can install TuriCreate in the cloud — as long as the cloud host is Linux.
You may be angry that I’m kidding you – if I knew how to use Linux, I’d just install it locally! You want me to use Linux remotely when I don’t know how to use Linux locally? ! What do you mean?
Don’t worry. Just hear me out.
Linux hosts in the cloud mostly just give you an operating system on which you can install software and execute commands.
Such cloud systems often require considerable IT expertise to navigate.
What’s more, this kind of rented cloud host is either very weak or very expensive.
The mainframe, which costs tens of dollars a month, often has only one CPU core. Running deep learning programs? In case you haven’t got the results, other people’s papers are out.
Is there a high-performance mainframe? B: of course.
Amazon’s AWS, for example, offers configurations like p2.xlarge. It makes running deep learning tasks easy.
But it’s expensive.
It has been calculated that if you need to use deep learning for the long term, it might be cheaper to build a high-performance computer locally.
Plus, amazon has already done a lot of prep work for you. You start to toss this cloud host to really master the use, or take some effort.
After finding a good tutorial online, practicing, and asking for help, you’ll eventually master the following skills:
- Hardware configuration meaning;
- Cloud platform credit card payment method;
- Control panel use;
- Accounting principle;
- Bidding rules;
- Instance usage restriction;
- Custom instance type selection;
- Safety rule setting;
- Use of public and private keys;
- Encrypted communication SSH connection;
- File permission setting;
- Other……
You learned how to run highly configured AWS virtual hosts most cheaply and when to start and shut down instances. At the end of the month, you’ll feel a great sense of accomplishment when you see how little your bill is worth.
The question is, what did you want to do in the first place?
I think you’re just going to make a quick classification model of the photos you have, using the convolutional neural network on TuriCreate?
So this is not the way to go.
At some point, making the right choice is more important than blindly trying.
You should choose a cloud platform that has the following features:
You don’t have to know Linux or install basic software from scratch. Turn it on and you can use it. If you need any extra functionality, you can do it with one command. Providing high-performance Gpus to run deep learning code… It better be free.
Do you think I’m dreaming? Wondering if you should wake me up?
It’s not a dream. It’s a real thing.
found
The cloud environment recommended for you in this article is Colaboratory (hereinafter referred to as Colab) provided by Google.
This tool has been around for years.
The original version was developed by the team of Google and Jupyter. It’s only recently that the iteration has gotten better. After the promotion of this Medium article, it has attracted the attention of many researchers and learners.
The official introduction is:
Colaboratory is a research tool for machine learning training and research. It is a Jupyter notebook environment that does not require any setup to use.
Open this link in Google Chrome and you can see the “Introduction to Colaboratory”.
It looks different, but it’s actually a Jupyter Notebook.
Let’s try to run one of these statements.
Notice that the statements in this notebook are actually in Python 2 format. The default laptop, however, is Python 3.
So, if you execute the first sentence directly (again with Shift+Enter), an error will be reported.
The solution is as simple as opening the Code Executor TAB in the upper toolbar.
Select Change Runtime Type at the bottom.
After changing the default Python 3 to Python 2, click the save button in the lower right corner.
Then we rerun the statement for the first code block. This time it will output normally.
Statement block 2 is even more interesting. It calls directly on Google’s own deep learning framework, tensorFlow.
I wrote a tutorial specifically for the installation of TensorFlow. But here, you don’t have TensorFlow installed, and it actually works for you.
Not only TensorFlow, but many common data analysis toolkits such as Numpy and Matplotlib are installed by default.
For these basic tools, you don’t need to install, configure, manage, just take them and use them.
We run the last unit of code.
Look, the picture output has no problem.
Plug and play in the field of programming environments! That’s great!
But after the excitement, you might think that’s all — these packages have been installed correctly on my local machine. Execution, after all, is more convenient local ah.
That’s right.
But when you install TuriCreate, your Windows operating system doesn’t support it, right?
Let me show you how to run TuriCreate with Colab for deep learning.
data
I put the image data and ipynb files that need to be classified into this Github project. Please click this link to download the package.
Download it and unzip it to a local hard disk.
As you can see, it contains an ipynb file and an image directory.
The image directory is what you wrote in “How to Find Similar Images with Python and Deep Neural Networks?” A picture of Doraemon and WALL · E already seen in this article.
Here’s a picture of the blue Fat man:
Here’s a picture of WALL · E:
Please open this link in Google Chrome (Colab does not currently support other browsers) to open your Google Drive.
Of course, if you don’t already have a Google account, you need to sign up and use it.
Next, drag the folder you just unzipped to the Google Drive page and the system will automatically upload it for you.
After uploading, open the folder in Google Drive.
Right-click on the demo_PYTHon_image_classification. Ipynb file. Select Colaboratory as the opening mode.
The ipynb file opened by Colab is shown in the following figure.
We first need to determine the operating environment. Click “Modify” in the menu bar and select “Laptop Settings”.
Ensure that the runtime type is Python 2 and the hardware accelerator is GPU. If this is not the case, modify it. Then click Save.
The data is available and the environment is configured. Now we’re ready to run the code.
code
We tried to read in the TuriCreate package.
import turicreate as tc
Copy the codeThe result is the following error.
It is normal to encounter this error.
Because we haven’t installed TuriCreate yet.
Don’t you need to install a deep learning framework?
Well, that depends on whose deep learning framework it is.
Colab installs Tensorflow by default because it is Google’s own deep learning framework.
TuriCreate is an Apple product, so we need to install it manually.
Is manual installation troublesome?
You don’t.
Open a new code unit and type the following statement:
! pip install turicreateCopy the codeYou can watch Colab painstakingly install TuriCreate and all dependencies without having to worry about it yourself. It was installed in a flash.
Let’s call TuriCreate again.
import turicreate as tc
Copy the codeThis successful execution, no error.
The next thing we need to do is allow Colab to read from our data folder.
By default, Colab had no idea where our data folder was — even though we had opened the ipynb file from under the Google Drive demo folder.
We first need Colab to find the root of Google Drive.
This should have been a relatively complicated problem. But the good news is that we have code out there that we can use.
Execute the code for the cell below. Don’t worry if you don’t understand. Because you don’t need to adjust any of the statements.
! apt-get install -y -qq software-properties-common python-software-properties module-init-tools ! add-apt-repository -y ppa:alessandro-strada/ppa2> &1> /dev/null ! apt-get update -qq2> &1> /dev/null ! apt-get -y install -qq google-drive-ocamlfuse fusefrom google.colab import auth
auth.authenticate_user()
from oauth2client.client import GoogleCredentials
creds = GoogleCredentials.get_application_default()
importgetpass ! google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret} < /dev/null2> &1| grep URL vcode = getpass.getpass() ! echo {vcode} | google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret}Copy the codeAt the beginning of execution, you should see the following running state.
After a while, you’ll notice that the program stops. Give you a link to click on. And ask you to fill in the text box below.
Click on the link and you’ll see the image below.
Click on your Own Google account.
You will then be prompted for the Google Cloud SDK permission request.
Click Allow and you get a long string of characters. Copy them.
Go back to the Colab page, paste in this long string of characters, press Enter.
You might think it’s done. No, there’s a second step.
Another link came up.
Once you click on it, you’ll still be asked to select your account.
Then the Google Cloud SDK came up with permission requirements. Note that the number of permissions requested is not the same as last time.
You need to copy another string of characters.
Paste it back, enter. This time the execution was finally completed.
Well, now Colab has taken over your Google Drive. Let’s call the root directory of Google Drive Drive Drive.
! mkdir -p drive ! google-drive-ocamlfuse driveCopy the codeWe then told Colab to set our current working directory to the Demo-python-image-classification -Google-colab-master folder under Google Drive.
import os
os.chdir("drive/demo-python-image-classification-Google-colab-master/")
Copy the codeAll right, all set up. Let’s move on.
We need to tell TuriCreate where the image data folder is.
img_folder = 'image'
Copy the codeThen, we read the entire image file into the data box data.
data = tc.image_analysis.load_images(img_folder, with_path=True)
Copy the codeHere, you’ll notice that it’s slow to read. This is a real problem, is it because TuriCreate’s SFrame data box is a bit uncomfortable on Colab? I’m not sure at the moment.
Fortunately, the total number of files in our sample is small and acceptable.
Finally finished reading.
Let’s take a look at what data is contained in data.
data
Copy the codeThe result is the same as the local result of the Jupyter Notebook, with the file path, and the size of the image information.
Now, let’s label the image again.
From the Doraemon folder, labeled Doraemon; Otherwise, mark it as Walle.
data['label'] = data['path'].apply(lambda path: 'doraemon' if 'doraemon' in path else 'walle')
Copy the codeLet’s look at the data data box.
data
Copy the codeAs you can see, the mark has been successfully marked.
We tried using explore() to navigate the data box and view the images.
data.explore()
Copy the codeUnfortunately, TuriCreate informs us that this feature is currently only available on macOS.
Remember, we are using a Linux operating system, so explore() cannot be used properly. But this is only temporary, may someday support.
Fortunately, this feature has little to do with our task of sorting images. Let’s move on.
The data were divided into training sets and test sets, and uniform random seed values were used to ensure that the results we obtained could be repeatedly verified.
train_data, test_data = data.random_split(0.8, seed=2)
Copy the codeNow we formally build and train the model.
model = tc.image_classifier.create(train_data, target='label')
Copy the codeWhen you run it, you will find that the pre-training model parameters, which should take a long time, can be downloaded in an instant.
What’s going on here? As a thought question, I leave it to you to explore the solution. Here’s a hint: cloud storage.
TuriCreate automatically helped us deal with image size normalization and went through several iterations to find suitable hyperparameter Settings.
All right, let’s try to make a prediction on the test set using the model generated by the training set.
predictions = model.predict(test_data)
Copy the codeWhat are the predictions? Let’s test this with the evaluate() function.
metrics = model.evaluate(test_data)
print(metrics['accuracy'])
Copy the codeThe results are as follows:
0.935483870968
Copy the codeLet’s look at the predictions:
predictions
Copy the codedtype: str
Rows: 31
['doraemon'.'walle'.'walle'.'walle'.'walle'.'walle'.'doraemon'.'doraemon'.'doraemon'.'walle'.'walle'.'walle'.'doraemon'.'doraemon'.'walle'.'doraemon'.'doraemon'.'doraemon'.'doraemon'.'walle'.'walle'.'doraemon'.'doraemon'.'doraemon'.'walle'.'doraemon'.'doraemon'.'doraemon'.'doraemon'.'doraemon'.'walle']
Copy the codeNow look at the actual tag:
test_data['label']
Copy the codedtype: str
Rows: 31
['walle'.'walle'.'walle'.'walle'.'walle'.'walle'.'doraemon'.'doraemon'.'doraemon'.'doraemon'.'walle'.'walle'.'doraemon'.'doraemon'.'walle'.'doraemon'.'doraemon'.'doraemon'.'doraemon'.'walle'.'walle'.'doraemon'.'doraemon'.'doraemon'.'walle'.'doraemon'.'doraemon'.'doraemon'.'doraemon'.'doraemon'.'walle']
Copy the codeBy contrast, we wanted to find out where the wrong predictions were stored:
test_data[test_data['label'] != predictions]['path']
Copy the code
Now we need to take a quick look at the image of the wrong prediction.
Read into the Image module of Jupyter for displaying images.
from IPython.display import Image
Copy the codeLet’s start with the first image:
Image(test_data[test_data['label'] != predictions]['path'] [0])
Copy the codeThe results are as follows:
As always, the 50-layer deep neural network model is no longer intuitive. So we can’t figure out exactly which part of the decision went wrong.
At first glance, however, it turns out that wall-E doesn’t dominate the picture at all. Instead, round-headed, round-brained robots took the lead. In this way, the picture formed a more serious noise.
Let’s look at another picture:
Image(test_data[test_data['label'] != predictions]['path'] [1])
Copy the codeThe result is this:
In this picture, there is also a lot of interference information, and even Doraemon is a pirate cosplay.
Well, at this point, our code migration to Colab is complete.
As you can see, we don’t need to install any packages locally. TuriCreate deep learning is (almost) complete with a browser and a folder downloaded from Github.
Isn’t it easier than installing Linux on a virtual machine or setting up your own cloud Linux host?
summary
Hopefully, you’ve already mastered the following:
- Some deep learning frameworks, such as TuriCreate, have platform dependencies;
- In addition to locally installed development environments, cloud platforms are also an option;
- When choosing a cloud platform, pay special attention to the ease of setting and cost performance;
- How to migrate data and code to Colab via Google Drive;
- How to install missing packages in Colab;
- How to get Colab to find the data file path.
Also, when choosing tools for your needs, remember the classic words of Harvard marketing professor Theodore Levitt:
People don’t really want to buy a quarter-inch drill. All they want is a 1/4 inch hole.
This phrase is not only useful for learners and developers.
For the product provider, the significance is even more significant.
discuss
Did you install TuriCreate correctly? What operating system are you using? Have you ever tried running Python code in the cloud? Is there a better environment to run code in the cloud than Colab? Welcome to leave a message, share your experience and thinking to everyone, we exchange and discuss together.
If you like, please give it a thumbs up. You can also follow and top my official account “Nkwangshuyi” on wechat.
If you’re interested in data science, check out my series of tutorial index posts entitled how to Get started in Data Science Effectively. There are more interesting problems and solutions.