Hello, I am Xiao Wu 🧐

Last week we shared 10 fun tips for using Jupyter Notebook 👇

5 Tips for using the Notebook of Jupyter

5 Tips for using the Jupyter Notebook

Wouldn’t it be fun to spend a few minutes mastering some of these techniques to increase your productivity and save time?

Today is the third article, this article continues to introduce interesting and useful magic functions.

Magic command

What is a magic command?

The official definition is that IPython has a pre-defined set of so-called Magic Functions that you can access via command line syntax. These instructions are independent of Python syntax and can perform special functions.

There are two types of magic commands:

• Line magic command: prefix is “%”, and all instructions (including the main parameters) can not be newline.
• Cell magic command: prefix is “%%”, the entire cell is magic command, cell first line must be “%%”

To view all magic commands, type %lsmagic in the Cell, or if you can’t remember how to spell a function name. If you forget %lsmagic, ask a search engine…

There are two other functions that are commonly used to query magic commands:

• %quickref: Output a simple version of the help document for all magic commands
• %Magics_Name?: Prints a detailed help document for a magic command

I’ll start with a list of common magic functions and give a detailed demonstration of five of them.

Note: Due to IPython’s built-in magic function, this is not supported in Pycharm.

Calculate running time

Sometimes when we need to optimize code, we need to compare the running time of a function or procedure to measure the efficiency of the code.

There are several magic functions that can be implemented in Jupyter Notebook, each with different effects.

This way we can quickly get the runtime of the code to compare the optimization of the code.

View current variable

As you write line by line in the Jupyter Notebook, you’ll find yourself defining more and more variables, and it will be hard to remember which variables were defined in which cells.

Instead, we can use the magic command — %who_ls to see what variables have been defined.

Module aliases including imports are also included.

We can also specify variable types to view, for example we only want to see which lists have been defined so far:

%who_ls list
Copy the code

Save the cell contents to a file

We can save only part of the.py file in Jupyter Notebook. You need to use the magic command %%writefile, which saves the contents of the cell to an external file.

In the following demonstration, the code

message = "Hello World"
print(message)
Copy the code

Save as test.py file

Open the current working directory and you can see that the test.py file has been generated

To verify that the contents of the file are consistent, there is a magic command %pycat that displays the contents of the external file in a pop-up box. So we don’t have to leave Jupyter Notebook to look at the.py file.

Setting environment Variables

In machine learning/deep learning, we often encounter situations where environment variables are used.

In Jupyter Notebook, environment variables can also be set with the % magic command.

For example, %env or %set_env can also be used simply %env MY_VAR=MY_VALUE or %env MY_VAR MY_VALUE.

In addition, %env alone prints out the current environment variable.

%env THIS_IS_ENV_EXAMPLE "TEST2021"
Copy the code

Where THIS_IS_ENV_EXAMPLE is the environment variable and its value is the string TEST2021

Code share

For example, we’re running code on Jupyter Notebook and we’re having some problems and want to share the code. Should I take a screenshot or save it as a code file and then send it?

There’s another option — sharing code as a link.

%pastebin
Copy the code

This magic command uploads the code to Pastebin and returns a link. Pastebin is an online content hosting service where we can store plain text, such as snippets of source code, and share links with others.

We can also specify shared cells, for example:

%pastebin 1-4
Copy the code

This generates a private link URL for us to share the code with.

Open the link, you can get the following interface. Note that the generated links are only valid for 7 days.

Beep Beep: This kind of sharing still requires an Internet connection. Some of my friends in the company use machines that are not connected to the Internet, and it is always difficult to ask others to share their code. Recently, Xiaowu found that someone developed a method of passing code through two-dimensional code. It is very interesting. I have time to have a good understanding of it.

Ok, so today we introduced the magic commands in Jupyter Notebook, which make it easy to do things that pure Python would have trouble doing.

Limited to space, only introduced 5 interesting magic commands, % PDB code debugging are not detailed, we want to know more about the words can refer to the previous help document view. Stay tuned for the next installment, which continues with a wave of very useful plug-ins.

Don’t forget to give this article a thumbs up at 👍 if you find some tips to increase your productivity/fun

1111

---

---

Note that these commands work with the Python kernel, not with other kernels

What is magic function?

%matplotlib inline is a Magic function. As you can see, “%matplotlib inline” is a form unique to IPython that mimics the command line to access magic functions. Magic functions are divided into two kinds: one is line-oriented, the other is unit oriented. The line magic function is prefixed with “%”, much like the way we use the command line on systems, such as the Mac, where your username is followed by “$”. The “%” is followed by the arguments to magic, but the arguments are not passed in parentheses or quotes. The unit magic function is prefixed with two “%%” arguments, and takes arguments not only after the current “%%” line, but also after the current line.

%%writefile calls an external Python script %run Calls an external Python script %timeit Tests the execution time of a single line of statements %%timeit tests the execution time of code in an entire unit % matplotlib Inline displays matplotlib Package generated graphics %%writefile writefile % PDB debugger % PWD view the current working directory %ls view the list of directory files %reset clear all variables %who view the names of all global variables, if given a type parameter, Returns only a list of variables of that type. % whOS Displays all global variable names, types, values/information

Earlier, we introduced Jupyter Notebook as a practical and efficient application that provides an environment in which you can record code, run it, visualize data, and view results.

In my opinion, students of economics should also learn to use Python for data processing and statistical modeling, after all, software such as Excel and Stata are really limited in handling big data.

Of course, the reason why the artifact is a artifact is not only the various benefits mentioned above, but also many convenient tips in the specific use process, which contribute to improving your work efficiency and saving time. Here are some tips that you can use more frequently in your daily life.

Jupyter Notebook is a powerful Python interactive IDE beloved by data analysts and algorithm engineers. Jupyter Notebook has a great user experience in using a combination of text, code, images and other elements to show what a designer wants to do. And its own some commonly used Magic Command can make it more handy.

Magic functions mainly contain two categories, one is Line magic prefix is %, one is Cell magic prefix is %%;

The most commonly used magic functions are the following:

0

1

Fast calculation of running time

Sometimes we need to calculate the running time of a function or procedure to measure the efficiency of the code. In other ides, we might need to write a function or use a third-party module to do this

• %time: The time that code takes to run once in line mode
• %%time: The time it takes for code to run once in unit mode
• %timeit: In line mode, execute the code block several times to get the best result
• %%timeit: In unit mode, execute the code block several times to get the best result

This allows us to quickly get the running time of a block of code with just a few keystrokes

View current variable

As we write more and define more variables, it’s sometimes easy to forget which ones were named. It’s a pain to go back to the code, but with the Notebook, you can use %who_ls to see how many variables are currently defined

You can also specify variable types to view, such as which variables are strings

6. Jupyter Magic-%env: Set environment variables

You can manage the Notebook’s environment variables without restarting the Jupyter server process. Some libraries (such as Theano) use environment variables to control their behavior, and %env is the most convenient way.

In [55]:    # Running %env without any arguments



            # lists all environment variables



 



            # The line below sets the environment



            # variable OMP_NUM_THREADS



            %env OMP_NUM_THREADS=4
Copy the code

---

            env: OMP_NUM_THREADS=4
Copy the code

7. Jupyter Magic – %run: Run Python code

%run can run Python code in.py format — this is well known. Less well known is the fact that it can also run other Jupyter Notebook files, which is useful. Note: Using %run is different from importing a Python module.

In [56]:    # this will execute and show the output from



            # all code cells of the specified notebook



            %run ./two-histograms.ipynb
Copy the code

Magic command

At Jupyter Notebook, there are a number of magic Commands that can be very helpful in making your job easier and more efficient

Use %lsmagic to see all magic commands

For unfamiliar magic commands, it’s nice to select it and press Shift + TAB to view the help information

Setting environment Variables

In machine learning/deep learning, we often use environment variables. In Jupyter Notebook, environment variables can also be set and modified by using %env

%env THIS_IS_ENV_EXAMPLE "xugaoxiang.com"
Copy the code

Here, THIS_IS_ENV_EXAMPLE is the environment variable and its value is the string xugaoxiang.com

To set up environment variables in Jupyter notebook, just use % magic command, such as %env or %set_env, such as %env MY_VAR=MY_VALUE or %env MY_VAR MY_VALUE. (using %env alone can print out the current environment variables.)

Saves cell contents to a file and displays file contents in a cell

Use %%writefile to save the contents of a cell to an external file

Use % Pycat to display the contents of the external file as a pop-up box

Code debugging

With % PDB, you can debug in the notebook

Automatic pdb calling has been turned ON
---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
<ipython-input-10-869a75f9e69b> in <module>
      4     raise NotImplementedError()
      5 
----> 6 pick_and_take()

<ipython-input-10-869a75f9e69b> in pick_and_take()
      2 def pick_and_take():
      3     picked = np.random.randint(0, 1000)
----> 4     raise NotImplementedError()
      5 
      6 pick_and_take()

NotImplementedError: 

> <ipython-input-10-869a75f9e69b>(4)pick_and_take()
      2 def pick_and_take():
      3     picked = np.random.randint(0, 1000)
----> 4     raise NotImplementedError()
      5 
      6 pick_and_take()

ipdb> 
Copy the code

As you can see, in the ipdb > prompt after can input the corresponding instruction, concrete can reference links docs.Python.org/3.8/library…