Dynamic execution of Python code

As a “dynamic” language, Python is much more convenient to load a piece of code and execute it at run time than a “static language” (such as C or Java) that requires compilation.

Implement way

There are two simple types: exec and eval, depending on whether a result is returned or not.

exec

Exec is responsible for executing string code, can support multiple lines, can define variables, but cannot return results

def pr(x):
    print('My result: {}'.format(x))


if __name__ == "__main__":
    s = '''
a = 15
b = 3
if a > b:
    pr(a+b)
'''
    exec(s)

> My result: 18

eval

Eval can return a result, but only a one-line expression can be executed

def select_max(x, y):
    return x if x > y else y


if __name__ == "__main__":
    a = 3
    b = 5
    c = eval('select_max(a , b)')
    print("c is {}".format(c))

> c is 5

Runtime environment

Can be seen from the above code sample, whether the exec or eval, their operating environment, just like code calls their position: whether it is a global function, or local variables, as long as specified in the execution code defined before, you can use, and variables defined in the exec, references can also be at the back of the code.

If necessary, we can also add or mask information by specifying the content of the environment definition while running the dynamic code. Both exec and evel have more than one argument, and their second and third arguments can specify the globals and locals environment for dynamic code, respectively.

Globals is the global environment in which the code is executed. It can be obtained through the globals() function. The result returned is a dict, which lists all global variables and global methods, including modules and methods imported with import. Similarly, the locals() function returns all local variables and local methods.

When we call dynamic code, if we pass arguments like this:

def select_max(x, y):
    return x if x > y else y

c = eval('select_max(3 , 5)', {}, {})

The default globals(second parameter) and locals(third parameter) Settings will be overridden and the buildin method will be used and the above code will report an error because the select_max method cannot be found. For this method to work, we need to assign a value to one of these endings:

def select_max(x, y):
    return x if x > y else y

c = eval('select_max(3 , 5)', {'select_max':select_max}, {})

This looks a bit like farting out of your pants: why overwrite it and assign it again when you can just use it? It’s actually for security reasons.

security

Dynamic code capabilities, which are usually exposed to the outside of the program, allow configurators to extend the program logic. But if left unchecked, this ability can be dangerous: by calling the open method, for example, you can open any file and delete its contents.

So it is safer to disable the built-in functions and expose only the methods that allow external calls:

def select_max(x, y):
    return x if x > y else y

c = eval('select_max(3 , 5)', {"__builtins__": {}}, {'select_max':select_max})

Note: There are many articles on the web stating that __builtins__ is set to NONE. In fact, at least in a Python 3.7 environment, it is not possible and should be set to an empty dictionary

To optimize the

compile

In both of the above examples, strings are executed directly. As you can certainly imagine, these strings will be “parsed” by the Python runtime before they can be executed, and the parsing process can be time-consuming. Therefore, in order to optimize the efficiency, you can use the compile function first, in advance “compile” good, the string into “code”, each time the efficiency of the execution will be greatly improved.

def select_max(x, y):
    return x if x > y else y


if __name__ == "__main__":
    exp = compile('select_max(a , b)', '', 'eval')
    for i in range(10):
        a = i
        b = i + 10
        c = eval(exp)
        print("c is {}".format(c))

As you can see, after we compile a string, we turn it into an expression, and then calling that expression in a loop over and over again is much more efficient than parsing a string each time. Similarly, what exec executes can be first compiled into an expression using compile.

Note: The second argument to compile is the filename (you can read the code directly from the file). If not, it can be left blank

Compiled environment

Compile and eval/exec may not be called in the same function, so they have different execution environments. In fact, compile does not check the environment, and variables or methods used in dynamic code may not exist at compile time. For example, change the above code to look like this:

exp = compile('select_max(a , b)', '', 'eval')


def select_max(x, y):
    return x if x > y else y


if __name__ == "__main__":
    for i in range(10):
        a = i
        b = i + 10
        c = eval(exp)
        print("c is {}".format(c))

Before the select_max method is defined, the expression is compiled without any effect:

c is 10
c is 11
c is 12
c is 13
c is 14
c is 15
c is 16
c is 17
c is 18
c is 19