Moment For Technology

In-depth analysis of the Python metaclasses ABC | Python theme month

Posted on Oct. 14, 2023, 11:46 p.m. by Cynthia Norton
Category: The back-end Tag: python The back-end

This article is participating in Python Theme Month. See the link to the event for more details

Yuan class background

Let's first look at one way to implement the singleton pattern in Python:

class Singleton(type) :

    def __init__(cls, *args, **kwargs) :
        cls._instance = None
        super().__init__(*args, **kwargs)

    def __call__(cls, *args, **kwargs) :
        if cls._instance is None:
            cls._instance = super().__call__(*args, **kwargs)
        return cls._instance
class Test(metaclass=Singleton) :

    def test(self) :

test1 = Test()
test2 = Test()

print(id(test1), id(test2))
# 4307114640, 4307114640,
Copy the code

Test results, it is obvious that the function is implemented, two instances of the corresponding object for the same:

 test1 = Test()
.test2 = Test()
__main__.Test object at 0x10aa13d68
__main__.Test object at 0x10aa13d68
Copy the code

Yes, the way to do this is to control the instantiation of aclass with a metaclass.

Metaclass secrets

Now that we have a basic understanding of metaclasses, let's take a look at what metaclasses are.

1. Recognize metaclasses

Type can be used in two ways:

  • type(object)Gets the object type.
  • type(name, bases, dict)Create class objects.
 class A(object) :
.    pass
class '__main__.A'   A()"__main__.A object at 0x108a4ac50
 type("B", (object.), dict())
class '__main__.B'   type("B", (object.), dict()) ()"__main__.B object at 0x108a0eba8
Copy the code

As you can see from the above example, type can initialize class objects in the same way as the class is defined. So type (metaclass) is a class/object initialization that can be implemented. In other words: "A metaclass is the class that created the class."

class 'type'   type(str)
class 'type'   type(bytes)
class 'type'
Copy the code

Python's base data types all point to type. In Python, the built-in class type is a metaclass.

A new metaclass is defined in Python by providing the keyword argument metaclass to the class definition. Define and use metaclasses as follows:

class NewType(type) :
class A(metaclass=NewType) :
Copy the code

So in Python we must distinguish between object and type. The two are not the same thing but have inextricably linked.

Here's an interesting example from a wiki:

r = object
c = type
class M(c) : pass

class A(metaclass=M) : pass

class B(A) : pass

b = B()
Copy the code

The most important thing is:

  • objectThe class is the ancestor of all classes.
  • typeMetaclasses are the ancestors of all metaclasses.

That is, all objects (including type) are inherited from object. The type of all objects (including object) is derived from the type (metaclass). The following code looks like this:

class 'type'   isinstance(object.type)
Copy the code

Finally, post a picture to understand the relationship between the two:

The differences between python-types and objects are different. The differences between python-types and objects are different. There's not much to discuss here.

2. Functions of metaclasses

Metaclasses can interfere with the creation of classes. For example, the Python standard library has a metaclass, ABC.ABCMeta, that defines abstract classes similar to Java's abstract classes. Here's how it works:

class Base(metaclass=abc.ABCMeta) :

    def read(self) :

    def write(self) :

class Http(Base, abc.ABC) :
Copy the code

Let's test the code:

Traceback (most recent call last):
  File "input", line module
TypeError: Can't instantiate abstract class Base with abstract methods read, write  Http() Traceback (most recent call last): File "", line 1, in 
        TypeError: Can'
      t instantiate abstract class Http with abstract methods read.write

Copy the code

Found that the abstract class cannot be instantiated. In order to instantiate a class, a subclass must implement an abstract method. We modify the subclass code:

 class Http(Base, abc.ABC) :
.    def read(self) :
.        pass
.    def write(self) :
.        pass
.    def open(self) :
.        print(" open method ")
 Http().open(a)open method 
Copy the code

You can see that Http inherits Base and implements abstract methods without any problems.

3. Customize metaclasses

The following is how to implement caching of class objects.

When an object of a class is created, a cache reference to it is returned if the object was previously created using the same parameters.

import weakref

class Cached(type) :
    def __init__(self, *args, **kwargs) :
        super().__init__(*args, **kwargs)
        self.__cache = weakref.WeakValueDictionary()

    def __call__(self, *args) :
        if args in self.__cache:
            return self.__cache[args]
            obj = super().__call__(*args)
            self.__cache[args] = obj
            return obj

# Example
class Spam(metaclass=Cached) :
    def __init__(self, name) :
        print('Creating Spam({! r})'.format(name)) = name
Copy the code

Verify the function:

 a = Spam('Guido')
Creating Spam('Guido')
 b = Spam('Diana')
Creating Spam('Diana')
 c = Spam('Guido') # Cached
 a is b
 a is c # Cached value returned
Copy the code

4. Precautions

To quote Tim Peters, leader of the Python community:

Metaclasses are deep magic, and 99% of users shouldn't even bother. If you're trying to figure out if you need a metaclass at all, then you don't need it. Those who actually use metaclasses know very well what they need to do and don't need to explain why they use them at all. While metaclasses are powerful, they can change the behavior of a class when it is created. However, it is not recommended for beginners to use metaclasses too much in their code, it can complicate your code. More often than not, you can use class decorators to implement metaclass functionality.

In the previous step, we can use a simple function to implement caching class objects. Here's how to do it in the normal way:

import weakref
_spam_cache = weakref.WeakValueDictionary()

class Spam:
    def __init__(self, name) : = name

def get_spam(name) :
    if name not in _spam_cache:
        s = Spam(name)
        _spam_cache[name] = s
        s = _spam_cache[name]
    return s
Copy the code

Similarly, our verification results:

 a = get_spam('foo')
 b = get_spam('bar')
 a is b
 c = get_spam('foo')
 a is c
Copy the code

In fact, a very simple function can be implemented, but the metaclass way code is more elegant.

The practice of metaclasses in Python can well reflect the charm of dynamic languages. Metaclasses can be used to change some properties of objects and dynamically change objects during the execution of programs.


In fact, I have seen the implementation of ABC.ABCMeta for a long time, but I have never understood it. Recently suddenly saw an article, then suddenly enlightened, the following analysis of the specific implementation.

Definition of the Python ABC metaclass

First take a look at two tools that are used to define abstract classes

  • @abc.abstractmethodThe source code.
def abstractmethod(funcobj) :
    """A decorator indicating abstract methods. Requires that the metaclass is ABCMeta or derived from it. A class that has a metaclass derived from ABCMeta cannot be instantiated unless all of its abstract methods are overridden. The abstract methods can be called using any of the normal 'super' call mechanisms. Usage: class C(metaclass=ABCMeta): @abstractmethod def my_abstract_method(self, ...) :... "" "
    funcobj.__isabstractmethod__ = True
    return funcobj
Copy the code

This decorator simply adds the __isabstractMethod__ field to the decorated function.

  • metaclass=abc.ABCMetaSource code, here only intercepts the use of the code block.
class ABCMeta(type) :

    _abc_invalidation_counter = 0

    def __new__(mcls, name, bases, namespace, **kwargs) :
        cls = super().__new__(mcls, name, bases, namespace, **kwargs)
        # Compute set of abstract method names
        abstracts = {name
                     for name, value in namespace.items()
                     if getattr(value, "__isabstractmethod__".False)}
        for base in bases:
            for name in getattr(base, "__abstractmethods__".set()):
                value = getattr(cls, name, None)
                if getattr(value, "__isabstractmethod__".False):
        cls.__abstractmethods__ = frozenset(abstracts)
        # Set up inheritance registry
        cls._abc_registry = WeakSet()
        cls._abc_cache = WeakSet()
        cls._abc_negative_cache = WeakSet()
        cls._abc_negative_cache_version = ABCMeta._abc_invalidation_counter
        return cls
Copy the code

ABCMeta is initialized by caching the abstractmethod to cls.__abstractmethods__.

Initialization of the Cpython object

Cpython-typeobject. c (object_new) ¶ Cpython-typeobject. c (object_new) ¶

static PyObject *
object_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
    if (excess_args(args, kwds)) {
        if(type-tp_new ! = object_new) { PyErr_SetString(PyExc_TypeError,"object.__new__() takes exactly one argument (the type to instantiate)");
            return NULL;
        if (type-tp_init == object_init) {
            PyErr_Format(PyExc_TypeError, "%.200s() takes no arguments",
            return NULL; }}if (type-tp_flags  Py_TPFLAGS_IS_ABSTRACT) {
        PyObject *abstract_methods;
        PyObject *sorted_methods;
        PyObject *joined;
        PyObject *comma;
        _Py_static_string(comma_id, ",");
        Py_ssize_t method_count;

        /* Compute ", ".join(sorted(type.__abstractmethods__)) into joined. */
        abstract_methods = type_abstractmethods(type, NULL);
        if (abstract_methods == NULL)
            return NULL;
        sorted_methods = PySequence_List(abstract_methods);
        if (sorted_methods == NULL)
            return NULL;
        if (PyList_Sort(sorted_methods)) {
            return NULL;
        comma = _PyUnicode_FromId(comma_id);
        if (comma == NULL) {
            return NULL;
        joined = PyUnicode_Join(comma, sorted_methods);
        method_count = PyObject_Length(sorted_methods);
        if (joined == NULL)
            return NULL;
        if (method_count == - 1)
            return NULL;

                     "Can't instantiate abstract class %s "
                     "with abstract method%s %U",
                     method_count  1 ? "s" : "",
        return NULL;
    return type-tp_alloc(type, 0);
Copy the code


A blind guess is a check on the initialization of an object, PyErr_Format(PyExc_TypeError, "Can't instantiate abstract class %s ""with abstract method%s %U", type-tp_name, method_count 1 ? "s" : "", joined); .

The logic is that Cpython will interfere with the creation of new objects based on the parameters set to methods in Python.

Reference documentation

  • Liao Xuefeng uses metaclasses
  • Use CPython to summarize the relationship between Types and objects in Python
  • what-are-metaclasses-in-python
  • What is the relationship between Types and objects in Python
About (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.