This article is participating in Python Theme Month. See the link to the event for more details
Yuan class background
Let’s first look at one way to implement the singleton pattern in Python:
class Singleton(type) :
def __init__(cls, *args, **kwargs) :
cls._instance = None
super().__init__(*args, **kwargs)
def __call__(cls, *args, **kwargs) :
if cls._instance is None:
cls._instance = super().__call__(*args, **kwargs)
return cls._instance
class Test(metaclass=Singleton) :
def test(self) :
print("test")
test1 = Test()
test2 = Test()
print(id(test1), id(test2))
# 4307114640, 4307114640,
Copy the codeTest results, it is obvious that the function is implemented, two instances of the corresponding object for the same:
>>> test1 = Test()
.test2 = Test()
>>> test1
<__main__.Test object at 0x10aa13d68>
>>> test2
<__main__.Test object at 0x10aa13d68>
>>> id(test1)
4473306472
>>> id(test2)
4473306472
Copy the codeYes, the way to do this is to control the instantiation of aclass with a metaclass.
Metaclass secrets
Now that we have a basic understanding of metaclasses, let’s take a look at what metaclasses are.
1. Recognize metaclasses
Type can be used in two ways:
type(object)Gets the object type.type(name, bases, dict)Create class objects.
>>> class A(object) :
. pass
.
>>> A
<class '__main__.A'> > > >A()"__main__.A object at 0x108a4ac50>
>>> type("B", (object.), dict())
<class '__main__.B'> > > >type("B", (object.), dict()) ()"__main__.B object at 0x108a0eba8>
Copy the codeAs you can see from the above example, type can initialize class objects in the same way as the class is defined. So type (metaclass) is a class/object initialization that can be implemented. In other words: “A metaclass is the class that created the class.”
>>> type(int)
<class 'type'> > > >type(str)
<class 'type'> > > >type(bytes)
<class 'type'>
Copy the codePython’s base data types all point to type. In Python, the built-in class type is a metaclass.
A new metaclass is defined in Python by providing the keyword argument metaclass to the class definition. Define and use metaclasses as follows:
class NewType(type) :
pass
class A(metaclass=NewType) :
pass
Copy the codeSo in Python we must distinguish between object and type. The two are not the same thing but have inextricably linked.
Here’s an interesting example from a wiki:
r = object
c = type
class M(c) : pass
class A(metaclass=M) : pass
class B(A) : pass
b = B()
Copy the codeThe most important thing is:
objectThe class is the ancestor of all classes.typeMetaclasses are the ancestors of all metaclasses.
That is, all objects (including type) are inherited from object. The type of all objects (including object) is derived from the type (metaclass). The following code looks like this:
>>> type(object)
<class 'type'> > > >isinstance(object.type)
True
Copy the codeFinally, post a picture to understand the relationship between the two:
The differences between python-types and objects are different. The differences between python-types and objects are different. There’s not much to discuss here.
2. Functions of metaclasses
Metaclasses can interfere with the creation of classes. For example, the Python standard library has a metaclass, ABC.ABCMeta, that defines abstract classes similar to Java’s abstract classes. Here’s how it works:
class Base(metaclass=abc.ABCMeta) :
@abc.abstractmethod
def read(self) :
pass
@abc.abstractmethod
def write(self) :
pass
class Http(Base, abc.ABC) :
pass
Copy the codeLet’s test the code:
>>> Base()
Traceback (most recent call last):
File "<input>", line 1.in <module>
TypeError: Can't instantiate abstract class Base with abstract methods read, write >>> Http() Traceback (most recent call last): File "", line 1, in
TypeError: Can'
t instantiate abstract class Http with abstract methods read.write
Copy the codeFound that the abstract class cannot be instantiated. In order to instantiate a class, a subclass must implement an abstract method. We modify the subclass code:
>>> class Http(Base, abc.ABC) :
.
. def read(self) :
. pass
.
. def write(self) :
. pass
.
. def open(self) :
. print(" open method ")
.
>>> Http().open(a)open method
Copy the codeYou can see that Http inherits Base and implements abstract methods without any problems.
3. Customize metaclasses
The following is how to implement caching of class objects.
When an object of a class is created, a cache reference to it is returned if the object was previously created using the same parameters.
import weakref
class Cached(type) :
def __init__(self, *args, **kwargs) :
super().__init__(*args, **kwargs)
self.__cache = weakref.WeakValueDictionary()
def __call__(self, *args) :
if args in self.__cache:
return self.__cache[args]
else:
obj = super().__call__(*args)
self.__cache[args] = obj
return obj
# Example
class Spam(metaclass=Cached) :
def __init__(self, name) :
print('Creating Spam({! r})'.format(name))
self.name = name
Copy the codeVerify the function:
>>> a = Spam('Guido')
Creating Spam('Guido')
>>> b = Spam('Diana')
Creating Spam('Diana')
>>> c = Spam('Guido') # Cached
>>> a is b
False
>>> a is c # Cached value returned
True
Copy the code4. Precautions
To quote Tim Peters, leader of the Python community:
Metaclasses are deep magic, and 99% of users shouldn’t even bother. If you’re trying to figure out if you need a metaclass at all, then you don’t need it. Those who actually use metaclasses know very well what they need to do and don’t need to explain why they use them at all. While metaclasses are powerful, they can change the behavior of a class when it is created. However, it is not recommended for beginners to use metaclasses too much in their code, it can complicate your code. More often than not, you can use class decorators to implement metaclass functionality.
In the previous step, we can use a simple function to implement caching class objects. Here’s how to do it in the normal way:
import weakref
_spam_cache = weakref.WeakValueDictionary()
class Spam:
def __init__(self, name) :
self.name = name
def get_spam(name) :
if name not in _spam_cache:
s = Spam(name)
_spam_cache[name] = s
else:
s = _spam_cache[name]
return s
Copy the codeSimilarly, our verification results:
>>> a = get_spam('foo')
>>> b = get_spam('bar')
>>> a is b
False
>>> c = get_spam('foo')
>>> a is c
True
Copy the codeIn fact, a very simple function can be implemented, but the metaclass way code is more elegant.
The practice of metaclasses in Python can well reflect the charm of dynamic languages. Metaclasses can be used to change some properties of objects and dynamically change objects during the execution of programs.
abc.ABCMeta
In fact, I have seen the implementation of ABC.ABCMeta for a long time, but I have never understood it. Recently suddenly saw an article, then suddenly enlightened, the following analysis of the specific implementation.
Definition of the Python ABC metaclass
First take a look at two tools that are used to define abstract classes
@abc.abstractmethodThe source code.
def abstractmethod(funcobj) :
"""A decorator indicating abstract methods. Requires that the metaclass is ABCMeta or derived from it. A class that has a metaclass derived from ABCMeta cannot be instantiated unless all of its abstract methods are overridden. The abstract methods can be called using any of the normal 'super' call mechanisms. Usage: class C(metaclass=ABCMeta): @abstractmethod def my_abstract_method(self, ...) :... "" "
funcobj.__isabstractmethod__ = True
return funcobj
Copy the codeThis decorator simply adds the __isabstractMethod__ field to the decorated function.
metaclass=abc.ABCMetaSource code, here only intercepts the use of the code block.
class ABCMeta(type) :
_abc_invalidation_counter = 0
def __new__(mcls, name, bases, namespace, **kwargs) :
cls = super().__new__(mcls, name, bases, namespace, **kwargs)
# Compute set of abstract method names
abstracts = {name
for name, value in namespace.items()
if getattr(value, "__isabstractmethod__".False)}
for base in bases:
for name in getattr(base, "__abstractmethods__".set()):
value = getattr(cls, name, None)
if getattr(value, "__isabstractmethod__".False):
abstracts.add(name)
cls.__abstractmethods__ = frozenset(abstracts)
# Set up inheritance registry
cls._abc_registry = WeakSet()
cls._abc_cache = WeakSet()
cls._abc_negative_cache = WeakSet()
cls._abc_negative_cache_version = ABCMeta._abc_invalidation_counter
return cls
Copy the codeABCMeta is initialized by caching the abstractmethod to cls.__abstractmethods__.
Initialization of the Cpython object
Cpython-typeobject. c (object_new) ¶ Cpython-typeobject. c (object_new) ¶
static PyObject *
object_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
if (excess_args(args, kwds)) {
if(type->tp_new ! = object_new) { PyErr_SetString(PyExc_TypeError,"object.__new__() takes exactly one argument (the type to instantiate)");
return NULL;
}
if (type->tp_init == object_init) {
PyErr_Format(PyExc_TypeError, "%.200s() takes no arguments",
type->tp_name);
return NULL; }}if (type->tp_flags & Py_TPFLAGS_IS_ABSTRACT) {
PyObject *abstract_methods;
PyObject *sorted_methods;
PyObject *joined;
PyObject *comma;
_Py_static_string(comma_id, ",");
Py_ssize_t method_count;
/* Compute ", ".join(sorted(type.__abstractmethods__)) into joined. */
abstract_methods = type_abstractmethods(type, NULL);
if (abstract_methods == NULL)
return NULL;
sorted_methods = PySequence_List(abstract_methods);
Py_DECREF(abstract_methods);
if (sorted_methods == NULL)
return NULL;
if (PyList_Sort(sorted_methods)) {
Py_DECREF(sorted_methods);
return NULL;
}
comma = _PyUnicode_FromId(&comma_id);
if (comma == NULL) {
Py_DECREF(sorted_methods);
return NULL;
}
joined = PyUnicode_Join(comma, sorted_methods);
method_count = PyObject_Length(sorted_methods);
Py_DECREF(sorted_methods);
if (joined == NULL)
return NULL;
if (method_count == - 1)
return NULL;
PyErr_Format(PyExc_TypeError,
"Can't instantiate abstract class %s "
"with abstract method%s %U",
type->tp_name,
method_count > 1 ? "s" : "",
joined);
Py_DECREF(joined);
return NULL;
}
return type->tp_alloc(type, 0);
}
Copy the codeType ->tp_flags & Py_TPFLAGS_IS_ABSTRACT
A blind guess is a check on the initialization of an object, PyErr_Format(PyExc_TypeError, “Can’t instantiate abstract class %s “”with abstract method%s %U”, type->tp_name, method_count > 1 ? “s” : “”, joined); .
The logic is that Cpython will interfere with the creation of new objects based on the parameters set to methods in Python.
Reference documentation
- Liao Xuefeng uses metaclasses
- Use CPython to summarize the relationship between Types and objects in Python
- what-are-metaclasses-in-python
- What is the relationship between Types and objects in Python