Reflection is closely related to interfaces, which was the subject of our last article. Before we start the body, a little digression.

After the last article about Interface was published, it got a lot of attention and reading. For example, the first item in the daily news on GoCN:

Maybe the editor didn’t think this article could be called “deep decryption” and changed the title slightly

Top 48 hours of reading at Blogland:

I gained 150 favorites on the developer Toutiao APP (similar to Toutiao, but the content is technology-related and interesting), and was recommended to the most prominent banner position on the home page, with more than 1W views. I just don’t know whether this number is true, which is a little hard to believe.

Many students in the background told me that the article is too long, not conducive to reading, suggested to split. I quite understand that in the screen reading era, people need to read the full text quickly and get benefits. And code nongtaohuayuan articles are very long, it is difficult for readers to read in a short time, and obtain the corresponding benefits.

First of all thank you very much for your advice! Here’s what I think: People are saying that we are in an age of information overload, with too much information to read and a lot of anxiety. But, I mean, is there really that much good information out there? In my opinion, the quality of the articles is uneven, and many of them have no content or value. It is not worthwhile to waste time on such information. Because you read these articles, you don’t have the energy to read other good articles.

So, code farmers taohuayuan want to do a quality information source, to provide quality content. Every article has depth, content and harvest. It usually takes me about 2 weeks to write an article, which is more than half a month, compared with those days. Of course, just not in numbers. In this era, there is no shortage of quantity.

In addition, the length of the article is also a feature of mine. I can split it up, down, middle, etc., but I want to deliver all the valuable content to my readers at once. This way, you can concentrate on a text for an hour or more.

Wechat public number reading articles is sometimes not too convenient, here we recommend reading with wechat, directly search the name of the public number can see all the articles of the public number, very convenient. This feature has been around for a long time, and I just found out about it a while ago.

So, that’s it. I’m going to talk about reflection. Let’s get down to business.

What is reflection

Go straight to the definition on Wikipedia:

In computer science, reflection refers to the ability of a computer program to access, detect, and modify its own state or behavior during Run time. Reflection, figuratively speaking, is the ability of a program to “observe” and modify its behavior as it runs.

That begs the question: can’t it access, detect, and modify its own state and behavior at run time without reflection?

The answer to this question is to first understand what it means to access, detect and modify its own state or behavior. What is its nature?

In fact, it is the essence of the program at runtime to detect the type of object information and memory structure, without reflection can work? Can!!!! Using assembly language, dealing directly with the inner layer, what information cannot be obtained? However, when programming is migrated to a high-level language, it doesn’t work! The only way to achieve this skill is through reflection.

Reflection models vary from language to language, and some languages do not support reflection. The Go Language Bible defines reflection as follows:

The Go language provides a mechanism to update variables and check their values and call their methods at run time, but the exact types of these variables are not known at compile time. This is called reflection.

Why reflection

There are two common scenarios that require reflection:

  1. Sometimes you need to write a function, but you don’t know what type of argument is being passed to you. It may also be that many types are passed in and cannot be represented uniformly. That’s where reflection comes in.
  2. Sometimes you need to decide which function to call based on certain criteria, such as user input. This is where the function and its parameters need to be reflected, executing the function dynamically at run time.

Before we talk about how reflection works and how to use it, here are a few reasons not to use reflection:

  1. The code associated with reflection is often difficult to read. In software engineering, code readability is also a very important indicator.
  2. As a static language, the compiler can detect some type errors in advance during the coding process, but it cannot do anything to reflect the code. Therefore, including the code related to reflection is likely to run for a long time before errors occur, which is often a direct panic, which may cause serious consequences.
  3. Reflection still has a significant performance impact, running an order of magnitude or two slower than normal code. Therefore, for code that is critical to performance in a project, avoid using reflection features.

How is reflection implemented

The last article covered interface, which is a very powerful tool for abstraction in the Go language. When an entity type is assigned to an interface variable, the interface stores the type information of the entity. Reflection is based on the type information of the interface.

The Go language defines various types in the Reflect package and implements various functions for reflection that can detect type information and change type values at run time.

The types and the interface

In Go, each variable has a static type, which is determined at compile time, such as int, float64, []int, etc. Note that this type is the declared type, not the underlying data type.

Here’s an example from the Go blog:

type MyInt int

var i int
var j MyInt
Copy the code

Although the underlying type of I and j is int, we know that they are different static types and cannot appear on both sides of the equal sign unless we cast them. The static type of j is MyInt.

Reflection is primarily related to the interface{} type. The underlying structure of interfaces was explored in the previous article on interfaces, so let’s review it again.

type iface struct {
	tab  *itab
	data unsafe.Pointer
}

type itab struct {
	inter  *interfacetype
	_type  *_type
	link   *itab
	hash   uint32
	bad    bool
	inhash bool
	unused [2]byte
	fun    [1]uintptr
}
Copy the code

Itab consists of the specific type _type and interfaceType. _type represents a specific type, while interfacetype represents the interfacetype implemented by the specific type.

In fact, iFace describes a non-empty interface that contains methods; In contrast, eface describes an empty interface that contains no methods, and all types in the Go language “implement” an empty interface.

type eface struct {
    _type *_type
    data  unsafe.Pointer
}
Copy the code

Compared to iFace, eFace is relatively simple. Only one _type field is maintained, representing the specific entity type hosted by the empty interface. Data describes specific values.

I’m going to use the Go official reflection blog as an example. Of course, I’ll explain it in graphic detail, but it’s a little bit clearer when you combine the two. By the way, technical people should not be afraid of English materials. To become technical experts, reading English raw materials is a necessary way to improve your skills.

Just to be clear: an interface variable can store any variable that implements all the methods defined by the interface.

The most common ones in Go are the Reader and Writer interfaces:

type Reader interface {
    Read(p []byte) (n int, err error)
}

type Writer interface {
    Write(p []byte) (n int, err error)
}
Copy the code

Next, there are various conversions and assignments between interfaces:

var r io.Reader
tty, err := os.OpenFile("/Users/qcrao/Desktop/test", os.O_RDWR, 0)
iferr ! =nil {
    return nil, err
}
r = tty
Copy the code

Let’s first declare that r is of type IO.Reader, and notice that this is a static type of R, so its dynamic type is nil, and its dynamic value is nil.

Then, the statement r = tty changes the dynamic type of r to * os.file and the dynamic value to non-empty, indicating the open File object. At this point, r can be represented as

pairs:

.
,>
,>

*os.File also contains a Write function, which implements the IO.Writer interface. So the following assertion statement can be executed:

var w io.Writer
w = r.(io.Writer)
Copy the code

The reason we use assertions rather than direct assignments is because r’s static type is IO.Reader and does not implement the IO.Writer interface. The success of the assertion depends on whether r’s dynamic type meets the requirements.

Thus, w can also be represented as

, even though it is the same as W, but the function w can call depends on its static type io.writer, that is, it can only have the call form w.write (). The memory form of W is shown as follows:
,>

Only the function corresponding to fun changes compared to w: Read -> Write.

Finally, one more assignment:

var empty interface{}
empty = w
Copy the code

Since Empty is an empty interface, all types implement it, and w can be assigned to it without performing an assertion.

As you can see from the three figures above, the interface contains three parts of information: *data refers to the actual value of the actual type. Itab contains information about the actual type, including the size, package path, and various methods bound to the type (the methods are not shown).

At the end of this section, let’s review one of the techniques mentioned in the previous article on interface, and show it here again:

First refer to the source code and define a “disguised” iFace and eface structure respectively.

type iface struct {
	tab  *itab
	data unsafe.Pointer
}
type itab struct {
	inter uintptr
	_type uintptr
	link uintptr
	hash  uint32
	_     [4]byte
	fun   [1]uintptr
}

type eface struct {
	_type uintptr
	data unsafe.Pointer
}
Copy the code

Next, force the memory contents occupied by the interface variable to the type defined above and print it:

package main

import (
	"os"
	"fmt"
	"io"
	"unsafe"
)

func main(a) {
	var r io.Reader
	fmt.Printf("initial r: %T, %v\n", r, r)

	tty, _ := os.OpenFile("/Users/qcrao/Desktop/test", os.O_RDWR, 0)
	fmt.Printf("tty: %T, %v\n", tty, tty)

	// Assign r
	r = tty
	fmt.Printf("r: %T, %v\n", r, r)

	rIface := (*iface)(unsafe.Pointer(&r))
	fmt.Printf("r: iface.tab._type = %#x, iface.data = %#x\n", rIface.tab._type, rIface.data)

	// Assign w
	var w io.Writer
	w = r.(io.Writer)
	fmt.Printf("w: %T, %v\n", w, w)

	wIface := (*iface)(unsafe.Pointer(&w))
	fmt.Printf("w: iface.tab._type = %#x, iface.data = %#x\n", wIface.tab._type, wIface.data)

	// Assign a value to empty
	var empty interface{}
	empty = w
	fmt.Printf("empty: %T, %v\n", empty, empty)

	emptyEface := (*eface)(unsafe.Pointer(&empty))
	fmt.Printf("empty: eface._type = %#x, eface.data = %#x\n", emptyEface._type, emptyEface.data)
}
Copy the code

Running results:

initial r: <nil>, <nil>
tty: *os.File, &{0xc4200820f0}
r: *os.File, &{0xc4200820f0}
r: iface.tab._type = 0x10bfcc0, iface.data = 0xc420080020
w: *os.File, &{0xc4200820f0}
w: iface.tab._type = 0x10bfcc0, iface.data = 0xc420080020
empty: *os.File, &{0xc4200820f0}
empty: eface._type = 0x10bfcc0, eface.data = 0xc420080020
Copy the code

R, w, empty have the same dynamic type and value. I won’t explain it in detail, but I can see it very clearly with the previous picture.

Basic function of reflection

The Reflect package defines an interface and a structure, reflect.type and Reflect.value, which provide functions to retrieve Type information stored in the interface.

Reflect. Type mainly provides type-specific information, so it is closely associated with _type; Reflect. Value combines both _type and data, so programmers can get or even change the Value of the type.

The Reflect package provides two basic reflection functions to get the above interfaces and constructs:

func TypeOf(i interface{}) Type 
func ValueOf(i interface{}) Value
Copy the code

The TypeOf function is used to extract type information for values in an interface. Since its input argument is an empty interface{}, when this function is called, the argument is first converted to interface{}. In this way, the type, method set, and value of the argument are stored in the interface{} variable.

Take a look at the source:

func TypeOf(i interface{}) Type {
	eface := *(*emptyInterface)(unsafe.Pointer(&i))
	return toType(eface.typ)
}
Copy the code

The emptyInterface here is the same as the eface mentioned above (the field name is slightly different, the field is the same) and is in a different source package: the former is in the Reflect package and the latter is in the Runtime package. Eface. Typ is a dynamic type.

type emptyInterface struct {
	typ  *rtype
	word unsafe.Pointer
}
Copy the code

As for the toType function, it just does a type conversion:

func toType(t *rtype) Type {
	if t == nil {
		return nil
	}
	return t
}
Copy the code

Note that the return value Type is actually an interface that defines methods to get various types of information, while * rType implements the Type interface.

type Type interface {
    // All types can call the following functions

	// The number of bytes used when a variable of this type is aligned
	Align() int
	
	// If it is a struct field, the number of bytes used after alignment
	FieldAlign() int

	// Return the I 'th (passed argument) method in the set of methods of type
	Method(int) Method

	// Get the method by name
	MethodByName(string) (Method, bool)

	// Get the number of methods exported from the type method set
	NumMethod() int

	// Type name
	Name() string

	// Path of the return type, for example, encoding/base64
	PkgPath() string

	// the Sizeof the returned type, similar to the safe.sizeof function
	Size() uintptr

	// Returns a string representation of the type
	String() string

	// Return the type value of the type
	Kind() Kind

	// Whether the type implements interface u
	Implements(u Type) bool

	// Can I assign to u
	AssignableTo(u Type) bool

	// Can the type be converted to u
	ConvertibleTo(u Type) bool

	// Whether the type can be compared
	Comparable() bool

	// The following functions can be called only by specific types
	// Methods such as Key and Elem can only be called with Map type
	
	// The number of bits occupied by the type
	Bits() int

	// Return channel direction, can only be chan type call
	ChanDir() ChanDir

	// If the return type is mutable, it can only be a func call
	Func (x int, y... float64)
	// then t.isvariadic () == true
	IsVariadic() bool

	// Returns the internal child type, which can only be called by the types Array, Chan, Map, Ptr, or Slice
	Elem() Type

	// Returns the ith field of the struct type. This can only be a struct type call
	// If I exceeds the total number of fields, panic occurs
	Field(i int) StructField

	// Returns the fields of the nested structure
	FieldByIndex(index []int) StructField

	// Get the field by the field name
	FieldByName(name string) (StructField, bool)

	// FieldByNameFunc returns the struct field with a name
	// Returns a field whose name matches the func function
	FieldByNameFunc(match func(string) bool) (StructField, bool)// Get the first of the function typeiThe type of the parameterIn(i int) Type/ / returnmapkeyType, by type onlymapcallKey(a) Type/ / returnArrayThe length can only be determined by typeArraycallLen(a) int// Return the number of type fields, which can only be determined by typeStructcallNumField(a) int// Returns the number of input arguments of the function typeNumIn(a) int// Returns the number of values returned by the function typeNumOut(a) int// Return the first of the function typeiThe type of the valueOut(i int) Type// Returns the same part of the type structurecommon(a) *rtype// Returns different parts of the type structureuncommon(a) *uncommonType
}
Copy the code

As you can see, Type defines a number of methods through which you can get all the information about a Type. Be sure to go through all the above methods in their entirety.

Note that common, the penultimate method in the set of Type methods, returns the same Type of rtype as the _type in the previous article, and that the source code notes:

 // rtype must be kept in sync with .. /runtime/type.go:/^type._type.
Copy the code
type rtype struct {
	size       uintptr
	ptrdata    uintptr
	hash       uint32
	tflag      tflag
	align      uint8
	fieldAlign uint8
	kind       uint8
	alg        *typeAlg
	gcdata     *byte
	str        nameOff
	ptrToThis  typeOff
}
Copy the code

All types contain the rtype field, which represents common information for each type; In addition, different types contain some unique parts of themselves.

For example, arrayType and chanType contain rytPE, and arrayType contains slice, len, and other array-related information. The latter contains information that dir represents channel direction.

// arrayType represents a fixed array type.
type arrayType struct {
	rtype `reflect:"array"`
	elem  *rtype // array element type
	slice *rtype // slice type
	len   uintptr
}

// chanType represents a channel type.
type chanType struct {
	rtype `reflect:"chan"`
	elem  *rtype  // channel element type
	dir   uintptr // channel direction (ChanDir)
}
Copy the code

Note that the Type interface implements the String() function, which satisfies the fmt.stringer interface, so when using fmt.println, the output is String(). In addition, the fmt.printf () function, if %T is used as the format argument, outputs the result of reflect.typeof, which is the dynamic type. Such as:

fmt.Printf("%T".3) // int
Copy the code

With TypeOf functions out of the way, let’s look at the ValueOf function. The return Value reflect.Value represents the actual variable stored in interface{}, which can provide various information about the actual variable. Related methods often require a combination of type information and value information. For example, if you want to extract the field information of a structure, you need to use the field information about the structure held by the _type (in this case, structType) type, the offset information, and what *data points to — the actual value of the structure.

The source code is as follows:

func ValueOf(i interface{}) Value {
	if i == nil {
		return Value{}
	}
	
   / /...
	return unpackEface(i)
}

/ / eface decomposition
func unpackEface(i interface{}) Value {
	e := (*emptyInterface)(unsafe.Pointer(&i))

	t := e.typ
	if t == nil {
		return Value{}
	}
	
	f := flag(t.Kind())
	if ifaceIndir(t) {
		f |= flagIndir
	}
	return Value{t, e.word, f}
}
Copy the code

From the source, relatively simple: I is converted to the *emptyInterface type, and its TYP field and word field and a flag bit field are assembled into a Value structure. This is the return Value of ValueOf, which contains the pointer to the type structure, the address of the real data, and the flag bit.

The Value structure defines a number of methods by which you can directly manipulate the actual data pointed to by the Value field PTR:

// Set the len field for slices. If the type is not slice, panic will occur
 func (v Value) SetLen(n int)// Set slicingcapfieldfunc (v Value) SetCap(n int)// Set the dictionarykv
 func (v Value) SetMapIndex(key, val Value)// Returns index of slice, string, arrayiPlace the value of thefunc (v Value) Index(i int) Value// Get the internal field values of the structure by namefunc (v Value) FieldByName(name string) Value/ /...Copy the code

There are many other methods for the Value field. Such as:

// To get a value of type int
func (v Value) Int(a) int64// To get the number of struct fields (members)func (v Value) NumField(a) int// Try to send data to the channel (without blocking)func (v Value) TrySend(x reflect.Value) bool// Pass the parameter listincallvThe function (or method) represented by the valuefunc (v Value) Call(in []Value) (r []Value)// call a function with a variable lengthfunc (v Value) CallSlice(in []Value) []Value 
Copy the code

I’m not going to list them all, but there are a lot of them. Go to SRC /reflect/value.go and search func (v value) to see the source.

In addition, Interface, Type and Value can be connected through Type() and Interface() methods. The Type() method can also return Type information for a variable, equivalent to the reflect.typeof () function. The Interface() method restores Value to the original Interface.

Here is a picture from Lao Qian’s “Quick learn Go Language Lesson 15 – Reflection” :

TypeOf()
ValueOf()

Here’s a picture:

In the figure above, rtye implements the Type interface, which is a common part of all types. The emptyFace structure and the eface structure are the same thing, while the rtype and the _type are the same thing, except that some fields are slightly different. For example, the word field of emptyface and the data field of eface have different names, but the data type is the same.

The three laws of reflection

According to Go’s official reflection blog, there are three laws of reflection:

  1. Reflection goes from interface value to reflection object.
  1. Reflection goes from reflection object to interface value.
  1. To modify a reflection object, the value must be settable.

The first is the most basic: reflection is a mechanism for detecting types and values stored in an interface. This can be obtained via TypeOf and ValueOf functions.

The second is actually the opposite mechanism to the first, by reversing the return value from ValueOf into the Interface variable via the Interface() function.

Reflect. Type and reflect.Value can be converted to each other.

The third rule is a little trickier: if you need to manipulate a reflection variable, it must be settable. The essence of a reflection variable is that it stores the original variable itself, so that operations on a reflection variable are reflected in the original variable itself; Conversely, if the reflection variable does not represent the original variable, manipulating the reflection variable will have no effect on the original variable, which will cause confusion for the user. So the second case is not allowed at the linguistic level.

Take a classic example:

var x float64 = 3.4
v := reflect.ValueOf(x)
v.SetFloat(7.1) // Error: will panic.
Copy the code

Executing the above code causes panic because the reflection variable v does not represent x itself. Why? Because when reflect.valueof (x) is called, the argument passed inside the function is just a copy of the value passed, so v represents only a copy of x, so operating on v is prohibited.

Settable is a property of the reflection variable Value, but not all values are settable.

Just like in normal functions, when we want to change the variable passed in, we can use Pointers.

var x float64 = 3.4
p := reflect.ValueOf(&x)
fmt.Println("type of p:", p.Type())
fmt.Println("settability of p:", p.CanSet())
Copy the code

The output looks like this:

type of p: *float64
settability of p: false
Copy the code

P does not yet stand for x, p.lem () really stands for x, so you can actually manipulate x:

v := p.Elem()
v.SetFloat(7.1)
fmt.Println(v.Interface()) / / 7.1
fmt.Println(x) / / 7.1
Copy the code

For the third item, remember: if you want to manipulate the source variable, the reflection variable Value must hold the address of the source variable.

Use of reflection correlation functions

Code sample

There are a lot of sample code using reflection in various blog articles on the Internet. After reading this article, there is basically nothing I can’t understand, haha! But here’s an example:

package main

import (
	"reflect"
	"fmt"
)

type Child struct {
	Name     string
	Grade    int
	Handsome bool
}

type Adult struct {
	ID         string `qson:"Name"`
	Occupation string
	Handsome   bool
}

// If the input parameter I is Slice, the element is a structure, and there is a field named 'Handsome',
// And there is a field tag or field Name 'Name',
// If the value of 'Name' is' qcrao ',
// Set the value of the column named 'Handsome' to true.
func handsome(i interface{}) {
	// Get the reflection variable Value of I
	v := reflect.ValueOf(i)

	V is a Slice
	ifv.Kind() ! = reflect.Slice {return
	}

	// Make sure that the element v is a structure
	ife := v.Type().Elem(); e.Kind() ! = reflect.Struct {return
	}

	// Make sure the structure field name contains "ID" or json tag 'name'
	// Set the column name of the structure to "Handsome".
	st := v.Type().Elem()

	// Search for fields named Name or tag with the value Name
	foundName := false
	for i := 0; i < st.NumField(); i++ {
		f := st.Field(i)
		tag := f.Tag.Get("qson")

		if (tag == "Name" || f.Name == "Name") && f.Type.Kind() == reflect.String {
			foundName = true
			break}}if! foundName {return
	}

	if niceField, foundHandsome := st.FieldByName("Handsome"); foundHandsome == false|| niceField.Type.Kind() ! = reflect.Bool {return
	}

	// Set the "Handsome" field for the object named "qcrao" to true
	for i := 0; i < v.Len(); i++ {
		e := v.Index(i)
		handsome := e.FieldByName("Handsome")

		// Search for fields named Name or tag with the value Name
		var name reflect.Value
		for j := 0; j < st.NumField(); j++ {
			f := st.Field(j)
			tag := f.Tag.Get("qson")

			if tag == "Name" || f.Name == "Name" {
				name = v.Index(i).Field(j)
			}
		}

		if name.String() == "qcrao" {
			handsome.SetBool(true)}}}func main(a) {
	children := []Child{
		{Name: "Ava", Grade: 3, Handsome: true},
		{Name: "qcrao", Grade: 6, Handsome: false},
	}

	adults := []Adult{
		{ID: "Steve", Occupation: "Clerk", Handsome: true},
		{ID: "qcrao", Occupation: "Go Programmer", Handsome: false},
	}

	fmt.Printf("adults before handsome: %v\n", adults)
	handsome(adults)
	fmt.Printf("adults after handsome: %v\n", adults)

	fmt.Println("-- -- -- -- -- -- -- -- -- -- -- -- --")

	fmt.Printf("children before handsome: %v\n", children)
	handsome(children)
	fmt.Printf("children after handsome: %v\n", children)
}
Copy the code

Code running results:

adults before handsome: [{Steve Clerk true} {qcrao Go Programmer false}]
adults after handsome: [{Steve Clerk true} {qcrao Go Programmer true}]
-------------
children before handsome: [{Ava 3 true} {qcrao 6 false}]
children after handsome: [{Ava 3 true} {qcrao 6 true}]
Copy the code

The main thing the code does is to find out if the passed parameter is Slice and the Slice element is a structure, if there is a field Name of Name or a tag Name of Name, and another field Name is Handsome. If found, and the actual value of the field named Name is qcrao, set the value of the other field Handsome to true.

The program does not care what the structure passed in is, as long as its field Name contains Name and Handsome, both of which are objects that the Handsome function works with.

Note that the Adult structure qson:”Name” does not have Spaces in it, otherwise tag.get (“qson”) will not recognize it.

Unexported member

With reflection, unexported members of a structure can be read, but their values cannot be modified.

Note that normally, code cannot read unexported members of a structure, but reflection can override this restriction. Also, by reflection, the only members in the structure that can be modified are the exported members, that is, the first letter of the field name is uppercase.

A reflect.Value variable that takes the address records whether a structure member is an unexported member and rejects the modification if it is. CanAddr does not indicate whether a variable can be modified. CanSet checks if the corresponding reflect.Value is available and can be modified.

package main

import (
	"reflect"
	"fmt"
)

type Child struct {
	Name     string
	handsome bool
}

func main(a) {
	qcrao := Child{Name: "qcrao", handsome: true}

	v := reflect.ValueOf(&qcrao)

	f := v.Elem().FieldByName("Name")
	fmt.Println(f.String())

	f.SetString("stefno")
	fmt.Println(f.String())

	f = v.Elem().FieldByName("handsome")
	
	// This causes panic because the handsome field is not exported
	//f.SetBool(true)
	fmt.Println(f.Bool())
}
Copy the code

Execution Result:

qcrao
stefno
true
Copy the code

In the above example, the Handsome field is not exported and can be read, but the related set method cannot be called, otherwise it will panic. Reflection must be used with care. Calling methods with mismatched types can cause panic.

Practical application of reflection

The practical application of reflection is very wide: IDE code auto-completion function, Object serialization (JSON function library), FMT related function implementation, ORM (full name: Object Relational Mapping, Object Relational Mapping)……

Here are two examples: json serialization and the DeepEqual function.

Json serialization

For those of you who have developed Web services, you’ve probably used the JSON data format. Json is a language-independent data format. It was originally used for real-time stateless data exchange between browsers and servers and developed from there.

Go provides two functions for serialization and deserialization:

func Marshal(v interface{}) ([]byte, error)
func Unmarshal(data []byte, v interface{}) error
Copy the code

Both functions take an interface as an argument, and are implemented using reflection-related features.

For both serialization and deserialization functions, you need to know all the fields of the parameter, including the field type and value, and then call the relevant GET or set function to perform the actual operation.

Function and principle of DeepEqual

In test functions, you often need a function that determines that the actual contents of two variables are exactly the same.

For example, how to determine that all elements of two slices are identical; How to determine whether the keys and values of two maps are the same?

The above problem can be implemented through the DeepEqual function.

func DeepEqual(x, y interface{}) bool
Copy the code

The DeepEqual function takes two interfaces, which can be typed as either true or flase to indicate whether the two input variables are of equal depth.

Just to be clear, if they are different types, even if the underlying type is the same and the corresponding value is the same, then they are not equal in “depth”.

type MyInt int
type YourInt int

func main(a) {
	m := MyInt(1)
	y := YourInt(1)

	fmt.Println(reflect.DeepEqual(m, y)) // false
}
Copy the code

In the above code, m and y are both underlying ints with values of 1, but they are statically typed differently. The former is MyInt and the latter YourInt, so they are not “deep” equal.

The DeepEqual function is annotated very clearly in the source code, listing how the different types of DeepEqual compare.

type Equal depth case
Array Elements at the same index have equal “depth”
Struct Corresponding fields, including export and not export, have the same “depth”
Func Only if both are nil
Interface Both store the same “depth” of specific values
Map 1. Both are nil; 2. Non-empty, with the same length, pointing to the same map entity object, or corresponding keys pointing to the value of the same “depth”
Pointer 1. Use == to compare results equally; 2. Pointing to entities with equal “depth”
Slice 1. Both are nil; The first element refers to the same element in the same underlying array, i.e. &x[0] == &y[0] or the elements at the same index have the same “depth”
numbers, bools, strings, and channels The result of the comparison using == is true

In general, the implementation of DeepEqual simply needs to recursively call == to compare whether two variables are truly “deep” or not.

However, there are some exceptions: func types are not comparable and are “deep” only if both func types are nil; The float type, for precision reasons, also cannot be compared with ==; Struct, interface, array, etc., of type func or float.

For Pointers, when two Pointers are equal, they are equal in depth because they point to the same thing, even if they point to func or float, in which case they don’t care what the Pointers point to.

Similarly, for the same slice, both variables of the map have the same “depth”, regardless of the slice or map content.

For “looped” types, such as circular linked lists, the process of comparing whether the two Pointers have the same “depth” needs to mark the contents of the comparison. Once it is found that the two Pointers have been compared before, the comparison should be stopped immediately and the two Pointers are judged to have the same depth. The reason for doing this is to stop the comparison in time to avoid getting stuck in an infinite loop.

Look at the source code:

func DeepEqual(x, y interface{}) bool {
	if x == nil || y == nil {
		return x == y
	}
	v1 := ValueOf(x)
	v2 := ValueOf(y)
	ifv1.Type() ! = v2.Type() {return false
	}
	return deepValueEqual(v1, v2, make(map[visit]bool), 0)}Copy the code

First check to see if either of them is nil, in which case the function returns true only if both are nil.

Next, using reflection, get the reflection objects of x and y, and immediately compare their types. Based on the previous content, this is actually a dynamic type, and return false if the types are different.

Finally, the core content is in the subfunction deepValueEqual.

The code is longer, but the idea is simple and clear: The core is a switch statement that identifies the different types of input parameters, recursively calls deepValueEqual, recursively down to the most basic data types, compares int, string, etc., and returns true or false. Finally, the comparison result of “depth” is equal.

In fact, all types of comparisons are similar, so here’s an excerpt from a slightly more complex map comparison:

/ / deepValueEqual function
/ /...

case Map:
	ifv1.IsNil() ! = v2.IsNil() {return false
	}
	ifv1.Len() ! = v2.Len() {return false
	}
	if v1.Pointer() == v2.Pointer() {
		return true
	}
	for _, k := range v1.MapKeys() {
		val1 := v1.MapIndex(k)
		val2 := v2.MapIndex(k)
		if! val1.IsValid() || ! val2.IsValid() || ! deepValueEqual(v1.MapIndex(k), v2.MapIndex(k), visited, depth+1) {
			return false}}return true
	
/ /...
Copy the code

The idea of comparing maps for equality is consistent with the table summarized above, and nothing more needs to be said. To clarify one thing, Visited is a map, which records the “pairs” compared in the recursive process:

type visit struct {
	a1  unsafe.Pointer
	a2  unsafe.Pointer
	typ Type
}

map[visit]bool
Copy the code

In the comparison process, if it is found that the “pair” of the comparison has already appeared in the map, the direct judgment of the “depth” comparison result is true.

conclusion

As a static language, Go has limited flexibility when writing compared to a dynamic language such as Python. But interface plus reflection enables the ability of a dynamic language to capture and even change type information and values dynamically while the program is running.

Go’s reflection implementation is based on the type, or interface, and when we use reflection, we actually use the type-related information stored in the interface variable, commonly known as

pairs.
,>

Only interface has reflection.

Reflection is implemented in the Reflect package and involves two related functions:

func TypeOf ( i interface{}) Type
func ValueOf ( i interface{}) Value
Copy the code

Type is an interface that defines a number of related methods to get Type information. Value holds the specific Value of the type. Type, Value, and Interface are converted to each other using functions TypeOf, ValueOf, and Interface.

Finally, let’s review the three laws of reflection:

  1. Reflection goes from interface value to reflection object.
  2. Reflection goes from reflection object to interface value.
  3. To modify a reflection object, the value must be settable.

Translation:

  1. Reflection converts interface variables into reflection objects Type and Value.
  2. Reflection can be restored to the original interface variable through the reflection object Value.
  3. Reflection can be used to modify the value of a variable if the value can be modified.

The resources

zh.wikipedia.org/wiki/ reflection _(calculate…

[yard hole old money reflex] juejin.cn/post/684490…

Go Official blog Reflection blog.golang.org/laws-of-ref…

【GCTT 原 文 】mp.weixin.qq.com/s/dkgJ_fA0s…

Json library source code analysis zhuanlan.zhihu.com/p/37165706 】

Better reflect the code examples and figure 】 【 blog.gopheracademy.com/advent-2018…

[Reflection use good] juejin.cn/post/684490…

Interface and the reflection of the relationship, English 】 【 blog.gopheracademy.com/advent-2018…

【 summary into knowledge 】 www.cnblogs.com/susufufu/p/…

【Type Value】colobu.com/2016/07/09/…

www.lijiaocn.com/ programming /2017/11/…

Github.com/Chasiny/Blo DeepEqual 】 【…

Reflection usage scenarios yq.aliyun.com/articles/59 】…