1 introduction
Last month I wrote a blog post called “Using Reflection to Paginate Gorm” to document how to use Go reflection to implement a Spring Mybatis style pagination. I mentioned that I could use generics to wrap Gorm after Go 1.18 was released last week. To rub off on the heat here, try using generics instead of reflection to implement Gorm paging encapsulation. Section 4 covers the basic use of generics using the List example, and Section 5 covers the core code used to encapsulate paging with generics. See GitHub for the complete working code for this article
2 Project Structure
The Demo structure of this time is similar to that of last time. Gin is still used to build the Demo, but it has been optimized to some extent. Again, the country and city tables in MySQL 8’s built-in World database are paginated. If the world database is not available in Mysql, you can obtain it from the Mysql website (Mysql :: Other Mysql Documentation). Database is the initialization of the database. The structure corresponding to city and country tables and the conditional query structure are recorded in Model, and the structure returned to the front end is recorded in Response. go. Service is the specific business query logic; There are only two routes in main.go, one is paging to query the route of the city, the other is paging to query the route of the country. See GitHub for the full code
| - gorm_page | | - database | | - mysql. Go / / initializes connection | | -- model. Go / / page [T] structure and paging package | | - model | | -- city. Go / / city table corresponding to the structure | | - country. Go / / country table corresponding structure | | -- page. Go / / paging conditions | | -- response. Go / / back to the front of the structure | | - service | | -- city. Go | | - country. Go | | -- go. Mod | | -- go. Sum | | -- main. GoCopy the codeThree structures
For details on City, Country, paging queries, and other constructs, see my previous blog post, which focuses on Page[T any] and Response constructs. Page[T any] is used to store the total number of pages and records queried from the database. The Data field is used to store the list of Data using generics. The Page is not returned directly to the front end, because the list in the Page may require some conversion (for example, the user information returned from the database contains first_name and last_name, but the front end only needs full name, so concates the name). So define a PageResponse to receive the transformed Page structure. The Response structure is returned to the front end, and the Data field can be string, int, map, PageResponse, and other arbitrary types of Data, so instead of generics, interface is used.
/* database/model.go */
type Page[T any] struct {
CurrentPage int64
PageSize int64
Total int64
Pages int64
Data []T
}
/* model/response.go */
type Response struct {
Code int `json:"code"`
Msg string `json:"msg"`
Data interface{} `json:"data"`
}
type PageResponse[T any] struct {
CurrentPage int64 `json:"currentPage"`
PageSize int64 `json:"pageSize"`
Total int64 `json:"total"`
Pages int64 `json:"pages"` / / the total number of pages
Data []T `json:"data"`
}
// Convert the original Page structure to the PageResponse structure required by the front end
func NewPageResponse[T any](page *database.Page[T]) *PageResponse[T] {
return &PageResponse[T]{
CurrentPage: page.CurrentPage,
PageSize: page.PageSize,
Total: page.Total,
Pages: page.Pages,
// You would normally convert elements in Data, such as concatenating names and time formats, but there are no fields to convert
Data: page.Data,
}
}
Copy the codeBasic use of generics
4.1 Use of generics in functions
Before we wanted to use the Max function, we might have to write the different cases of int, float, string. Now with generics, we can use the brackets before the function to indicate that the function supports all comparable types. The parameter is of type T, just like in other languages. Specific usage is as follows:
The supported generic type is Comparable
func max[T comparable](T a, b) {
if a > b {
return a
} else {
return b
}
}
Copy the code4.2 Use of generics in structs
Here we use generics to define the linked List structure. When we define the List, we use brackets to indicate that the structure supports any type T, and we use brackets after the generic Element field to indicate that the field is of type T. Struct methods are also defined with [T] on the receiver, and arguments and return values are specified as T if necessary.
type List[T any] struct {
Len int
root *Element[T] // Pseudo header node
}
type Element[T any] struct {
next *Element[T]
Value T
}
func (l *List[T]) Front(a) *Element[T] {
return l.root.next
}
func (l *List[T]) Init(a) {
l.Len = 0
l.root = &Element[T]{}
}
func (l *List[T]) Add(a T) {
node := new(Element[T])
node.Value = a
node.next = l.root.next
l.root.next = node
}
func (e *Element[T]) Next(a) *Element[T] {
return e.next
}
Copy the codeOnce you have defined the generic List, write a simple program to test it by declaring two List variables, one that stores int and one that stores FLOAT64, adding data to both lists and printing them out. In this case, the Value field of the element can be directly assigned to the variable of the corresponding type, without making explicit assertions such as E. Value.(int). The code is as follows:
func main(a) {
var list1 List[int]
list1.Init()
list1.Add(1)
list1.Add(2)
list1.Add(3)
fore := list1.Front(); e ! =nil; e = e.Next() {
var tmp int = e.Value // Value can be assigned directly to a variable of type int
fmt.Print(tmp)
fmt.Print("")
}
fmt.Print("\n")
var list2 List[float64]
list2.Init()
list2.Add(11.22)
list2.Add(23.55)
list2.Add(39.17)
fore := list2.Front(); e ! =nil; e = e.Next() {
var tmp float64 = e.Value // Value can be assigned directly to a variable of type float64
fmt.Print(tmp)
fmt.Print("")
}
fmt.Print("\n")}Copy the codeThe test run yields the following results, indicating that the generic List can be used to create linked lists of any type
$: Go run main.go 3 2 1 39.17 23.55 11.22Copy the codeGorm paging with generics
The core code for paging with reflection is as follows: first, use reflection to extract type information from the model from the upper layer, then use reflection to create a slice of the corresponding type, and finally store the queried list in the Data field of the page.
/* Paging encapsulation with reflection */
// Inside the wrapper are the query conditions and the model is the concrete structure
// Query City paging data -- database.selectPage (page, wrapper, City{})
-- database.selectPage (Page, wrapper, Country{})
func SelectPage(page *Page, wrapper map[string]interface{}, model interface{}) (e error) {
e = nil
DB.Model(&model).Where(wrapper).Count(&page.Total)
if page.Total == 0 {
// If there is no data matching the criteria, return an empty list
page.Data = []interface{} {}return
}
// reflection gets the type
t := reflect.TypeOf(model)
// Create an array of the corresponding type by reflection
list := reflect.Zero(reflect.SliceOf(t)).Interface()
e = DB.Model(&model).Where(wrapper).Scopes(Paginate(page)).Find(&list).Error
page.Data = list
return
}
// An example of a paging function provided by GORM
// You can set the total number of pages, total number of records, and other classification data, and set the query conditions limit, offset
func Paginate(page *Page) func(db *gorm.DB) *gorm.DB {
return func(db *gorm.DB) *gorm.DB {
if page.CurrentPage <= 0 {
page.CurrentPage = 0
}
switch {
case page.PageSize > 100:
page.PageSize = 100
case page.PageSize <= 0:
page.PageSize = 10
}
page.Pages = page.Total / page.PageSize
ifpage.Total%page.PageSize ! =0 {
page.Pages++
}
p := page.CurrentPage
if page.CurrentPage > page.Pages {
p = page.Pages
}
size := page.PageSize
offset := int((p - 1) * size)
return db.Offset(offset).Limit(int(size))
}
}
Copy the codeNow with generics we can modify the above code to avoid reflection. At this point, Page. Data can be passed directly to Gorm’s Find function as an argument, because page.Data has a definite type at compile time.
type Page[T any] struct {
CurrentPage int64
PageSize int64
Total int64
Pages int64
Data []T
}
func (page *Page[T]) SelectPage(wrapper map[string]interface{}) (e error) {
e = nil
var model T
DB.Model(&model).Where(wrapper).Count(&page.Total)
if page.Total == 0 {
// Return an empty list of type T
page.Data = []T{}
return
}
// Query results can be stored directly in the Page's Data field, because the Page.Data is typed at compile time
e = DB.Model(&model).Where(wrapper).Scopes(Paginate(page)).Find(&page.Data).Error
return
}
// Paginate plus T
func Paginate[T any](page *Page[T]) func(db *gorm.DB) *gorm.DB {
return func(db *gorm.DB) *gorm.DB {
if page.CurrentPage <= 0 {
page.CurrentPage = 0
}
switch {
case page.PageSize > 100:
page.PageSize = 100
case page.PageSize <= 0:
page.PageSize = 10
}
page.Pages = page.Total / page.PageSize
ifpage.Total%page.PageSize ! =0 {
page.Pages++
}
p := page.CurrentPage
if page.CurrentPage > page.Pages {
p = page.Pages
}
size := page.PageSize
offset := int((p - 1) * size)
return db.Offset(offset).Limit(int(size))
}
}
Copy the codeThe code to invoke a paging query in a service is shown below, with the current page count and paging size and corresponding query criteria in queryVo. You only need to specify whether the type is City or Country when you create the Page structure instance, and you don’t need to pass in extra arguments to specify the type to be queried when you call SelectPage. This style is very similar to Spring’s Mybatis/Mybatis- Plus,
type CityService struct{}
// Call paging queries using generics to query the City data that meets the criteria
func (c *CityService) SelectPageList(queryVo model.CityQueryInfo) (*model.PageResponse[model.City], error) {
p := &database.Page[model.City]{
CurrentPage: queryVo.CurrentPage,
PageSize: queryVo.PageSize,
} // Specify the Page type as City
wrapper := make(map[string]interface{}, 0)
ifqueryVo.CountryCode ! ="" {
wrapper["CountryCode"] = queryVo.CountryCode
}
ifqueryVo.District ! ="" {
wrapper["District"] = queryVo.District
}
// There is no extra information to tell the SelectPage function that we are looking for data of type City
err := p.SelectPage(wrapper)
iferr ! =nil {
return nil, err
}
// Convert the Page structure to the PageResponse structure required by the front end
pageResponse := model.NewPageResponse(p)
return pageResponse, err
}
Copy the code6 Test Run
6.1 Functional Tests
The complete project code is shown in my Github example. Enter Go run main. Go to run the Demo program. It can be seen that there are pages, total and other information in response, and the list data in data is normal, indicating that the SelectPage function encapsulated by generics has effects on different types.
6.2 Performance Test
Here we try Postman to perform performance tests on reflective and generic-wrapped Gorm pages, executing multiple paging queries under the same conditions to see which encapsulation is more efficient. There may be a future blog post explaining how to do complex tests with Postman. Here, the external CSV file is used as input. There are 85 queries in the file, which will query cities in China, the United States, Japan, India and other countries. Repeat the 85 query tests 5 times and get the following test results:
| Test number/packaging technology | Reflection encapsulation | Generic packaging |
|---|---|---|
| 1 | 587ms | 496ms |
| 2 | 540ms | 509ms |
| 3 | 501ms | 780ms |
| 4 | 478ms | 459ms |
| 5 | 480ms | 463ms |
With the exception of the third set of tests, reflection encapsulation takes slightly more time than generic encapsulation, but the difference is not significant, probably because the City structure here is relatively simple and does not capture the performance gap. Someone summed up on Zhihu that reflection loss is divided into two parts, one part is reflect.new () to create an object, the other part is value.field ().set () to Set object properties, see Golang reflection performance optimization – Zhihu.com.
7 thinking
Golang officially launched generics in 1.18, marking a milestone in Golang’s development. The addition of generics can improve project performance in some cases, such as the encapsulation of Gorm paging mentioned in this article, instead of reflection. At the same time, there was controversy in the Go community, with some developers arguing that the introduction of generics undermined the simplicity of the Avenue; Some developers don’t think that brackets are as elegant as Angle brackets; Some of the bigwigs directly criticized Google, saying that Google would cause division in the Go community. I have obsessive-compulsive disorder and want to use the latest version of the system, software and operating environment, so I upgraded the project to 1.18 at the first time and experienced a wave of generics. Personally, I think the introduction of generics is really convenient for my coding. If you have doubts, you can wait for one or two versions.