CGO entry and OCR character recognition (non-third-party API, source code, good effect) actual combat

This is the fifth day of my participation in the August Wen Challenge.More challenges in August

For the series of articles:

• [day 4] GDB debug guide: C++ how to debug the production environment program?
• [Day 3] IM sensitive word algorithm principle and implementation
• [Day 2] Modern IM Architecture Research Note (1) : Melon Seed IM and OpenIM
• [Day 1] How to properly use the Sarama package to operate Kafka in Golang?

What is the CGO

Simply put, if you want to call C++, C – written libraries (dynamic libraries, static libraries), then you need to use Cgo. In other cases, you don’t need to know that Go can call C, and C can also call back to Go.

There are 2 positions to use Cgo:

1. Write C code directly in GO
2. Go calls the so dynamic library (c++ will export with extern “c”)

To get familiar with CGO, let’s introduce the first method, which is to write C code directly in Go.

Get started, write C code directly in Go

Command cgo

First, import the fake package with import “C” (this package does not exist and will not be seen by the COMPILE component of Go, it will be caught by the CGO tool before compiling and will do some code rewriting and pile file generation).

import "C"
Copy the code

Go can then use C variables and functions, types such as C. sze_t, variables such as C. tdout, or functions such as C. puchar.

func main(a){
    cInt := C.int(1)     // Use the int type in C
    fmt.Println(goInt)

    ptr := C.malloc(20)  // Call the function in C
    fmt.Println(ptr)     // Prints the pointer address
    C.free(ptr)          #include 
      
}
Copy the code

If the import of “C” immediately precedes a comment, that comment is called a prologue. Such as:

// #include <stdio.h>
/* #include <errno.h> */
import "C"
Copy the code

The prologue can contain any C code, including function and variable declarations and definitions. They can then be referenced from the Go code as if they were defined in package “C”. You can use all the names declared in the prologue, even if they start with a lowercase letter. Exception: static variables in the prologue cannot be referenced from the Go code; Static functions are allowed.

So, you can write C code directly inside /**/ (note, C++ does not work!). :

package main

/* int add(int a,int b){ return a+b; } * /
import "C"
import "fmt"

func main(a) {
    a, b := 1.2
    c := C.add(a, b)
}
Copy the code

FMT.Println(c.dd (1, 2)) compiles. :

./main.go:20:12: cannot use a (type int) as type _Ctype_int in argument to _Cfunc_add
./main.go:20:12: cannot use b (type int) as type _Ctype_int in argument to _Cfunc_add
Copy the code

Why is that? C cannot use the Go type, so it must be converted to CGO first.

func main(a) {
   cgoIntA, cgoIntB := C.int(1), C.int(2)
   c := C.add(cgoIntA, cgoIntB)
   fmt.Println(c)
}
Copy the code

Output after running:

3
Copy the code

CGO Basic type

Just like the code above, Go has no way to use C directly. It must first be converted to CGO. Here is a table of basic types.

C type	CGO type	GO type
char	C.char	byte
signed char	C.schar	int8
unsigned char	C.uchar	uint8
short	C.short	int16
unsigned short	C.ushort	uint16
int	C.int	int32
unsigned int	C.uint	uint32
long	C.long	int32
unsigned long	C.ulong	uint32
long long int	C.longlong	int64
unsigned long long int	C.ulonglong	uint64
float	C.float	float32
double	C.double	float64
size_t	C.size_t	uint

If you use #include

in C, the type relationship is more consistent, for example:

C type	CGO type	GO type
int8_t	C.int8_t	int8
int16_t	C.int16_t	int16
uint32_t	C.uint32_t	uint32
uint64_t	C.uint64_t	uint64

Strings, arrays, and function calls

So how do you pass strings, byte arrays, and Pointers in Go? The C virtual package for CGO provides the following set of functions for bidirectional array and string conversions between Go and C:

// Go string to C string
// The C string is allocated in the C heap using malloc.
// It is the caller's responsibility to arrange for it to be
// freed, such as by calling C.free (be sure to include stdlib.h
// if C.free is needed).
func C.CString(string) *C.char

// Go []byte slice to C array
// The C array is allocated in the C heap using malloc.
// It is the caller's responsibility to arrange for it to be
// freed, such as by calling C.free (be sure to include stdlib.h
// if C.free is needed).
func C.CBytes([]byte) unsafe.Pointer

// C string to Go string
func C.GoString(*C.char) string

// C data with explicit length to Go string
func C.GoStringN(*C.char, C.int) string

// C data with explicit length to Go []byte
func C.GoBytes(unsafe.Pointer, C.int) []byte
Copy the code

String, which can be freed by the c.string () function (don’t forget to free) :

// Cgo uses malloc to create a new space, which needs to be released when it is used, otherwise memory leaks
imagePath := C.CString("a.png")
defer C.free(unsafe.Pointer(imagePath))
Copy the code

Byte array, directly use the array of go, and then cast:

// Can only use arrays, can not use slices as buffers for C
var buffer [20]byte
// &buffer[0]: Arrays are stored consecutively in memory, taking the first address
// unsafe.pointer () : converts to an unsafe Pointer of type * unsafe.pointer
// (* c.char)()
cBuffer := (*C.char)(unsafe.Pointer(&buffer[0]))
Copy the code

Select * from Cgo; select * from Cgo;

bufferLen := C.int(20)
cPoint := &bufferLen      // cPoint is of type *C.int in CGO and of type *int in C.
Copy the code

Suppose the OCR recognition function is as follows:

int detect(const char* image_path, char * out_buffer, int *len);
Copy the code

There are three parameters:

• Image_path: indicates the image path to identify.
• Out_buffer: The output to the identified text is an array of char bytes.
• Len: Indicates the size of the output byte buffer. After a successful call, the value becomes a string length for external reading.

Call in GO as follows:

imagePath := C.CString("a.png")
defer C.free(unsafe.Pointer(imagePath))

var buffer [20]byte
bufferLen := C.int(20)
cInt := C.detect(imagePath, (*C.char)(unsafe.Pointer(&buffer[0])), &bufferLen)
if cInt == 0 {
   fmt.Println(string(buffer[0:bufferLen]))
}
Copy the code

Separate the Go and C code

In order to simplify the code, we can put C code into xxx.h and XXX.c implementation.

It has the following structure:

├── Heavy exercises ── heavy exercisesCopy the code

Hello. H contents:

#include <stdio.h>

void sayHello(const char* text);
Copy the code

C: hello.

#include "hello.h"

void sayHello(const char* text){
    printf("%s", text);
}
Copy the code

Call the hello.h function in main.go:

#include "hello.h"
import "C" // Must be placed after the comment importing c code header file, otherwise does not take effect

func main(a) {
    cStr := C.CString("hello from go")
    defer C.free(unsafe.Pointer(cStr))
    C.sayHello(cStr)
}
Copy the code

Commonly used CGO compilation instructions

If we put the h and C files in another directory, the compilation will report an error:

├── heavy exercises ── heavy exercises ── heavy exercises ── heavy exercisesCopy the code

Undefined symbols for architecture x86_64:
  "_sayHello", referenced from:
      __cgo_7ab15a91ce47_Cfunc_sayHello in _x002.o
     (maybe you meant: __cgo_7ab15a91ce47_Cfunc_sayHello)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Copy the code

All lags behind the # cGO script (CFLAGS, CPPFLAGS, CXXFLAGS, FFLAGS and LDFLAGS) :

// #cgo CFLAGS: -DPNG_DEBUG=1 -I ./include
// #cgo LDFLAGS: -L /usr/local/lib -lpng
// #include <png.h>
import "C"
Copy the code

• The CFLAGS: -d section defines the macro PNG_DEBUG with a value of 1. -I defines the search directory contained in the header file
• LDFLAGS: -l specifies the directory to retrieve library files when linking, and -l specifies the PNG library when linking

Most of the cases where cGO is used in real work are calls to the dynamic library, so there is no further discussion of how to solve the above error.

Call C static and dynamic libraries

The directory structure is as follows:

├── ├─ ├─ exercises.go├── ├─ ├─ exercises.go└ ─ ─ mylib ├ ─ ─ hello. C ├ ─ ─ hello. H ├ ─ ─ libhello. A └ ─ ─ libhello. SoCopy the code

1) Static library

Generate hello.h and hello.c as static libraries (GCC installation required, omitted) :

Generate the o object
$ gcc -c hello.c
Generate static libraries
$ ar crv libhello.a hello.o
# See what's in it
# ar -t libhello.a
Use static libraries
#gcc main.c libhello.a -o main
Copy the code

Call C static library in Go:

package main

/* #cgo CFLAGS: -I .. /mylib #cgo LDFLAGS: -L .. /mylib -lhello #include 
      
        #include "hello.h" */
      
import "C"
import "unsafe"

// Use readme. md to generate the libhello.a static library file
func main(a) {
   cStr := C.CString("hello from go")
   defer C.free(unsafe.Pointer(cStr))

   C.sayHello(cStr)
}
Copy the code

2) Dynamic library

generate

Generate the o object
$ gcc -fPIC -c hello.c
Build dynamic libraries
$ gcc -shared -fPIC -o libhello.so hello.o
# Use dynamic libraries
#gcc main.c -L. -lhello -o main
Copy the code

The call code is the same as above, LDFLAGS plus -lstDC ++ :

#cgo LDFLAGS: -L .. /mylib -lhello -lstdc++Copy the code

Note that the generated so file must be libhello.so, then in Go you just need to write -lhello instead of libhello, Linux automatically adds the lib prefix.

The only difference is that the static library needs to specify the search path for the so file or copy the so dynamic library to /usr/lib and configure it in the environment variable:

$ exportLD_LIBRARY_PATH=.. /mylib $ go run main.goLD_LIBRARY_PATH=.. Run -> Edit Configurations -> Environment LD_LIBRARY_PATH=.. /mylib, easy to debug
Copy the code

More about the difference between static library and dynamic library: segmentfault.com/a/119000002…

Call the C++ dynamic library

Go is the same as calling the c dynamic library, but it needs to be exported in C style:

#ifdef __cplusplus
extern "C"
{
#else

// Export C named functions with the same name as defined. C++ supports overloading, so the name of the exported function is changed by the compiler

#ifdef __cplusplus
}
#endif
Copy the code

The defect of CGO

Cgo is Not Go summarizes the shortcomings of CGO:

1. The compilation is slow, the actual use of C language compilation tools, but also to deal with the cross-platform issues of C language
2. Compilation becomes complicated
3. Cross-compilation is not supported
4. Many other go language tools are not available
5. The intercall between C and Go is cumbersome and has performance overhead
6. C is dominant, and go becomes unimportant, just like when you call C in Python
7. Deployment is complex and is no longer just a simple binary

This article describes the CGO by reasons of the go to call C performance overhead: blog.csdn.net/u010853261/…

• The coroutine stack of GO must be switched to the main stack of the system thread to execute C functions
• It involves scheduling system calls and coroutines.
• Because of the need to preserve both C/C++ runtimes, CGO requires translation and coordination between the two runtimes and the two ABIs (abstract binary interfaces). That’s a lot of overhead.

The reason is further explained in The Runtime source code in GO Original

So, be flexible when using it, depending on the situation.

Summary of CGO best use scenarios

Some disadvantages of CGO:

1. Memory isolation

2. C function execution switch to G0 (system thread)

3. A GOMAXPROC thread limit was received

4. Performance loss for CGO air conditioning (50+ns)

5. Compile losses (CGO actually has an intermediate layer)

CGO suitable scenarios:

1. C function is a big computation task (don’t care about CGO call performance loss)

2. C functions are called infrequently

3. There is no blocking IO in C functions

4. There is no new thread in C function (with potential interaction with coroutine scheduling in GO)

5. Don’t care about the complexity of compilation and deployment

More can be read:

• 【Free Style】CGO: Go and C interoperability technology (a) : Go tuning C basic principle
• 【Free Style】CGO: Go and C interoperability technology (three) : Go tuning C performance analysis and optimization
• Memory management notes when the Go language uses CGO

Ocr of actual combat

1. Chineseocr_lite is introduced

Making: github.com/DayBreak-u/… Star: 7.1K Introduction: Ultra-lightweight Chinese OCR, support vertical text recognition, support NCNN, MNN, TNN reasoning (DBNET (1.8m) + CRNN (2.5m) + Anglenet (378KB)) total model is only 4.7m.

This open source project provides C++, JVM, Android,.net and other implementation, without any tripartite dependence, by the author’s practice, the recognition effect is medium, the smaller the picture is faster.

For example, it only takes about 50ms to identify an invoice number:

Complex image recognition is about 500-900ms:

Table recognition effect is mediocre

Therefore, it is suitable for a consistent format identification scenario. Such as the location of the invoice, ID card, bank card, etc.

2. Compile chineseocr_lite

The readme. md file in Chineseocr_lite /cpp_projects/OcrLiteOnnx is recommended for Linux, but I failed it on Windows and Macos.

Then I need to transform it into a dynamic library. The changes I made are as follows:

• By default, dynamic libraries are generated for use by ocr_HTTP_server
• Remove JNI support
• Add OCR. H and export C-style function

3. Export c functions

ocr.h

/** @file ocr.h * @brief encapsulate to GO call * @author teng.qing * @date 8/13/21 */
#ifndef ONLINE_BASE_OCRLITEONNX_OCR_H
#define ONLINE_BASE_OCRLITEONNX_OCR_H
#ifdef __cplusplus
extern "C"
{
#else
    // c
    typedef enum{
        false.true
    }bool;
#endif

const int kOcrError = 0;
const int kOcrSuccess = 1;
const int kDefaultPadding = 50;
const int kDefaultMaxSideLen = 1024;
const float kDefaultBoxScoreThresh = 0.6 f;
const float kDefaultBoxThresh = 0.3 f;
const float kDefaultUnClipRatio = 2.0 f;
const bool kDefaultDoAngle = true;
const bool kDefaultMostAngle = true;

/**@fn ocr_init *@brief Initializes OCR *@param numThread: number of threads, no more than the number of cpus *@param dbNetPath: path of the DBnet model *@param anglePath: Angle identification model path *@param crnnPath: CRNN inference model path *@param keyPath: keys.txt Sample path *@return <0: error, >0: instance */
int ocr_init(int numThread, const char *dbNetPath, const char *anglePath, const char *crnnPath, const char *keyPath);

/**@fn ocr_cleanup *@brief, run */ before exiting the program
void ocr_cleanup(a);

/**@fn ocr_detect *@brief image recognition *@param image_path: indicates the complete image path. The image recognition box selection effect is generated in the same path for debugging. *@param out_json_result: Indicates the identification result output in JSON format. *@param buffer_len: Output buffer size *@param PADDING: 50 *@param maxSideLen: 1024 *@param boxScoreThresh: 0.6f *@param boxThresh: 0.3F *@param unClipRatio: 2.0f *@param doAngle: true *@param mostAngle: true *@return Successful or not */
int ocr_detect(const char *image_path, char *out_buffer, int *buffer_len, int padding, int maxSideLen,
                float boxScoreThresh, float boxThresh, float unClipRatio, bool doAngle, bool mostAngle);
                
/**@fn ocr_detect *@brief Uses the default parameters to identify the image. *@param image_path: indicates the full path of the image. *@param out_buffer: Indicates the identification result. *@param buffer_len: output buffer size *@return success or not */
int ocr_detect2(const char *image_path, char *out_buffer, int *buffer_len);

#ifdef __cplusplus
}
#endif
#endif //ONLINE_BASE_OCRLITEONNX_OCR_H
Copy the code

ocr.cpp

/** @file ocr.h * @brief * @author teng.qing * @date 8/13/21 */
#include "ocr.h"
#include "OcrLite.h"
#include "omp.h"
#include "json.hpp"
#include <iostream>
#include <sys/stat.h>
using json = nlohmann::json;
static OcrLite *g_ocrLite = nullptr;
inline bool isFileExists(const char *name) {
    struct stat buffer{};
    return (stat(name, &buffer) == 0);
}
int ocr_init(int numThread, const char *dbNetPath, const char *anglePath, const char *crnnPath, const char *keyPath) {
    omp_set_num_threads(numThread); // Parallel computation
    if (g_ocrLite == nullptr) {
        g_ocrLite = new OcrLite(a); } g_ocrLite->setNumThread(numThread);
    g_ocrLite->initLogger(
            true.//isOutputConsole
            false.//isOutputPartImg
            true);//isOutputResultImg
    g_ocrLite->Logger(
            "ocr_init numThread=%d, dbNetPath=%s,anglePath=%s,crnnPath=%s,keyPath=%s \n",
            numThread, dbNetPath, anglePath, crnnPath, keyPath);
    if (!isFileExists(dbNetPath) || !isFileExists(anglePath) || !isFileExists(crnnPath) || !isFileExists(keyPath)) {
        g_ocrLite->Logger("invalid file path.\n");
        return kOcrError;
    }
    g_ocrLite->initModels(dbNetPath, anglePath, crnnPath, keyPath);
    return kOcrSuccess;
}
void ocr_cleanup(a) {
    if(g_ocrLite ! =nullptr) {
        delete g_ocrLite;
        g_ocrLite = nullptr; }}int ocr_detect(const char *image_path, char *out_buffer, int *buffer_len, int padding, int maxSideLen,
               float boxScoreThresh, float boxThresh, float unClipRatio, bool doAngle, bool mostAngle) {
    if (g_ocrLite == nullptr) {
        return kOcrError;
    }
    if (!isFileExists(image_path)) {
        return kOcrError;
    }
    g_ocrLite->Logger(
            "padding(%d),maxSideLen(%d),boxScoreThresh(%f),boxThresh(%f),unClipRatio(%f),doAngle(%d),mostAngle(%d)\n",
            padding, maxSideLen, boxScoreThresh, boxThresh, unClipRatio, doAngle, mostAngle);
    OcrResult result = g_ocrLite->detect("", image_path, padding, maxSideLen,
                                         boxScoreThresh, boxThresh, unClipRatio, doAngle, mostAngle);
    json root;
    root["dbNetTime"] = result.dbNetTime;
    root["detectTime"] = result.detectTime;
    for (const auto &item : result.textBlocks) {
        json textBlock;
        for (const auto &boxPoint : item.boxPoint) {
            json point;
            point["x"] = boxPoint.x;
            point["y"] = boxPoint.y;
            textBlock["boxPoint"].push_back(point);
        }
        for (const auto &score : item.charScores) {
            textBlock["charScores"].push_back(score);
        }
        textBlock["text"] = item.text;
        textBlock["boxScore"] = item.boxScore;
        textBlock["angleIndex"] = item.angleIndex;
        textBlock["angleScore"] = item.angleScore;
        textBlock["angleTime"] = item.angleTime;
        textBlock["crnnTime"] = item.crnnTime;
        textBlock["blockTime"] = item.blockTime;
        root["textBlocks"].push_back(textBlock);
        root["texts"].push_back(item.text);
    }
    std::string tempJsonStr = root.dump(a);if (static_cast<int>(tempJsonStr.length()) > *buffer_len) {
        g_ocrLite->Logger("buff_len is too small \n");
        return kOcrError;
    }
    *buffer_len = static_cast<int>(tempJsonStr.length()); : :memcpy(out_buffer, tempJsonStr.c_str(), tempJsonStr.length());
    return kOcrSuccess;
}
int ocr_detect2(const char *image_path, char *out_buffer, int *buffer_len) {
    return ocr_detect(image_path, out_buffer, buffer_len, kDefaultPadding, kDefaultMaxSideLen, kDefaultBoxScoreThresh,
                      kDefaultBoxThresh, kDefaultUnClipRatio, kDefaultDoAngle, kDefaultMostAngle);
}
Copy the code

ocr_wrapper.go

package ocr

// -i: configures compilation options
// -l: dependent library path

/* #cgo CFLAGS: -I .. /.. /.. /OcrLiteOnnx/include #cgo LDFLAGS: -L .. /.. /.. /OcrLiteOnnx/lib -lOcrLiteOnnx -lstdc++ #include 
      
        #include 
       
         #include "ocr.h" */
       
      
import "C"
import (
        "runtime"
        "unsafe"
)

//const kModelDbNet = "dbnet.onnx"
//const kModelAngle = "angle_net.onnx"
//const kModelCRNN = "crnn_lite_lstm.onnx"
//const kModelKeys = "keys.txt"

const kDefaultBufferLen = 10 * 1024

var (
        buffer [kDefaultBufferLen]byte
)

func Init(dbNet, angle, crnn, keys string) int {
        threadNum := runtime.NumCPU()

        cDbNet := C.CString(dbNet) // to c char*
        cAngle := C.CString(angle) // to c char*
        cCRNN := C.CString(crnn)   // to c char*
        cKeys := C.CString(keys)   // to c char*

        ret := C.ocr_init(C.int(threadNum), cDbNet, cAngle, cCRNN, cKeys)

        C.free(unsafe.Pointer(cDbNet))
        C.free(unsafe.Pointer(cAngle))
        C.free(unsafe.Pointer(cCRNN))
        C.free(unsafe.Pointer(cKeys))
        return int(ret)
}

func Detect(imagePath string) (bool.string) {
        resultLen := C.int(kDefaultBufferLen)

        // Construct a buffer for C
        cTempBuffer := (*C.char)(unsafe.Pointer(&buffer[0]))
        cImagePath := C.CString(imagePath)
        defer C.free(unsafe.Pointer(cImagePath))

        isSuccess := C.ocr_detect2(cImagePath, cTempBuffer, &resultLen)
        return int(isSuccess) == 1, C.GoStringN(cTempBuffer, resultLen)
}

func CleanUp(a) {
        C.ocr_cleanup()
}
Copy the code

3. Set environment variables

The path contains the library directory, or you can copy the dynamic library directly to /usr/lib. The latter is recommended:

exportLD_LIBRARY_PATH=.. /mylibCopy the code

4. Run

Results the following

reference

• 【Free Style】CGO: Go and C interoperability technology (a) : Go tuning C basic principle
• 【Free Style】CGO: Go and C interoperability technology (three) : Go tuning C performance analysis and optimization
• Command cgo
• C? Go? Cgo!
• How to use C++ in Go?
• Learn more about CGO
• How to increase the performance of Go call C by 10 times?
• Memory management notes when the Go language uses CGO
• GO originally:cgo
• CGO and CGO performance puzzle
• cgo is not Go
• In-depth CGO Programming (Gopherchina2018)
• how-to-use-c-in-go
• Use GCC to generate static and dynamic libraries
• How do I use GCC to generate dynamic and static libraries
• The GCC compiler generates one of the dynamic and static libraries —- introduction