Introduction to the

In the last article, we introduced Gorilla/MUX, the routing management library in the Gorilla Web Development Kit, and ended the article by showing you how to use middleware to handle common logic. In everyday Go Web development, developers encounter many of the same middleware requirements, and Gorilla/Handlers (later referred to as “Handlers”) collect some of the more commonly used middleware. Let’s take a look

The previous articles have covered a lot about middleware. I won’t repeat it here. The Handlers library provides middleware that can be used by the standard library NET/HTTP and all frameworks that support the http.handler interface. Since Gorilla/MUX also supports the http.Handler interface, it can also be used in conjunction with the Handlers library. This is the benefit of standards compatibility.

Project Initialization & Installation

The code in this article uses Go Modules.

Create a directory and initialize it:

$ mkdir gorilla/handlers && cd gorilla/handlers
$ go mod init github.com/darjun/go-daily-lib/gorilla/handlers

Install Gorilla/Handlers library:

$ go get -u github.com/gorilla/handlers

Each middleware and corresponding source code are described in turn below.

The log

Handlers provide two logging middleware:

  • LoggingHandler: with ApacheCommon Log FormatLog format records HTTP request log;
  • CombinedLoggingHandler: with ApacheCombined Log FormatLogging format logs HTTP requests. Both Apache and Nginx use this logging format by default.

The differences between the two Log formats are very small. The Common Log Format is as follows:

%h %l %u %t "%r" %>s %b

The meaning of each indicator is as follows:

  • %h: IP address or host name of the client;
  • %l:RFC 1413Define the client identity by the client machineidentdProgram generation. If it does not, the field is-;
  • %u: The authenticated user name. If not, the field is-;

  • % t: time, format for day/month/year: hour: minute: second zone, including:

    • day: 2 digits;
    • month: Abbreviation for a month with three letters, such asJan;
    • year: 4 digits;
    • hour: 2 digits;
    • minute: 2 digits;
    • second: 2 digits;
    • zone:+or-Followed by four digits;
    • Such as:21/Jul/2021:06:27:33 +0800
  • %r: Contains HTTP request line information, for exampleGET/index. HTTP / 1.1 HTML;
  • %>s: Status code sent by the server to the client, for example200;
  • %b: Response length (number of bytes).

Combined Log Format:

%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i"

Compared to the Common Log Format:

  • %{Referer}i: HTTP headerRefererInformation;
  • %{User-Agent}i: HTTP headerUser-AgentInformation.

For middleware, we can make it work globally, for all processors, or we can make it work only for some processors. If you want it to work on all processors, you can call the Use() method. If you only need to work on a specific processor, wrap the processor with middleware at registration time:

func index(w http.ResponseWriter, r *http.Request) {
  fmt.Fprintln(w, "Hello World")
}

type greeting string

func (g greeting) ServeHTTP(w http.ResponseWriter, r *http.Request) {
  fmt.Fprintf(w, "Welcome, %s", g)
}

func main() {
  r := mux.NewRouter()
  r.Handle("/", handlers.LoggingHandler(os.Stdout, http.HandlerFunc(index)))
  r.Handle("/greeting", handlers.CombinedLoggingHandler(os.Stdout, greeting("dj")))

  http.Handle("/", r)
  log.Fatal(http.ListenAndServe(":8080", nil))
}

In the code above, LoggingHandler only applies to the index handler function, and combinedLoggingHandler only applies to the processor greeting(” DJ “).

Run the code and access localhost:8080 and localhost:8080/greeting:

::1 -- [21/Jul/2021:06:39:45 +0800] "GET/HTTP/1.1" 200 12 ::1 -- [21/Jul/2021:06:39:54 +0800] "GET /greeting HTTP/1.1" 200 11 "" Mozilla/5.0 (Windows NT 10.0; Win64; X64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.164 Safari/537.36"

It is easy to see the parts by comparing the indicators analyzed earlier.

Since the Use() method of * mux.router accepts middleware of type MIDDLEWAREFUNC:

type MiddlewareFunc func(http.Handler) http.Handler

And handlers. LoggingHandler/CombinedLoggingHandler are not meet, so also need a layer of packaging to Use () method:

func Logging(handler http.Handler) http.Handler {
  return handlers.CombinedLoggingHandler(os.Stdout, handler)
}

func main() {
  r := mux.NewRouter()
  r.Use(Logging)
  r.HandleFunc("/", index)
  r.Handle("/greeting/", greeting("dj"))

  http.Handle("/", r)
  log.Fatal(http.ListenAndServe(":8080", nil))
}

Handlers also provide a CustomLoginghHandler that we can use to define our own log middleware:

func CustomLoggingHandler(out io.Writer, h http.Handler, f LogFormatter) http.Handler

The most critical LogFormatter type definition is:

type LogFormatterParams struct {
  Request    *http.Request
  URL        url.URL
  TimeStamp  time.Time
  StatusCode int
  Size       int
}

type LogFormatter func(writer io.Writer, params LogFormatterParams)

We implement a simple LogFormatter that records the time + the request line + the response code:

func myLogFormatter(writer io.Writer, params handlers.LogFormatterParams) {
  var buf bytes.Buffer
  buf.WriteString(time.Now().Format("2006-01-02 15:04:05 -0700"))
  buf.WriteString(fmt.Sprintf(` "%s %s %s" `, params.Request.Method, params.URL.Path, params.Request.Proto))
  buf.WriteString(strconv.Itoa(params.StatusCode))
  buf.WriteByte('\n')

  writer.Write(buf.Bytes())
}

func Logging(handler http.Handler) http.Handler {
  return handlers.CustomLoggingHandler(os.Stdout, handler, myLogFormatter)
}

Use:

func main() {
  r := mux.NewRouter()
  r.Use(Logging)
  r.HandleFunc("/", index)
  r.Handle("/greeting/", greeting("dj"))

  http.Handle("/", r)
  log.Fatal(http.ListenAndServe(":8080", nil))
}

The log is now in the following format:

2021-07-21 07:03:18 +0800 "GET /greeting/ HTTP/1.1" 200

Read the source code, we can find LoggingHandler/CombinedLoggingHandler/CustomLoggingHandler LoggingHandler are based on the bottom, The difference is that LoggingHandler uses the predefined WriteLog as its logFormatter, while CombinedLoggingHandler uses the predefined WriteCombinedLog as its logFormatter, And CustomLoggingHandler uses our own defined LogFormatter:

func CombinedLoggingHandler(out io.Writer, h http.Handler) http.Handler {
  return loggingHandler{out, h, writeCombinedLog}
}

func LoggingHandler(out io.Writer, h http.Handler) http.Handler {
  return loggingHandler{out, h, writeLog}
}

func CustomLoggingHandler(out io.Writer, h http.Handler, f LogFormatter) http.Handler {
  return loggingHandler{out, h, f}
}

The predefined writeLog/writeCombinedLog implementation is as follows:

func writeLog(writer io.Writer, params LogFormatterParams) {
  buf := buildCommonLogLine(params.Request, params.URL, params.TimeStamp, params.StatusCode, params.Size)
  buf = append(buf, '\n')
  writer.Write(buf)
}

func writeCombinedLog(writer io.Writer, params LogFormatterParams) {
  buf := buildCommonLogLine(params.Request, params.URL, params.TimeStamp, params.StatusCode, params.Size)
  buf = append(buf, ` "`...)
  buf = appendQuoted(buf, params.Request.Referer())
  buf = append(buf, `" "`...)
  buf = appendQuoted(buf, params.Request.UserAgent())
  buf = append(buf, '"', '\n')
  writer.Write(buf)
}

They’re all building basic information based on BuildCommonLogLine, WriteCombinedLog also calls http.request. Referer() and http.request. userAgent respectively to get the Referer and User-Agent information.

LoggingHandler is defined as follows:

type loggingHandler struct {
  writer    io.Writer
  handler   http.Handler
  formatter LogFormatter
}

The LoggingHandler implementation has one neat twist: In order to record the response code and the response size, we define a type responseLogger wrapped around the original http.responseWriter to record the information when we write:

type responseLogger struct {
  w      http.ResponseWriter
  status int
  size   int
}

func (l *responseLogger) Write(b []byte) (int, error) {
  size, err := l.w.Write(b)
  l.size += size
  return size, err
}

func (l *responseLogger) WriteHeader(s int) {
  l.w.WriteHeader(s)
  l.status = s
}

func (l *responseLogger) Status() int {
  return l.status
}

func (l *responseLogger) Size() int {
  return l.size
}

LoggingHandler’s key method serveHttp () :

func (h loggingHandler) ServeHTTP(w http.ResponseWriter, req *http.Request) { t := time.Now() logger, w := makeLogger(w) url := *req.URL h.handler.ServeHTTP(w, req) if req.MultipartForm ! = nil { req.MultipartForm.RemoveAll() } params := LogFormatterParams{ Request: req, URL: url, TimeStamp: t, StatusCode: logger.Status(), Size: logger.Size(), } h.formatter(h.writer, params) }

Construct the LogFormatterParams object and call the corresponding LogFormatter function.

The compression

If the client request has an Accept-Encoding header, the server can use the algorithm indicated by that header to compress the response to save network traffic. Handlers.com PressHandler middleware enables compression. There is also an CompressHandlerLevel that can specify the level of compression. In fact, the CompressHandler is the same CompressHandlerLevel called with gzip.DefaultCompression:

func CompressHandler(h http.Handler) http.Handler {
  return CompressHandlerLevel(h, gzip.DefaultCompression)
}

Look at the code:

func index(w http.ResponseWriter, r *http.Request) {
  fmt.Fprintln(w, "Hello World")
}

type greeting string

func (g greeting) ServeHTTP(w http.ResponseWriter, r *http.Request) {
  fmt.Fprintf(w, "Welcome, %s", g)
}

func main() {
  r := mux.NewRouter()
  r.Use(handlers.CompressHandler)
  r.HandleFunc("/", index)
  r.Handle("/greeting/", greeting("dj"))

  http.Handle("/", r)
  log.Fatal(http.ListenAndServe(":8080", nil))
}

Run, request localhost:8080, and you can see the response using gzip compression from the Chrome Developer Tools’ Network TAB:

Ignoring some details, the CompressHandlerLevel function has the following code:

func CompressHandlerLevel(h http.Handler, level int) http.Handler { const ( gzipEncoding = "gzip" flateEncoding = "deflate" ) return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { var encoding string for _, curEnc := range strings.Split(r.Header.Get(acceptEncoding), ",") { curEnc = strings.TrimSpace(curEnc) if curEnc == gzipEncoding || curEnc == flateEncoding { encoding = curEnc break  } } if encoding == "" { h.ServeHTTP(w, r) return } if r.Header.Get("Upgrade") ! = "" { h.ServeHTTP(w, r) return } var encWriter io.WriteCloser if encoding == gzipEncoding { encWriter, _ = gzip.NewWriterLevel(w, level) } else if encoding == flateEncoding { encWriter, _ = flate.NewWriter(w, level) } defer encWriter.Close() w.Header().Set("Content-Encoding", encoding) r.Header.Del(acceptEncoding) cw := &compressResponseWriter{ w: w, compressor: encWriter, } w = httpsnoop.Wrap(w, httpsnoop.Hooks{ Write: func(httpsnoop.WriteFunc) httpsnoop.WriteFunc { return cw.Write }, WriteHeader: func(httpsnoop.WriteHeaderFunc) httpsnoop.WriteHeaderFunc { return cw.WriteHeader }, Flush: func(httpsnoop.FlushFunc) httpsnoop.FlushFunc { return cw.Flush }, ReadFrom: func(rff httpsnoop.ReadFromFunc) httpsnoop.ReadFromFunc { return cw.ReadFrom }, }) h.ServeHTTP(w, r) }) }

The compression algorithm that gets the client indication from the header of the request Accept-Encoding. If the client does not specify an Upgrade, or if there is an Upgrade in the request header, the compression is not performed. Otherwise, it compresses. Create an Io.Writer implementation object corresponding to gzip or flate based on the identified compression algorithm.

As with the previous logging middleware, we add a new type, CompressResponseWriter, to encapsulate http. responseWriter, override the Write() method, and pass the written byte stream to the IO.Writer we created earlier to compress the output:

type compressResponseWriter struct {
  compressor io.Writer
  w          http.ResponseWriter
}

func (cw *compressResponseWriter) Write(b []byte) (int, error) {
  h := cw.w.Header()
  if h.Get("Content-Type") == "" {
    h.Set("Content-Type", http.DetectContentType(b))
  }
  h.Del("Content-Length")

  return cw.compressor.Write(b)
}

Content type

We can through the handler. ContentTypeHandler specifies the content-type request must be given in our Type, only to the POST/PUT/PATCH method is effective. For example, we restrict login requests to be sent in application/x-www-form-urlencoded form:

func main() {
  r := mux.NewRouter()
  r.HandleFunc("/", index)
  r.Methods("GET").Path("/login").HandlerFunc(login)
  r.Methods("POST").Path("/login").
    Handler(handlers.ContentTypeHandler(http.HandlerFunc(dologin), "application/x-www-form-urlencoded"))

  http.Handle("/", r)
  log.Fatal(http.ListenAndServe(":8080", nil))
}

Thus, if the content-type of the request /login is not application/x-www-form-urlencoded, 415 errors will be returned. We can make a deliberate mistake and ask to see the performance:

Unsupported content type "application/x-www-form-urlencoded"; expected one of ["application/x-www-from-urlencoded"]

The implementation of the ContentTypeHandler is very simple:

func ContentTypeHandler(h http.Handler, contentTypes ... string) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { if ! (r.Method == "PUT" || r.Method == "POST" || r.Method == "PATCH") { h.ServeHTTP(w, r) return } for _, ct := range contentTypes { if isContentType(r.Header, ct) { h.ServeHTTP(w, r) return } } http.Error(w, fmt.Sprintf("Unsupported content type %q; expected one of %q", r.Header.Get("Content-Type"), contentTypes), http.StatusUnsupportedMediaType) }) }

To read the Content-Type header and see if it is in the Type we specified.

Method handler

In the above example, we registered the Path /login for the GET and POST methods using the verbous r.methods (“GET”).path (“/login”).handlerfunc (login). Handlers.MethodHandler simplifies this writing: Handlers.MethodHandler

func main() {
  r := mux.NewRouter()
  r.HandleFunc("/", index)
  r.Handle("/login", handlers.MethodHandler{
    "GET":  http.HandlerFunc(login),
    "POST": http.HandlerFunc(dologin),
  })

  http.Handle("/", r)
  log.Fatal(http.ListenAndServe(":8080", nil))
}

Underlies a Map [String]http.Handler type whose serveHttp () Method invokes different processing depending on the requested Method:

type MethodHandler map[string]http.Handler

func (h MethodHandler) ServeHTTP(w http.ResponseWriter, req *http.Request) {
  if handler, ok := h[req.Method]; ok {
    handler.ServeHTTP(w, req)
  } else {
    allow := []string{}
    for k := range h {
      allow = append(allow, k)
    }
    sort.Strings(allow)
    w.Header().Set("Allow", strings.Join(allow, ", "))
    if req.Method == "OPTIONS" {
      w.WriteHeader(http.StatusOK)
    } else {
      http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
    }
  }
}

Method 405 Method Not Allowed if the Method is Not registered. Except for one method, OPTIONS. This method returns which methods are supported by Allow header.

redirect

Handler.canonicalHost can redirect the request to the specified domain name and specify the redirect response code. This is useful for multiple domain names on the same server:

func index(w http.ResponseWriter, r *http.Request) {
  fmt.Fprintln(w, "hello world")
}

func main() {
  r := mux.NewRouter()
  r.Use(handlers.CanonicalHost("http://www.gorillatoolkit.org", 302))
  r.HandleFunc("/", index)
  http.Handle("/", r)
  log.Fatal(http.ListenAndServe(":8080", nil))
}

Above all the requests to a 302 redirect to http://www.gorillatoolkit.org.

The implementation of CanonicalHost is also simple:

func CanonicalHost(domain string, code int) func(h http.Handler) http.Handler {
  fn := func(h http.Handler) http.Handler {
    return canonical{h, domain, code}
  }

  return fn
}

Key types Canonical:

type canonical struct {
  h      http.Handler
  domain string
  code   int
}

Core methods:

func (c canonical) ServeHTTP(w http.ResponseWriter, r *http.Request) { dest, err := url.Parse(c.domain) if err ! = nil { c.h.ServeHTTP(w, r) return } if dest.Scheme == "" || dest.Host == "" { c.h.ServeHTTP(w, r) return } if ! strings.EqualFold(cleanHost(r.Host), dest.Host) { dest := dest.Scheme + "://" + dest.Host + r.URL.Path if r.URL.RawQuery ! = "" { dest += "?" + r.URL.RawQuery } http.Redirect(w, r, dest, c.code) return } c.h.ServeHTTP(w, r) }

The domain name is not valid or does not specify the protocol (Scheme) or the domain name (Host) under the request is not forwarded.

Recovery

Before, we implemented the PanicRecover middleware to avoid Panic during request processing. Handlers provide a reCoveryHandler that can be used directly:

func PANIC(w http.ResponseWriter, r *http.Request) {
  panic(errors.New("unexpected error"))
}

func main() {
  r := mux.NewRouter()
  r.Use(handlers.RecoveryHandler(handlers.PrintRecoveryStack(true)))
  r.HandleFunc("/", PANIC)
  http.Handle("/", r)
  log.Fatal(http.ListenAndServe(":8080", nil))
}

The option printreCoveryStack means to output stack information when PANIC.

The implementation of RecoveryHandler is basically the same as the one we wrote ourselves:

type recoveryHandler struct { handler http.Handler logger RecoveryHandlerLogger printStack bool } func (h recoveryHandler) ServeHTTP(w http.ResponseWriter, req *http.Request) { defer func() { if err := recover(); err ! = nil { w.WriteHeader(http.StatusInternalServerError) h.log(err) } }() h.handler.ServeHTTP(w, req) }

conclusion

There are many open source Go Web middleware implementations available on GitHub that can be used directly to avoid reinventing the wheel. Handlers are lightweight and easy to use with standard library NET/HTTP and Gorilla routing library MUX.

If you find a fun and useful Go library, please submit an issue😄 on Go Daily’s GitHub

reference

  1. Gorilla/handlers GitHub:github.com/gorilla/handlers
  2. Go a library GitHub:https://github.com/darjun/go-daily-lib daily

I

My blog: https://darjun.github.io

Welcome to pay attention to my WeChat public number [GOUPUP], learn together, make progress together ~