Go language development RESTFul JSON API

RESTful API is widely used in Web project development. This article explains how to implement RESTful JSON API step by step in Go language, and also involves RESTful design topics.

Maybe we’ve used all kinds of apis before, and when we come across one that’s badly designed, it’s just overwhelming. Hopefully, this article will give you an idea of a well-designed RESTful API.

What is the JSON API?

Before JSON, many web sites exchanged data through XML. If you move on to JSON after you’ve worked with XML, there’s no doubt that you’ll feel pretty good about the world. We will not go into the JSON API here. If you are interested, please refer to the JSON API.

Basic Web server

Ultimately, RESTful services are Web services first and foremost. So let’s start by looking at how a basic Web server is implemented in the Go language. The following example implements a simple Web server that responds to any request with the requested URL.

package main

import (
    "fmt"
    "html"
    "log"
    "net/http"
)

func main() {
    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        fmt.Fprintf(w, "Hello, %q", html.EscapeString(r.URL.Path))
    })

    log.Fatal(http.ListenAndServe(":8080", nil))
}Copy the code

The basic Web server above uses HandleFunc and ListenAndServe, two basic functions of the Go standard library.

func HandleFunc(pattern string, handler func(ResponseWriter, *Request)) {
    DefaultServeMux.HandleFunc(pattern, handler)
}

func ListenAndServe(addr string, handler Handler) error {
    server := &Server{Addr: addr, Handler: handler}
    return server.ListenAndServe()
}Copy the code

By running the basic Web service above, you can access it directly from your browser at http://localhost:8080.

> go run basic_server.goCopy the code

Add the routing

Although the library includes the Router, I find that many people are confused about how it works. I have used various third-party router libraries in my own projects. The most notable is Gorilla Web ToolKit’s MUx Router.

Another popular router is a package called Httprouter from Julien Schmidt.

package main

import (
    "fmt"
    "html"
    "log"
    "net/http"

    "github.com/gorilla/mux"
)

func main() {
    router := mux.NewRouter().StrictSlash(true)
    router.HandleFunc("/", Index)

    log.Fatal(http.ListenAndServe(":8080", router))
}

func Index(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "Hello, %q", html.EscapeString(r.URL.Path))
}Copy the code

To run the above code, first use go Get to get the source code for the MUx Router:

> go get github.com/gorilla/muxCopy the code

The above code creates a basic router that assigns the request “/” to an Index handler, which is executed when the client requests http://localhost:8080/.

Before if you are careful enough, you will find that the basic web services can visit http://localhost:8080/abc normal response: “Hello,”/ABC “, but after add the routing, can visit http://localhost:8080. The reason is simple, because we only added resolution for “/”, the other routes are invalid, so they are all 404.

Create some basic routes

Now that we have added routes, we can add more routes.

Suppose we wanted to create a basic ToDo application, so our code would look like this:

package main import ( "fmt" "log" "net/http" "github.com/gorilla/mux" ) func main() { router := mux.NewRouter().StrictSlash(true) router.HandleFunc("/", Index) router.HandleFunc("/todos", TodoIndex) router.HandleFunc("/todos/{todoId}", TodoShow) log.Fatal(http.ListenAndServe(":8080", router)) } func Index(w http.ResponseWriter, r *http.Request) { fmt.Fprintln(w, "Welcome!" ) } func TodoIndex(w http.ResponseWriter, r *http.Request) { fmt.Fprintln(w, "Todo Index!" ) } func TodoShow(w http.ResponseWriter, r *http.Request) { vars := mux.Vars(r) todoId := vars["todoId"] fmt.Fprintln(w, "Todo Show:", todoId) }Copy the code

Here we added two more routes: todos and todos/{todoId}.

This is where RESTful API design begins.

Notice that for the last route we added a variable to the end of the route called todoId.

This allows us to pass the ID to the route and respond to the request with a specific record.

The basic model

With the route now in place, it’s time to create a Model that you can use to send and retrieve data. In Go, models can be implemented using constructs, whereas in other languages models are generally implemented using classes.

package main

import (
    "time"
)

type Todo struct {
    Name      string
    Completed bool
    Due       time.Time
}

type Todos []TodoCopy the code

Above we define a Todo structure that represents the item to be done. We also have a type called Todos, which represents a to-do list, which is an array, or a shard.

As you’ll see later, this can be very useful.

Return some JSON

Now that we have the basic model, we can simulate some real responses. We can simulate some static lists of data for TodoIndex.

package main

import (
    "encoding/json"
    "fmt"
    "log"
    "net/http"

    "github.com/gorilla/mux"
)

// ...

func TodoIndex(w http.ResponseWriter, r *http.Request) {
    todos := Todos{
        Todo{Name: "Write presentation"},
        Todo{Name: "Host meetup"},
    }

    json.NewEncoder(w).Encode(todos)
}

// ...Copy the code

We have now created a static Todos shard in response to the client request. Note that if you request to http://localhost:8080/todos, you will get the following response:

[
    {
        "Name": "Write presentation",
        "Completed": false,
        "Due": "0001-01-01T00:00:00Z"
    },
    {
        "Name": "Host meetup",
        "Completed": false,
        "Due": "0001-01-01T00:00:00Z"
    }
]Copy the code

A better Model

For seasoned veterans, you may have spotted a problem. Each key that responds to JSON is typed with an initial letter, which may seem trivial, but it is not customary to capitalize a key that responds to JSON. Here’s how to solve the problem:

type Todo struct {
    Name      string    `json:"name"`
    Completed bool      `json:"completed"`
    Due       time.Time `json:"due"`
}Copy the code

It is simple to add a tag attribute to the structure to fully control how the structure is marshalled into JSON.

Break up the code

So far, all of our code is in one file. It’s messy. It’s time to split the code. We can split the code into the following files by function.

We are going to create the following file and move the corresponding code into the concrete code file:

  • Main. go: program entry file.
  • Handlers. go: routing related handlers.
  • Routes. Go: routing.
  • Todo. go: Todo-related code.
package main import ( "encoding/json" "fmt" "net/http" "github.com/gorilla/mux" ) func Index(w http.ResponseWriter, r *http.Request) { fmt.Fprintln(w, "Welcome!" ) } func TodoIndex(w http.ResponseWriter, r *http.Request) { todos := Todos{ Todo{Name: "Write presentation"}, Todo{Name: "Host meetup"}, } if err := json.NewEncoder(w).Encode(todos); err ! = nil { panic(err) } } func TodoShow(w http.ResponseWriter, r *http.Request) { vars := mux.Vars(r) todoId := vars["todoId"] fmt.Fprintln(w, "Todo show:", todoId) }Copy the code
package main import ( "net/http" "github.com/gorilla/mux" ) type Route struct { Name string Method string Pattern string  HandlerFunc http.HandlerFunc } type Routes []Route func NewRouter() *mux.Router { router := mux.NewRouter().StrictSlash(true) for _, route := range routes { router. Methods(route.Method). Path(route.Pattern). Name(route.Name). Handler(route.HandlerFunc)  } return router } var routes = Routes{ Route{ "Index", "GET", "/", Index, }, Route{ "TodoIndex", "GET", "/todos", TodoIndex, }, Route{ "TodoShow", "GET", "/todos/{todoId}", TodoShow, }, }Copy the code
package main

import "time"

type Todo struct {
    Name      string    `json:"name"`
    Completed bool      `json:"completed"`
    Due       time.Time `json:"due"`
}

type Todos []TodoCopy the code
package main

import (
    "log"
    "net/http"
)

func main() {

    router := NewRouter()

    log.Fatal(http.ListenAndServe(":8080", router))
}Copy the code

A better Routing

During our refactoring, we created a routes file with more functionality. This new file makes use of a structure that contains multiple pieces of routing information. Note that here we can specify the type of request, such as GET, POST, DELETE, and so on.

Outputting Web Logs

I also included an ulterior motive in the split routing file. As you’ll see later, it’s easy to use additional functions to decorate the HTTP processor after the split.

First we need the ability to log Web requests, as many popular Web servers do. In Go, there is no Web logging package or functionality on the side of the standard library, so we need to create our own.

package logger

import (
    "log"
    "net/http"
    "time"
)

func Logger(inner http.Handler, name string) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        start := time.Now()
        inner.ServeHTTP(w, r)

        log.Printf(
            "%s\t%s\t%s\t%s",
            r.Method,
            r.RequestURI,
            name,
            time.Since(start),
        )
    })
}Copy the code

Above we defined a Logger function that wraps the handler.

This is a very standard idiom in the Go language. In fact, it’s the canonical way of functional programming. Very efficient, we just pass the Handler to the function, and it wraps the Handler around it, adding web logging and time statistics.

Apply the Logger decorator

To apply Logger modifiers, we can create a router. We simply package all of our current routes into the router. Change the NewRouter function as follows:

func NewRouter() *mux.Router {

    router := mux.NewRouter().StrictSlash(true)
    for _, route := range routes {
        var handler http.Handler

        handler = route.HandlerFunc
        handler = Logger(handler, route.Name)

        router.
            Methods(route.Method).
            Path(route.Pattern).
            Name(route.Name).
            Handler(handler)
    }

    return router
}Copy the code

Now run our program again and we should see the log look something like this:

2014/11/19 12:41:39 GET/Todos TodoIndex 148.324usCopy the code

This routing file is crazy… Let’s refactor it

The routes file has now become slightly larger, so let’s break it up into multiple files:

  • routes.go
  • router.go
package main import "net/http" type Route struct { Name string Method string Pattern string HandlerFunc http.HandlerFunc  } type Routes []Route var routes = Routes{ Route{ "Index", "GET", "/", Index, }, Route{ "TodoIndex", "GET", "/todos", TodoIndex, }, Route{ "TodoShow", "GET", "/todos/{todoId}", TodoShow, }, }Copy the code
package main

import (
    "net/http"

    "github.com/gorilla/mux"
)

func NewRouter() *mux.Router {
    router := mux.NewRouter().StrictSlash(true)
    for _, route := range routes {
        var handler http.Handler
        handler = route.HandlerFunc
        handler = Logger(handler, route.Name)

        router.
            Methods(route.Method).
            Path(route.Pattern).
            Name(route.Name).
            Handler(handler)

    }
    return router
}Copy the code

And take on some responsibility

So far, we’ve had some pretty good boilerplate code, and it’s time to revisit our processor. We need a little more responsibility. First modify TodoIndex by adding the following two lines:

func TodoIndex(w http.ResponseWriter, r *http.Request) { todos := Todos{ Todo{Name: "Write presentation"}, Todo{Name: "Host meetup"}, } w.Header().Set("Content-Type", "application/json; charset=UTF-8") w.WriteHeader(http.StatusOK) if err := json.NewEncoder(w).Encode(todos); err ! = nil { panic(err) } }Copy the code

Two things are happening here. First, we set the response type and tell the client to expect JSON. Second, we explicitly set the response status code.

The Go language net/ HTTP server will try to guess the output content type for us (though not always correctly), but now that we know exactly what the response type is, we should always set it ourselves.

Wait a minute, where is our database?

Obviously, if we’re going to create RESTful apis, we need some place to store and retrieve data. However, this is beyond the scope of this article, so we will simply create a very rudimentary mock database (non-thread-safe).

We create a rebo. go file with the following contents:

package main

import "fmt"

var currentId int

var todos Todos

// Give us some seed data
func init() {
    RepoCreateTodo(Todo{Name: "Write presentation"})
    RepoCreateTodo(Todo{Name: "Host meetup"})
}

func RepoFindTodo(id int) Todo {
    for _, t := range todos {
        if t.Id == id {
            return t
        }
    }
    // return empty Todo if not found
    return Todo{}
}

func RepoCreateTodo(t Todo) Todo {
    currentId += 1
    t.Id = currentId
    todos = append(todos, t)
    return t
}
func RepoDestroyTodo(id int) error {
    for i, t := range todos {
        if t.Id == id {
            todos = append(todos[:i], todos[i+1:]...)
            return nil
        }
    }
    return fmt.Errorf("Could not find Todo with id of %d to delete", id)
}Copy the code

Add ID to Todo

We created the mock database, we used and assigned ids, so we need to update our Todo structure accordingly.

package main

import "time"

type Todo struct {
    Id        int       `json:"id"`
    Name      string    `json:"name"`
    Completed bool      `json:"completed"`
    Due       time.Time `json:"due"`
}

type Todos []TodoCopy the code

Update our TodoIndex

To use the database, we need to retrieve the data in TodoIndex. Modify the code as follows:

func TodoIndex(w http.ResponseWriter, r *http.Request) { w.Header().Set("Content-Type", "application/json; charset=UTF-8") w.WriteHeader(http.StatusOK) if err := json.NewEncoder(w).Encode(todos); err ! = nil { panic(err) } }Copy the code

POST JSON

So far, we’ve just been exporting JSON; now it’s time to go into storing some JSON.

Add the following routes to the routes.go file:

Route{
    "TodoCreate",
    "POST",
    "/todos",
    TodoCreate,
},Copy the code

The Create routing

func TodoCreate(w http.ResponseWriter, r *http.Request) { var todo Todo body, err := ioutil.ReadAll(io.LimitReader(r.Body, 1048576)) if err ! = nil { panic(err) } if err := r.Body.Close(); err ! = nil { panic(err) } if err := json.Unmarshal(body, &todo); err ! = nil { w.Header().Set("Content-Type", "application/json; charset=UTF-8") w.WriteHeader(422) // unprocessable entity if err := json.NewEncoder(w).Encode(err); err ! = nil { panic(err) } } t := RepoCreateTodo(todo) w.Header().Set("Content-Type", "application/json; charset=UTF-8") w.WriteHeader(http.StatusCreated) if err := json.NewEncoder(w).Encode(t); err ! = nil { panic(err) } }Copy the code

First we open the body of the request. Note that we use IO.LimitReader. This is a good way to protect your server from malicious attacks. What if someone wanted to send 500GB of JSON to your server?

After we read the body, we deconstruct the Todo structure. If it fails, we respond correctly, using the appropriate response code 422, but we still respond back using JSON. This allows the client to understand that something went wrong and have a way of knowing exactly what went wrong.

Finally, if all passes, we respond with a 201 status code indicating that the requested entity has been successfully created. We will also respond with json representing the entity we created, which will contain an ID that the client may need next.

POST some JSON

Now that we have the pseudo REPO and the CREATE route, we need to post some data. We use curl to do this with the following command:

curl -H "Content-Type: application/json" -d '{"name": "New Todo"}' http://localhost:8080/todosCopy the code

If you accessed through http://localhost:8080/todos again, probably will get the following response:

[
    {
        "id": 1,
        "name": "Write presentation",
        "completed": false,
        "due": "0001-01-01T00:00:00Z"
    },
    {
        "id": 2,
        "name": "Host meetup",
        "completed": false,
        "due": "0001-01-01T00:00:00Z"
    },
    {
        "id": 3,
        "name": "New Todo",
        "completed": false,
        "due": "0001-01-01T00:00:00Z"
    }
]Copy the code

Things we haven’t done yet

Although we have made a good start, there is still much to be done:

  • Version control: What if we need to change the API and it changes completely? Maybe we need to start our route with /v1/prefix?
  • Licensing: Unless these are public/free apis, we may also need licensing. Learn about JSON Web Tokens.

ETag – If you are building something that needs to be extended, you may need to implement eTag.

What else?

For all projects, it starts small but quickly gets out of control. But if we want to take it to the next level and get it production-ready, there are a few additional things we need to do:

  • Refactoring was extensive.
  • Create several packages for these files, such as JSON helpers, modifiers, handlers, and so on.
  • Testing, you can’t forget that. We didn’t do any tests here. Testing is a must for production systems.

The source code

Github.com/corylanou/….

conclusion

For me, the most important thing to remember is that we’re going to build a responsible API. Sending appropriate status codes, headers, etc., is key to widespread API adoption. I hope this article gets you started on your API as soon as possible.

Refer to the link

  • Go RESTful JSON API implementation
  • JSON API
  • Gorilla Web Toolkit
  • httprouter
  • JSON Web Tokens
  • eTag