- How to build an RPC Based API with Node.js
- 译 文 : Alloys Mila
- Translation from: Aliyun Translation Group
- Text link: github.com/dawn-teams/…
- Find Tong
- Proofreader: Yashi, Linguma
How to build AN RPC-based API system using NodeJS
Apis have been eroding our development efforts for a long time. Whether you’re building microservices that are only accessible to other microservices or exposed services, you need to develop apis.
Today, most apis are based on the REST specification, which is straightforward and built on top of the HTTP protocol. But for the most part, REST may not be right for you. Companies like Uber, Facebook, Google, Netflix, and others have built their own protocols for inter-service communication, and the question is when, not if, they do it.
Suppose you want to use traditional RPC, but you still want to pass JSON data over HTTP. How do you do this via Node.js? Read on.
The following two points should be made before reading this tutorial
- You should have at least some actual experience with Node.js
- To get ES6 support, you need to install Node.js
v4.0.0Above version.
Design principles
In this tutorial, we will set the following two constraints for the API:
- Keep it simple (no external wrappers and complicated operations)
- API and interface documentation should be written together
now
The full source code for this tutorial is available on Github, so you can clone it. First, we need to first define the types and the methods that will operate on them (these will be the same methods called through the API).
Create a new directory and create two files in the new directory, types.js and methods.js. If you are using a Linux or MAC terminal, type the following command.
mkdir noderpc && cd noderpc
touch types.js methods.js
Copy the codeIn the types.js file, enter the following.
'use strict';
let types = {
user: {
description:'the details of the user'.props: {
name: ['string'.'required'].age: ['number'].email: ['string'.'required'].password: ['string'.'required']}},task: {
description:'a task entered by the user to do at a later time'.props: {
userid: ['number'.'required'].content: ['string'.'require'].expire: ['date'.'required']}}}module.exports = types;
Copy the codeAt first glance it seems simple enough to hold our type in a key-value object, where key is the name of the type and value is its definition. This definition includes a description (which is a piece of readable text used to generate documentation), which describes the properties in props, which is designed for documentation generation and validation, and which is exposed through module.exports.
The following is available in methods.js.
'use strict';
let db = require('./db');
let methods = {
createUser: {
description: `creates a new user, and returns the details of the new user`.params: ['user:the user object'].returns: ['user'],
exec(userObj) {
return new Promise((resolve) = > {
if (typeof(userObj) ! = ='object') {
throw new Error('was expecting an object! ');
}
// you would usually do some validations here
// and check for required fields
// attach an id the save to db
let _userObj = JSON.parse(JSON.stringify(userObj));
_userObj.id = (Math.random() * 10000000) | 0; // binary or, converts the number into a 32 bit integerresolve(db.users.save(userObj)); }); }},fetchUser: {
description: `fetches the user of the given id`.params: ['id:the id of the user were looking for'].returns: ['user'],
exec(userObj) {
return new Promise((resolve) = > {
if (typeof(userObj) ! = ='object') {
throw new Error('was expecting an object! ');
}
// you would usually do some validations here
// and check for required fields
// fetchresolve(db.users.fetch(userObj.id) || {}); }); }},fetchAllUsers: {
released:false;
description: `fetches the entire list of users`.params: [].returns: ['userscollection'],
exec() {
return new Promise((resolve) = > {
// fetchresolve(db.users.fetchAll() || {}); }); }}};module.exports = methods;
Copy the codeAs you can see, it is very similar to the design of a type module, but the main difference is that each method definition contains a function called exec, which returns a Promise. This function exposes the functionality of this method, although other properties are also exposed to the user, but this must be abstracted through the API.
db.js
Our API needs to store data somewhere, but in this tutorial, we don’t want to complicate the tutorial with unnecessary NPM install, we create a very simple, native in-memory key-value store, because its data structure is of your own design, so you can change how the data is stored at any time.
Contains the following in db.js.
'use strict';
let users = {};
let tasks = {};
// we are saving everything inmemory for now
let db = {
users: proc(users),
tasks: proc(tasks)
}
function clone(obj) {
// a simple way to deep clone an object in javascript
return JSON.parse(JSON.stringify(obj));
}
// a generalised function to handle CRUD operations
function proc(container) {
return {
save(obj) {
// in JS, objects are passed by reference
// so to avoid interfering with the original data
// we deep clone the object, to get our own reference
let _obj = clone(obj);
if(! _obj.id) {// assign a random number as ID if none exists
_obj.id = (Math.random() * 10000000) | 0;
}
container[_obj.id.toString()] = _obj;
return clone(_obj);
},
fetch(id) {
// deep clone this so that nobody modifies the db by mistake from outside
return clone(container[id.toString()]);
},
fetchAll() {
let _bunch = [];
for (let item in container) {
_bunch.push(clone(container[item]));
}
return _bunch;
},
unset(id) {
deletecontainer[id]; }}}module.exports = db;
Copy the codeAn important one is the proc function. You can easily add, edit, and delete values on an object by taking it and wrapping it in a closure with a set of functions. If you’re not familiar with closures, you should read about JavaScript closures in advance.
So, now that we’re basically done with the program, we can store and retrieve data, and we can manipulate that data, what we need to do is expose it over the network. Therefore, the last part is to implement the HTTP service.
This is where most of us want to use Express, but we don’t, so we’ll use the HTTP module that comes with the node and implement a very simple routing table around it.
As expected, we continue to create the server.js file. In this file we tie everything together as shown below.
'use strict';
let http = require('http');
let url = require('url');
let methods = require('./methods');
let types = require('./types');
let server = http.createServer(requestListener);
const PORT = process.env.PORT || 9090;
Copy the codeThe beginning of the file introduces what we need to create an HTTP service using http.createserver. RequestListener is a callback function, which we will define later. And we identify the port on which the server will listen.
After this code we define the routing table, which specifies the different URL paths to which our application will respond.
// we'll use a very very very simple routing mechanism
// don't do something like this in production, ok technically you can...
// probably could even be faster than using a routing library :-D
let routes = {
// this is the rpc endpoint
// every operation request will come through here
'/rpc': function (body) {
return new Promise((resolve, reject) = > {
if(! body) {throw new (`rpc request was expecting some data... ! `);
}
let _json = JSON.parse(body); // might throw error
let keys = Object.keys(_json);
let promiseArr = [];
for (let key of keys) {
if (methods[key] && typeof (methods[key].exec) === 'function') {
let execPromise = methods[key].exec.call(null, _json[key]);
if(! (execPromiseinstanceof Promise)) {
throw new Error(`exec on ${key} did not return a promise`);
}
promiseArr.push(execPromise);
} else {
let execPromise = Promise.resolve({
error: 'method not defined'}) promiseArr.push(execPromise); }}Promise.all(promiseArr).then(iter= > {
console.log(iter);
let response = {};
iter.forEach((val, index) = > {
response[keys[index]] = val;
});
resolve(response);
}).catch(err= > {
reject(err);
});
});
},
// this is our docs endpoint
// through this the clients should know
// what methods and datatypes are available
'/describe': function () {
// load the type descriptions
return new Promise(resolve= > {
let type = {};
let method = {};
// set types
type = types;
//set methods
for(let m in methods) {
let _m = JSON.parse(JSON.stringify(methods[m]));
method[m] = _m;
}
resolve({
types: type,
methods: method }); }); }};Copy the codeThis is a very important part of the program because it provides the actual interface. We have a set of endpoints, each corresponding to a handler function that is called when the path matches. By design rules, each handler must return a Promise.
RPC endpoint takes a JSON object containing the contents of the request, parses each request into the corresponding method in the methods.js file, calls the exec function of that method, and either returns the result or throws an error.
Describe endpoint scans a description of methods and types and returns this information to the caller. Make it easy for developers who use the API to know how to use it.
Now let’s add the function requestListener we discussed earlier and start the service.
// request Listener
// this is what we'll feed into http.createServer
function requestListener(request, response) {
let reqUrl = `http://${request.headers.host}${request.url}`;
let parseUrl = url.parse(reqUrl, true);
let pathname = parseUrl.pathname;
// we're doing everything json
response.setHeader('Content-Type'.'application/json');
// buffer for incoming data
let buf = null;
// listen for incoming data
request.on('data', data => {
if (buf === null) {
buf = data;
} else{ buf = buf + data; }});// on end proceed with compute
request.on('end', () = > {letbody = buf ! = =null ? buf.toString() : null;
if (routes[pathname]) {
let compute = routes[pathname].call(null, body);
if(! (computeinstanceof Promise)) {
// we're kinda expecting compute to be a promise
// so if it isn't, just avoid it
response.statusCode = 500;
response.end('oops! server error! ');
console.warn(`whatever I got from rpc wasn't a Promise! `);
} else {
compute.then(res= > {
response.end(JSON.stringify(res))
}).catch(err= > {
console.error(err);
response.statusCode = 500;
response.end('oops! server error! '); }); }}else {
response.statusCode = 404;
response.end(`oops! ${pathname} not found here`)}}}// now we can start up the server
server.listen(PORT);
Copy the codeThis function is called every time a new request comes in, waits for the data, and then looks at the path and the corresponding processing method based on the path matching into the routing table. Then use server.listen to start the service.
Now we can start the service by running Node Server.js in the directory, and then using Postman or one of your familiar API debugging tools, send a request to http://localhost{PORT}/rpc with the following JSON content in the request body.
{
"createUser": {
"name":"alloys mila"."age":24}}Copy the codeThe server will create a new user based on the request you submitted and return the response. An RPC-based, well-documented API system has been built.
Note that we have not done any parameter validation for this tutorial interface, and you will have to manually ensure that the data is correct when calling the tests.