Serverless combat: to build personal reading tracking system | 🏆 7th essay technology projects

🏆 issue 7 | all things can be Serverless technology projects

Reading habits and personal knowledge management system

In the age of the Internet, knowledge has arguably never been cheaper, but good knowledge is nothing more than information noise if it has no value to the individual. In my personal knowledge management: the three forms of knowledge This article use the material – > information – > knowledge such as path to explain the flow of information, how fast and convenient and effective to collect materials, and then the sorting into valuable personal knowledge system structure, in this information era of severe fragmentation has become particularly important. In the knowledge Management Path of eliminating the false and preserving the true, the article also elaborated how to integrate the fragmented articles on the Internet into the unified later reading system. For example, sometimes I see a good article in the moments of friends, but I have no time to read it directly, or the article is worth reading again and carefully. Then I will put it into the later reading tool Instapaper, Pocket, and so on.

Read later the pain point of never finishing reading: lack of tracking

As time goes on, Instapaper will have more and more articles, just like the TODO we annotated in the code: will probably become Never Do, and Read it Later will also be widely criticized: Read it Later = Read Never. In fact, I found that one of the eternal pain points of article accumulation is that there is no effective way to track my reading needs and ability. The core reason is that the speed of reading cannot keep up with the speed of adding. As a result, they can’t evaluate their reading progress visually and arrange their reading plan rationally, so they can’t give themselves appropriate rewards and lose their motivation to read.

In my last blog post on GitHub Agile Learning, I mentioned using GitHub Issue to manage my learning schedule, which led to the idea of combining my later reading list with GitHub, Track and visualize your reading system with ZenHub’s rich charts.

Visualized Cumulative Flow Diagram

First of all, let’s take a look at the final concrete effect Diagram. Here, WE will briefly introduce CFD (Cumulative Flow Diagram), which is a chart that enables you to quickly understand the overall situation of the project or product. It focuses on the Flow efficiency of value. The flow of value is most directly reflected in the number of demand cards in each queue.

Little’s Law tells us that Delivery time depends on Work In Progress (WIP). WIP refers to all work that has been initiated but not yet completed, e.g., all work between Analysis and Done. The first thing you have to watch out for is the WIP. If the WIP increases, the delivery date is at risk. The most effective Release Report from ZenHub is the prediction of completion dates, in short, combined with agile methods, using project management to manage my own reading list. Of course, I’m also exploring this further. But each time you look at the chart, you gain more control and understanding of your reading list, which at least reduces the anxiety that comes when articles pile up.

IFTTT and Serverless architecture

So what’s behind this with APIs? Of course, let’s take a quick look at the Serverless architecture before we get down to business. Serverless refers to building Web applications without worrying about how to configure the server, but this does not mean that the application will not run on the server, but rather that the management of the server can be handed over to the appropriate cloud platform as much as possible, minimizing the deployment and configuration effort of the developer. The corresponding term might be Function As a Service (FAAS), As the AWS Lambda name implies, when we are building the Serverless architecture, we are actually writing Function after Function.

Streamline processes: APIs are services

First let’s introduce IFTTT, which stands for if this then That. In layman’s terms, the function of IFTTT is to perform a preset event if one event is triggered. The so-called “thing” refers to the interesting chain reaction between various applications and services. The purpose of IFTTT is to Put the Internet to work for you. The user can set any condition you want in IFTTT, and when the condition is met, the next specified action will be triggered. It’s like a magical bridge to all the web services we use every day.

The cascading scenario we have encountered is particularly suitable for Serverless architecture. Use IFTTT and bind it to Instapaper account, ** set the actions of adding, highlighting and archiving articles as trigger conditions. The information is then sent to a specified API endpoint. ** With the GitHub Issue and ZenHub APIs ready, which can be easily integrated with IFTTT triggers and markers, we can produce a flow chart of APIs interaction:

Initialize the Webtask project

While AWS Lambda is the poster child for the Serverless architecture, it has some flaws and I think it’s been said enough, so we’re going to take a dive today and focus on introducing and using Webtask. Auth0, the company behind this service, you may not have heard of it, but you’ve known JWT for JSON Web Token, which is an open standard (RFC 7519), It is usually used in scenarios where information needs to be transmitted securely, such as Authentication and information exchange.

First let’s install the tool to initialize the project and register the account, then log in using email:

npm install -g wt-cli

wt init <YOUR-EMAIL>
Copy the code

Create the project directory, add the index.js file and add the following:

module.exports = function (cb) {
  cb(null.'Hello World')}Copy the code

After deploying the application from that directory by running the following command, click on the URL output from the console to see one of the most famous HelloWorld! :

wt create index
Copy the code

Context binding for Webtask

Webtask has a utility, webtask-Tools, which can bind applications to the Webtask context. Let’s change the simple functions exported previously to Express app bound to Webtask. Then you can happily develop with Express, and everything is back to the familiar flavor:

const Express = require('express')
const Webtask = require('webtask-tools')
const bodyParser = require('body-parser')
const app = Express()

app.use(bodyParser.urlencoded({ extended: false }))
app.use(bodyParser.json())

require('./routes/reading')(app)

module.exports = Webtask.fromExpress(app)
Copy the code

Another important use of the Webtask Context is to transfer sensitive information such as security tokens at deployment time so that they are readily available within the application. ACCESS_TOKEN passed after secret in the following deployment command will be used in subsequent interactions with GitHub and ZenHub APIs.

wt create index --bundle --secret GITHUB_ACCESS_TOKEN=$GITHUB_ACCESS_TOKEN --secret ZENHUB_ACCESS_TOKEN=$ZENHUB_ACCESS_TOKEN --secret ZENHUB_ACCESS_TOKEN_V4=$ZENHUB_ACCESS_TOKEN_V4
Copy the code

# ./routes/reading.js

module.exports = (app) = > {
  app.post('/reading'.(req, res) = > {
    const { GITHUB_ACCESS_TOKEN, ZENHUB_ACCESS_TOKEN, ZENHUB_ACCESS_TOKEN_V4 } = req.webtaskContext.secrets
    }
}
Copy the code

Use GitHub Issue to track reading lists

IFTTT: Automatically creates a GitHub Issue after adding an Instapaper article

Thanks to IFTTT’s rich third-party services, IFTTT can directly create an Applet that integrates Instapaper with GitHub Issue: If a new item is saved, then create a new issue-ifttt, Automatically creates a new Issue in GitHub’s repository Issues · JimmyLv/ Reading with a title, link, and description.

But it’s not enough to just add an Issue, you also need to add that Issue to the specified Milestone to take advantage of ZenHub’s charting capabilities, Using GitHub’s Webhooks function, we can easily forward the status of Issue updates to the webTask address we specify:

Update the Milestone of the Issue using GitHub Webhook

Therefore, our Webtask needs to handle POST requests forwarded by GitHub Webhook, including the type and content of Issue, After we get an action of type ‘Opened’ (new Issue) we can add it to the Milestone:

if (action === 'opened') {
  fetch(`${url}? access_token=${GITHUB_ACCESS_TOKEN}`, {
    method: 'PATCH'.headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      milestone: 1,
    }),
  })
    .then(() = > console.info(`[END] set milestone successful! ${html_url}`))
    .catch((err) = > res.json(err))
}
Copy the code

Combined with ZenHub’s Milestone burn chart, we can clearly see how much we have left to read and compare it to our ideal reading speed to see when we can read all the articles. Some of you may wonder where these so-called Story points come from. So, the ZenHub API that we’re going to integrate.

Integrate ZenHub API: Read visualization

Update the estimate and Release of the Issue

Any changes to GitHub Issue trigger Webhook so we can process the next ‘Milestone’ action after the Issue has been added to the Milestone:

if (action === 'milestoned') {
  fetch(
    `https://api.zenhub.io/p1/repositories/${REPO_ID}/issues/${number}/estimate? access_token=${ZENHUB_ACCESS_TOKEN}`,
    {
      method: 'PUT'.headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ estimate: 1 }),
    },
  )
    .then(() = > {
      console.info(`[END] set estimate successful! ${html_url}`)
      return fetch(
        `https://api.zenhub.io/v4/reports/release/591dc19e81a6781f839705b9/items/issues? access_token=${ZENHUB_ACCESS_TOKEN_V4}`,
        {
          method: 'POST'.headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
          body: `add_issues%5B0%5D%5Bissue_number%5D=${number}&add_issues%5B0%5D%5Brepo_id%5D=${REPO_ID}`,
        },
      )
    })
    .then(() = > console.info(`[END] set release successful! ${html_url}`))
    .catch((err) = > res.json(err))
}
Copy the code

Now that we have evaluated each GitHub Issue and set the corresponding Release, all subsequent changes will be reflected in the ZenHub chart.

Close GitHub Issue after archiving Instapaper articles

Having said that, let’s not forget that the very core of the entire reading system is still “reading”. ** Among many later reading tools, I love Instapaper very much and have not transferred to Diigo because of its excellent, simple and pure reading experience, which allows people to focus on reading itself. Wouldn’t it be nice to make all the Premium features like full text search, unlimited highlighting/notes, and speed reading free after being bought by Pinterest?

After reading and archiving, the last step is to close the Issue on GitHub. However, IFTTT’s GitHub service does not provide a close Issue interface, so we have to create one by ourselves using the Maker newly launched by IFTTT. Instapaper is designed as an IF trigger, and Maker is used to issue a Web request, which can be GET,PUT, POST, HEAD, DELETE, PATCH or OPTIONS. You can even specify Content Type and Body.

app.get('/reading'.(req, res) = > {
    const { GITHUB_ACCESS_TOKEN } = req.webtaskContext.secrets
    const title = req.query.title
    let keyword = encodeURIComponent(title.replace(/\s/g.'+'))

    fetch(`https://api.github.com/search/issues?q=${keyword}%20repo:jimmylv/reading`)
      .then(response= > response.json())
      .then(data= > {
        console.info('[RESULT]', data)
        if (data.total_count > 0) {
          data.items.forEach(({ url, html_url }) = >
            fetch(`${url}? access_token=${GITHUB_ACCESS_TOKEN}`, {
              method: 'PATCH'.headers: { 'Content-Type': 'application/json' },
              body: JSON.stringify({ state: 'closed' }),
            })
              .then(() = > console.info(`[END] issue closed successful! ${html_url}`))
              .catch(err= > res.json('error', { error: err })))
          res.json({ message: 'Closed issue successful! '})}})}Copy the code

The above code can be used to process the GET request sent by IFTTT Marker. After obtaining the title of the article from the query parameter, we search for the corresponding Issues, and then close it through GitHub API.

At the same time, in the process of reading an article, we sometimes want to highlight the highlights of the article, or even comment on some of our own ideas and summaries. Then we can also use IFTTT Marker and Webtask routines to add to GitHub Issues comments, specific code will not be posted, more content has been put on GitHub: JimmyLv/demo serverless – mern.

Summary and follow-up plan

As time goes by, you just need to add articles and read articles in Instapaper, and the whole reading tracking system built by Serverless will help you record all traces and notes without any trouble. You only need to review, analyze and predict the effects of your reading regularly at specific times, and at the same time, with your own time statistics system, you can constantly improve your reading goals and plans.

Finally, I will consider the follow-up plan. For example, I just put the highlighted part and reading notes in Instapaper as comments on GitHub, but eventually I need to save them to my personal knowledge base, i.e. Diigo. This can also be done automatically through the API, and the parts that eventually need to be remembered intentionally need to be integrated with the Tinycards API to combat the Ebbinghaus forgetting curve.

At the same time, it is also necessary to divide the estimation points according to different types of articles and the degree of difficulty, instead of the current simple 1 point. For example, Instapaper also has the predicted reading minutes according to the word count, as well as the difficulty of reading different kinds of articles such as Chinese or English, technology or chicken soup. Thus the whole tracking system is more effective and reference.

Some references

• Agile Learning Methods based on GitHub – liqing Lu’s blog
• Build a Serverless MERN Story App With webtask. IO — Zero to Deploy: 1 — Scotch
• Trigger your Intelligent Life: IFTTT Primer – Minority
• Use IFTTT Maker DIY your Applet – Minority
• Personal Knowledge Management: Three Forms of Knowledge – lv Liqing’s blog
• End with GTD method read later – Minority
• Continuous Innovation, Continuous Improvement: The Way to Eliminate the False and Retain the True knowledge Management – Lv Liqing’s blog
• Cumulative Flow Diagram — CFD — Efficiency cloud