Tencent Cloud SCF + Typescript practice

purpose

Serverless is becoming more and more popular recently. I am in training and have time to try some new things, so TAKE this time to use Serverless, try to use typescript and NodeJS development, a small tool deployed on Tencent cloud SCF. Explore the best practices of typescript + NodeJS + SCF.

Project introduction

idea

Because I usually follow some novels, cartoons, TV dramas, etc., BUT WHEN they are updated, I generally do not perceive them, and I only go to their websites to check whether they are updated when I am free or before going to bed. If there is a tool that can tell me when they are updated, what they are updated, then I don’t need to use my phone to check, which is a certain convenience for me.

This was before I touched Serverless. My thinking was this: It’s easy to write a program like this, but I need to buy a machine to deploy it. If there is a problem can not be found in time and get on the machine to check the log; You have to control the logic of the program’s timed crawl and so on. In summary, the cost of implementing and maintaining such a program far outweighs the convenience it brings to me, which makes me think but not act.

However, after understanding the concept of Serverless, the problems that hindered my actions mentioned above are no longer problems, such as deployment problems. To use Serverless means to create functions by using the developer tools provided by the cloud provider, and to package and upload the code, the deployment will be successful. Another example is the timing crawl logic, using the timing trigger capability provided by it, which greatly facilitates my development and makes me more focused on code implementation. I won’t officially talk about the concept and benefits of Serverless here, just from a developer’s point of view.

practice

The flow chart

The whole flowchart of the program is shown in the figure below. The logic is very simple. The purpose of this project is not to realize a powerful function, but to explore the practical way of TS + Node + SCF.
The development of

Developing a NodeJS program that can run in SCF is not much different from developing a traditional NodeJS program in terms of language. The obvious difference is that we develop with an entry function, such as this:

For a more detailed primer, see the documentation here. Follow the documentation step by step to write a simple function. Let’s get back to business.
- Environment set up
  - First, to facilitate development, it is recommended to install the command line tool or VScode plug-in provided by Tencent Cloud SCF. However, the vscode plug-in was not released when I developed it, so I mainly used the command line tool here. The installation and use of the command line tool documents can be seen here for details.
  - Once installed, use SCF init to create a project and a project file. It is very simple. There are only four files in total. The fourth template.yaml file is called a template file, which is simply a file that describes the function, such as its environment variables, trigger types, and so on.
  - Json. If you don’t have typescript installed, go to typescript’s official website. TSC –init quickly generates tsconfig.json and configures it to your own needs.
  - Then, write NPM scripts.
    
    There are three main operations: build, dev, and deploy.
    
    NPM scripts can be used to concatenate typescript compilation with SCF CLI native debugging, packaging, and deployment, making typing commands concise and semantic
  - Finally, associate the local repository with the remote repository. (Here’s an optimization: One scenario is that the user has created a Git repository and now needs to write the code in the repository into SCF mode and use it with the SCF CLI. Currently, SCF CLI only supports init of a complete project. If you can quickly generate debugging and deployed YAML in an existing project, A convenient feature for developers)
- coding
  
  My main logic code is divided into the files above.
  - Index, nothing to say, is just an entry file that combines the logic of the rest of the modules.
  - The config file and config_extra, config_extra file put my privacy configuration, such as redis host, port and password, and mail authorization code, etc. together. Gitignore will not commit to remote Git repository. The config file, on the other hand, imports the configuration from the config_extra file, merges with some common configurations, and then outputs to the modules. (Note: you can also make good use of the environment variable function provided by SCF here, which is suitable for this scenario, detailed documentation)
  - Config_extra_demo tells other developers how config_extra files should be written.
  - Mailer, which encapsulates the initialization of the mail service and the method of sending the mail
  - Redis, which encapsulates redis connections and synchronizes set and GET methods
  - Task, which briefly encapsulates the general logic for initialization and execution. However, as the tool expands, you still need to consider how to abstract and generalize.
  - Util, which encapsulates common methods such as retry methods, wraps asynchronous functions.
  The above brief introduction of the main logic code files, specific implementation, interested in you can move to github address to view
debugging

The NPM scripts I wrote have NPM run dev in them. When I was developing this project, I executed NPM run dev for debugging. As a reminder, the test environment usually needs to be isolated from the formal environment. So create a new env.json file and fill it in
```
{
  "NODE_ENV": development
}
Copy the code
```
And the NPM dev command in the script to NPM run build && SCF local generate – event timer timeup | SCF native invoke – the template template.yaml –env-vars env.json

Then use the process.env.node_env variable in the configuration file to determine whether the environment is a test environment or a formal environment, and fill in the configuration of the services that the environment depends on.
The deployment of

The above mentioned so much, in fact, is not what I want to express, because I did not meet some difficult problems above. However, in the deployment process, I found that typescript could not meet the current deployment requirements of Tencent Cloud SCF and the file directory management of the project perfectly.

After discussing it later with colleagues, it’s still a good idea to strike a balance. Here is a process of my many attempts.

If you write nodeJS programs using JS instead of typescript, you don’t need to compile. When you deploy a function, you just package it and deploy it. With typescript, however, there is an additional step to compile TS code into JS code. In order to manage the file directory of the project well, I prefer to store TS and JS files in different folders. For example, SRC folder stores TS files, while dist is the COMPILED JS file. That’s how I started with my file directory.
- First try
  
  File directory:
  
  tsconfig.json
  
  Specifies that the ts file in the SRC folder be compiled and output to the dist folder
  
  template.yaml
  
  CodeUri points to the dist folder
  
  Based on the above configuration, it is possible to debug locally. But when deployed to the cloud, the test failed. If you’re familiar with it, you’ll notice immediately that node_modules is not packaged. None of the third-party libraries that the main logic code relies on can be found, and the test fails.
- Second attempt
  
  For my first try, I solved the problem by using NPM Scripts’ Pre hook, editing the TS code before deploying, copying node_modules to the dist folder, and then packaging the deployment.
  
  package.json
  
  copy_node_modules.js
  
  Files in folder dist
  
  While this works and manages the local file directory reasonably, the code submitted to the cloud is compiled and unreadable, just a working mess and the project code is incomplete. So in my opinion, it would be perfect if the project code developed locally is the same as the project code delivered to the cloud, without the need for additional scripts to castrate it. Although the webIDE of Tencent cloud SCF console can only view entry files at present, cloud Studio will be connected to at least see each file of the entire code folder, and maybe support typescript compilation online in the future (though I don’t know if it is possible). So I tried a third time.
- Third attempt
  
  I have an idea: the Handler specified in template.yaml, the entry function, is changed from index.main_handler to folder /index.main_handler, which means the entry function can be in a folder.
  
  I wrote the Handler at template.yaml as dist/index.main_handler, and the CodeUri as the root directory so I could package the entire folder, and then specified the Handler as main_handler for the index file in the dist folder.
  
  template.yaml
  
  When debugging locally, it was successful!
  
  But at the time of deployment,
  
  Well, I don’t think this scheme can work, because it doesn’t meet the requirements of SCF and can’t pass the verification.
- Fourth attempt
  
  This is my fourth attempt. But it’s not perfect, taking a step back in file management and allowing TS to be put together with compiled JS. This allows the whole project to be packaged and run, but ts and JS together, file management is not very reasonable. The modifications are as follows:
  
  The index.ts file is moved from the SRC folder to the root directory
  
  tsconfig.json
  
  Edit the ts files in the index.ts and SRC folders in the root directory to remove node_modules and output them to the root directory
  
  template.yaml
  
  Change CodeUri to root and Handler to index.main_handler, as generated by the CLI
  
  Compiled result
  
  Finally, SCF deployed to the cloud can be run, and the whole project is packaged up. In the future, Tencent cloud SCF is connected to Cloud Studio, and the file architecture seen by webIDE is consistent with that seen locally.
- Fifth try
  
  In a roundabout way, sometimes the solution is simple. After discussion with colleagues in the group, a big shot colleague pointed out:
  
  Can I write an index file in the root directory and call the compiled entry method of the index file? .
  
  Wake up a dream! Yeah, I didn’t notice it at the beginning, but you can solve it this way, thinking in a circle, not out of the way. It’s cheap to do, and it achieves the ideal I talked about earlier:
  
  The project code developed locally is the same as the project code delivered to the cloud and does not need to be castrated with additional scripts
  
  To do this, put the typescript file in the SRC folder, the edited JS file in the dist folder, and write an index.js file in the root directory. The main_hanlder method in the file calls the compiled index file entry function. Here’s the core code.
  
  index.js
  
  tsconfig.json
  
  template.json
  
  Compiled result
results

Just to show you what happens on the code line.

conclusion

With all that said, here’s a summary:

Although Tencent Cloud SCF doesn’t have native typescript support, there are ways to make the two work together.

First of all, there is nothing wrong with local development. The above attempts are all aimed at deploying to the cloud while local debugging is successful.

There are three possible attempts at deployment:

The first is to adapt through some additional methods, but the consistency between the project on the cloud and the actual project is not achieved, such as the second attempt.

The second is taking a step back from file management and not being extremely clear, such as the fourth attempt.

The third is to write an index.js file in the root directory, call an entry function with real logic, and do a forward, such as the fifth attempt, which I think is the best practice for now.

Finally, the above five attempts are my ideas and practices during development, which may not be correct and welcome to criticize. If you have a better way, welcome to discuss. The source code of the five attempts is all in github repository. The first four attempts all have corresponding branches. The master branch is the fifth attempt.