“This is the 15th day of my participation in the Gwen Challenge in November. Check out the details: The Last Gwen Challenge in 2021.”

As mentioned earlier, Github Actions can be customized and can be triggered in many ways, such as scheduled tasks, Git push, etc. We now try to achieve a regular push gold nuggets popular article function through it.

1 Climb the nuggets popular articles

Using Chrome’s Check check function, grab the API links to get popular articles:

https://api.juejin.cn/recommend_api/v1/article/recommend_all_feed?
Copy the code

Test implementing a simple crawler in Python:

headers = {
    "User-Agent": "Mozilla / 5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36"
}

url = "https://api.juejin.cn/recommend_api/v1/article/recommend_all_feed"
payload = {
    "sort_type": 200."limit": 20."cursor": "0"."id_type": 2."client_type": 2608
}

res = requests.post(url, json=payload, headers=headers)
data = res.json().get("data", [])

articles = []
for i in data:
    if i["item_type"] = =2:
        title = i["item_info"] ["article_info"] ["title"]
        url = "https://juejin.cn/post/{}".format(i["item_info"] ["article_id"])
    elif i["item_type"] = =14:
        title = i["item_info"] ["title"]
        url = i["item_info"] ["url"]
    else:
        title = i["item_info"] ["article_info"] ["title"]
        url = i["item_info"] ["article_info"] ["url"]
    articles.append({
        "title": title,
        "url": url
    })
Copy the code

Output:

[{'title': 'come on, use the code "draw cat"! 🏆 technical topics essay 13 contribute open ', 'url': 'https://juejin.cn/post/7024369534119182367?utm_source=web_feed1&utm_medium=banner&utm_campaign=catcat_2021111'}, {' title ':' [machine learning] know your cat ', 'url' : 'https://juejin.cn/post/7029217562600669221'}, {" title ": 'asked five localStorage soul. 5 m?? 10 m!!!' and 'url' : 'https://juejin.cn/post/7030585901524713508'}, {" title ": 'when the interviewer asks you soul torture 】 【 JavaScript precompiled', 'url' : 'https://juejin.cn/post/7030370931478364196'}, {" title ": '2021 for the last time the more challenge, play to upgrade the prize, the waiting for you to experience a', 'url' : 'https://zjsms.com/RLGH68Y/'}, {' title ', 'to their cat as the prototype, do a jumping cats small game', 'url' : 'https://juejin.cn/post/7030666347272994823'}, {' title ', 'pull force collection full, high-quality Web front-end 50 online resources ~', 'url' : "Https://juejin.cn/post/7030572979868139551"}, {' title ':' IntelliJ IDEA was out of the function of can in the cloud code? ', 'url' : "Https://juejin.cn/post/7030621203824033799"}, {' title ', '50 lines of code React Hooks used in rich text editor', 'url' : "Https://juejin.cn/post/7030584414652334093"}, {' title ', 'little Y learning algorithm 】 【 ⚡ ️ LeetCode daily clock in ⚡ ️ -- 47. Duplicate elements',' url ': 'https://juejin.cn/post/7030418271270600740'}, {' title ':' my open source with DevUI story ', 'url' : "Https://juejin.cn/post/7030414733295484935"}, {' title ':' the novice to: front-end programmers will learn basic skills -- debug JS code ', 'url' : 'https://juejin.cn/post/7030584939020042254'}, {' title ':' web front-end project - web design art training institutions (HTML + CSS + JavaScript) [source share] ', 'url' : 'https://juejin.cn/post/7030588458297098270'}, {' title ':' Python matplotlib drawing histogram ', 'url' : "Https://juejin.cn/post/7030405486860042276"}, {' title ':' crazy, interviewer asked me why I float inaccurate? ', 'url' : 'https://juejin.cn/post/7030619927274848287'}, {' title ':' the response object of introductory Flask series! 'and' url ': "Https://juejin.cn/post/7030400306454200333"}, {' title ':' in 21 diagram, the working principle of the Git thoroughly clear ', 'url' : "Https://juejin.cn/post/7030409006258585637"}, {' title ':' I go to, the Java IO also too above,,, ', 'url' : "Https://juejin.cn/post/7030599694300479518"}, {' title ':' digging force plan monthly list | Top the author list released in October 2021 ', 'url' : "Https://juejin.cn/post/7028026854669811749"}, {' title ':' [preheating | challenge cup boiling point simulation company 】 ', 'url' : 'https://juejin.cn/pin/7030629015060742151?utm_source=web20&utm_medium=banner&utm_campaign=business '}, {'title': 'those confusing design pattern, know the ~', 'url' : 'https://juejin.cn/post/7030415720676589581'}, {' title ':' the interviewer: tell me about you know Js regular expression ', 'url' : "Https://juejin.cn/post/7030418904392400926"}, {' title ':' algorithm: 557 III reversal in the string word 'and' url ': 'https://juejin.cn/post/7030388857233211405'}]Copy the code

One might ask: why does my output look so good?

This is because I used the pprint function

from pprint import pprint
pprint(articles)
Copy the code

Json content is still too plain, let’s convert it to HTML page! In Python, you can use the Jinjia library to do this by rendering templates. Let’s start by defining a template:

<! DOCTYPEhtml>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="Width = device - width, initial - scale = 1.0">
    <meta http-equiv="X-UA-Compatible" content="ie=edge">
<! -- <title>{% block title %}{% endblock %}</title>-->
</head>
<body>
    <ul>
        {% for article in articles %}
        <li>
            <a href="{{ article.url }}">{{ article.title }}</a>
        </li>
        {% endfor %}
    </ul>
</body>
</html>
Copy the code

As you can see, there is something in the template that looks a bit like Python code, which is the advantage of Jinja, which allows you to do some traversal, conditional judgment, and other processing on the data content during rendering.

The next step is to fuse the data with the template

from jinja2 import Environment, FileSystemLoader
env = Environment(loader=FileSystemLoader(". /"))
template = env.get_template("template.html")
with open("hot_article.html"."w") as f:
	f.write(template.render(articles=get_articles()))
Copy the code

The final output is:

2 Configure Github Actions

Github /workflows then create an action.yml:

name: 'juejinG Bot'

on:
  push:
  Receive notifications at 8:40 am every day
  schedule:
    - cron: '0 * * *'

jobs:
  bot:
    runs-on: ubuntu-latest
    steps:
      - name: 'Checkout codes'
        uses: actions/checkout@v1
      - name: 'prepare'
        run: pip install -r requirements.txt
      - name: 'run bot'
        run: python spider.py
      - name: 'Get Date'
        run: echo "REPORT_DATE=$(TZ=':Asia/Shanghai' date '+%Y-%m-%d %T')" >> $GITHUB_ENV
      - name: 'Send mail'
        uses: dawidd6/action-send-mail@master
        with:
          server_address: smtp.163.com
          server_port: 465
          username: The ${{ secrets.MAILUSERNAME }}
          password: The ${{ secrets.MAILPASSWORD }}
          subject: Juejin Hot Articles Report (${{env.REPORT_DATE}})
          body: file://hot_articles.html
          to: *******@163.com
          from: GitHub Actions
          content_type: text/html
Copy the code

Then configure the email information in Secret