preface

Deno’s all-in-one structure makes script development and execution easier than Python’s environment configuration process.

Yesterday saw python everyday automatic background image not fragrant? This article uses a Python script to get bing’s background image. A look at this project makes it a good time to write a deno demo. Hence the article.

Environmental installation

To prefer the command line, see the official website or install Deno in a single line command.

For those of you who prefer to work directly with executables, go here. Remember to configure environment variables.

The code is briefly

  1. Get bing background map information by parsing the DOM of the Bing wallpaper site. (Mainly because the site page is rendered on the server side and does not see the interface that requested the background image list)
import {
  DOMParser,
  Element,
} from "https://deno.land/x/deno_dom/deno-dom-wasm.ts";

async function getList(curPage = 1) :Promise<Array<WallpaperItem>> {
  let wallpaperItems: Array<WallpaperItem> = [];
  const res = await fetch(`${ORIGIN_URL}? p=${curPage}`);
  const { status, statusText } = res;
  if (status === STATUS_CODE.OK) {
    const rawData = await res.text();
    const doc = new DOMParser().parseFromString(rawData, "text/html");

    constbody = doc? .querySelector("body");
    constitems = body? .querySelectorAll(".item");
    if (items) {
      wallpaperItems = Array.from(items).map((item) = > {
        // <Element> for bug: https://github.com/b-fuze/deno-dom/issues/4
        const descriptionEl = (<Element> item).querySelector(
          ".description h3",);const dateEl = (<Element> item).querySelector(
          ".description .calendar .t",);const mark = (<Element> item).querySelector(".mark");
        constpath = mark? .attributes.href;constimgName = path? .substring( path.lastIndexOf("/") + 1,
          path.lastIndexOf("?"));const wallpaperItem: WallpaperItem = {
          name: imgName,
          description: descriptionEl? .textContent,date: dateEl? .textContent,url: `https://cn.bing.com/th?id=OHR.${imgName}_UHD.jpg`};returnwallpaperItem; }); }}else {
    console.log(`${status}: ${statusText}`);
  }

  return wallpaperItems;
}
Copy the code

The parsed data contains the image name, description, time, and image address.

export interface WallpaperItem {
  url: string; name? :string; description? :string; date? :string;
}
Copy the code

The image address uses the bing wallpaper source (there are high-resolution images, which are higher resolution than the bing wallpaper site).

  1. Save the image locally
export async function saveFile(node: WallpaperItem, saveDir = DEFAULT_FILE_SAVE_DIR,) {
  const { name, date, url } = node;
  const res = await fetch(url);
  const { status, statusText } = res;
  if (status === STATUS_CODE.OK) {
    const img = await res.arrayBuffer();
    const ext = url.substring(url.lastIndexOf("."));

    Deno.mkdirSync(saveDir, { recursive: true });
    Deno.writeFileSync(
      `${saveDir}/ [${date}]${name}${ext}`.new Uint8Array(img),
    );
  } else {
    console.log(`${status}: ${statusText}`); }}Copy the code
  1. Batch request image information

In order to request the whole site all picture information list, for example, first need to know how many pages there are. This part is also obtained by parsing the page DOM.

export async function getPageCount() :Promise<number> {
  const res = await fetch(`${ORIGIN_URL}`);
  const { status, statusText } = res;
  if (status === STATUS_CODE.OK) {
    const rawData = await res.text();
    const doc = new DOMParser().parseFromString(rawData, "text/html");

    constbody = doc? .querySelector("body");
    constpager = body? .querySelector(".page span");
    constpagerTextContent = pager? .textContent ||"";
    const [, pageCount = "0"] = pagerTextContent.split("/");

    return parseInt(pageCount);
  } else {
    console.log(`${status}: ${statusText}`);
    return 0; }}Copy the code

After batch request can. Be careful to disguise headers when requesting, adjust the frequency of requests, and use IP pools if you need to make frequent requests. The following uses asyncPool to limit the number of parallel requests.

Too high frequency, easy to request under IP ban for a period of time.

export async function getAll() :Promise<Array<WallpaperItem>> {
  let allWallpaperItems = [];
  const pageCount = await getPageCount();
  if (pageCount > 0) {
    const queue = new Array(pageCount).fill(undefined).map((val, index) = >
      index + 1
    ).map((curPage) = > getList(curPage));

    let i = 0;
    const timeout = (item: Promise<unknown>, array: Array<unknown>) = > {
      const time = Math.random() + 0.5;
      console.log(array.length, ++i);
      return new Promise((resolve) = > setTimeout(() = > resolve(item), time));
    };
    const group = await asyncPool(
      REQUEST_CONCURRENCY_LIMIT,
      queue,
      timeout,
    );

    allWallpaperItems = (<Array<WallpaperItem>> []).concat( ... (<Array<Array<WallpaperItem>>> group),
    );

    setCache(allWallpaperItems);
  } else {
    allWallpaperItems = getCache();
  }

  return allWallpaperItems;
}
Copy the code

After the request is complete, don’t forget to write the list of images to the cache file. After all, historical data don’t change.

function setCache(data: Array<WallpaperItem>) {
  Deno.writeTextFile("./cache.json".JSON.stringify(data, null.2));
}
Copy the code

Setting the background image

Look here

The source code

With reference to bing_wallpaper_get

A key operation

deno run --allow-read --allow-write --allow-net https://deno.land/x/bing_wallpaper_get/main.ts g
Copy the code

Timing task

Windows: look here