Jingbibao is a Chrome plug-in that automatically monitors price changes of purchased goods to apply for jingdong price protection.

I had an idea very early on, whether I could somehow record the data from the process of monitoring the prices of the goods that were purchased and create a price chart.

After some recent research, baidu Cloud’s “time series database” was finally implemented last weekend to create a mutual database of JD.com’s price history.

Cut the crap and look at the effect:

The price of Beijing USB3.0 transparent splitter was 39.9 yuan on September 26, and the standard price rose to 49 yuan on September 27, but the PRICE of PLUS was 39.9 yuan. Today, the price of PLUS also rose to 47 yuan.

Implementing this business with a sequential database has several obvious advantages:

  • Write quotas are extremely large and are usually limited by time periods
  • Storage costs are extremely low and storage is almost free
  • Queries are very convenient, and there is no limit to the number of requests

Due to the Node SDK provided by the product “Timing Database” of Baidu Cloud, the actual access process was very smooth and only took about one hour in total.

In fact, a total of two access to write data point and query data interface, simply paste two pieces of code good:

Record the data

async addPrice(priceInfo: PriceInfo): Promise<any> {
  function getTimestamp() {
    return Math.round(DateTime.local().startOf('hour').valueOf() / 1000)
  }

  function buildDataPoint(field: string, sku: string, price: number) {
    return {
      "metric": "jd"."field": field,
      "tags": {
        "sku": sku
      },
      "timestamp": getTimestamp(),
      "value": Number(price)
    }
  }
  let datapoints = [];
  datapoints.push(buildDataPoint('price', priceInfo.sku, priceInfo.price))
  if (priceInfo.plus_price) {
    datapoints.push(buildDataPoint('plus_price', priceInfo.sku, priceInfo.plus_price))
  }
  if (priceInfo.pingou_price) {
    datapoints.push(buildDataPoint('pingou_price', priceInfo.sku, priceInfo.pingou_price))
  }
  let result = null
  try {
    result = await this.TSDBClient.writeDatapoints(datapoints)
  } catch (error) {
    console.error(error);
  }
  return result
}
Copy the code

Query data

async getChartData(sku:string) {
  var queryList = [
    {
      "metric": "jd"."fields": ["price"."plus_price"."pingou_price"]."filters": {
        "start": "15 days ago"."tags": {
          "sku": [sku]
        }
      }
    }
  ];
  let result = null
  let chartData: any[] = []
  try {
    result = await this.TSDBClient.getDatapoints(queryList)
  } catch (error) {
    console.error(error);
  }

  if (result.body && result.body.results) {
    chartData = _.flatten(_.map(result.body.results[0].groups[0].values, (datapoint: number[]) => {
      let points = []
      if (datapoint[1] && datapoint[1] > 0) {
        points.push({
          "timestamp": datapoint[0],
          "key": "Standard price"."value": datapoint[1]
        })
      }
      if (datapoint[2] && datapoint[2] > 0) {
        points.push({
          "timestamp": datapoint[0],
          "key": "Plus price"."value": datapoint[2]
        })
      }
      if (datapoint[3] && datapoint[3] > 0) {
        points.push({
          "timestamp": datapoint[0],
          "key": "Purchase price"."value": datapoint[3]
        })
      }
      return points
    }))
  }
  return chartData;
}
Copy the code

The feature has been online for two and a half days and has so far collected 200,000 price data points, covering more than 20,000 SKUs, and is growing at a very rapid rate.

By the way, the front-end chart is using Alipay’s open source G2 chart library, which is quite easy to use.

Ps. The main reason for choosing Baidu Cloud’s “timing database” is that its price is very cheap. The plan of writing 15 million data points every month only costs 300 yuan per year. In comparison, Ali Cloud’s timing database can be said to be exorbitant — beggar version costs 408 yuan per month.