Modeling analysis

  • The discussion is based on the case of the visitor system

Purpose of modeling analysis

  • Analyze which levels, which tables, which fields are needed for the whole warehouse

  • ODS layer: The source data layer

    • Docking source data, keeping the same granularity as source data

  • DWD

    • Task:

    • 1. Cleaning.

      • Incomplete data
      • Expired or invalid data

    • 2. The conversion

      • Create_time – > (date) (month) (year)
      • Or timestamp
    • 3. Dimensions can be appropriately degenerated to reduce table associations

  • DWM: Middle layer

    • To further aggregate the detail layer, for example, the records of the day can be merged first, and the month can be merged more easily later

  • DWS: business layer

    • Fine aggregate statistics,

  • APP: application layer

    • The detailed statistical results were analyzed again
    • Can be omitted

  • Dim: Dimension layer

    • Place dimension table data