preface
Hello, everyone. Today, I will give you a comprehensive practical exercise of Python — taking hotpot, a popular dish, as an example, how to use Python to obtain the number of hotpot restaurants in different cities across the country and visually display the data? Next I will take you through this operation, ** based on a certain degree of map data, using Python technical knowledge to obtain data and visualization. ** presents a more intuitive way to browse the distribution of hot pot restaurants in different provinces and cities across the country. At the same time, AT the end of the paper, I will sort out the data visualization required for you to share with you.
1. Web page analysis
First, take a look at the data source. You can request the situation of hot pot restaurants in the whole country by following the operation below in a certain degree map (it is not shown in the picture below, but you can see the data through the Network).
In network, find the following packet
When opened, you can see the JSON data
##2. Get data
Once you’ve analyzed the web page, you can then use Python techniques to retrieve the data and save it to Excel.
Import related libraries
import json
import requests
import openpyxl
Copy the codeThe request data
Let’s start coding the request data (remember to include headers when requesting)
### headers = {" user-agent ": "Mozilla/5.0 (Windows NT 10.0; Win64; X64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36", 'Referer' : 'https://map.baidu.com/@12949550.923158279, 3712445.9716704674, 6.28 z', "Cookie", ";" Your cookie",} ## request link URL = "Https://map.baidu.com/?newmap=1&reqflag=pcmap&biz=1&from=webmap&da_par=direct&pcevaname=pc4.1&qt=s&da_src=searchBox.but E7 ton&wd = % % % 81% AB E9 BA E5% % % 94% 85% 97 & c = 1 & SRC = 0 & wd2 = & pn = 0 & sug = 0 & l = 6 & b = (10637065.476146251, 2368134.592189369; 12772445.910805061, 5056757.351151566) & the from = webmap&biz _forward = {% 22 and scaler %, % 22 styles % 22: % 22 pl % 22} & sug_forward = & auth = N TSwAZUMzIaTTdWD4WAv0731cWF3MQEauxLxREHzERRtykiOxAXXw1GgvPUDZYOYIZuVt1cv3uVtGccZcuVtPWv3GuztQZ3wWvUvhgMZSguxzBEHLNRTVtcEW e1GD8zv7ucvY1SGpuxVthgW1aDeuxtf0wd0vyMySFIAFM7ueh33uTtAffbDF&seckey=c6d9c7e05d7e627c56ed46fab5d7c5c792064779599d5e12b955 a6f18a1204375d1588206c94d22e4bdd1ade0ad06e78c21917e24c6223b96bc51b75ca38651a1b203a0609f126163c5e82fd0549a068e53730342483 7ab798acfc9088e5d76a66451c20ebd9599b41c9b4f1371850d20fa442ad464712f54c912422f4fa20b3052f8bb810f30d41c7c0e55af68f9d9d9735 37f03d0aa0a1d1617d78cae29b49c64c2d2dc3f44cf0f8799234b124a7a2dec18bfa011e097e31a508eae37b8603f97df8f935f04b3652f190eac52d 04816f302a582c53971e515ff2e0e2b4cc30446e0bee48d51c4be8b6fe4185589ed9&device_ratio=1&tn=B_NORMAL_MAP&nn=0&u_loc=12677548, Response = requests. Get (url,headers=headers).json()Copy the codeThe cookie can be copied in the browser network.
Provice is the name of the province, num is the number of hotpot restaurants. Each province has a list of cities (name is the name of the city, name is the name of the city, number of hotpot restaurants). Num is the number of hotpot restaurants in cities)
response = response['more_city']
for i in response:
city = i['city']
print(i['province'])
print(i['num'])
for j in city:
print(j['name'])
print(j['num'])
Copy the code
Save to excel
Provinces and cities are saved in two different Excel files
outwb_p = openpyxl.Workbook() outws_p = outwb_p.create_sheet(index=0) outws_p.cell(row=1, column=1, Value =" province ") outws_p.cell(row=1, column=2, Value =" number ") outwb_c = openPyxl.workbook () outws_c = outwb_c.create_sheet(index=0) outws_c.cell(row=1, column=1, Value =" city ") outws_c.cell(row=1, column=2, Value = "number") # # # # # # # # # # # # # # # # # # # # # write data in a loop # # # # # # # # # # # # # # # # # # # # # to save the national provinces pot number - Li Yunchen outwb_p. "save (" hot pot number - Li Yunchen national provinces. XLS") # ### save national hot pot number - outwb_c.save(" National hot pot number - li Yunchen.xls ") # saveCopy the code
3. Data visualization
1. Distribution of hot pot restaurants nationwide
XLS 'data = pd. Read_excel (datafile) attr = data[' datafile '].tolist() value = data[' datafile '].tolist() Name = [] for I in attr: if "province" in I: name.append(i.place (" province ","")) else: name.append(i) from pyecharts import options as opts from pyecharts.charts import Map from pyecharts.faker import Faker C = (Map().add("数 ", [list(z) for z in zip(name, value)], "China ").set_global_opts(title_opts= opts.titleopts (title_opts= opts.titleopts (title_opts= opts.titleopts)).render(" China "))Copy the code
Or you could draw it like this
XLS 'df = pd. Read_excel (datafile) province_distribution = df[[' province_distribution ', ].value.tolist () geo = geo () geo. Set_series_opts (is_show=False) Geo. Add_schema (mapType =" China ") geo. Set_global_opts (visualMap_opts = opts.visualMapopts (max_=110000)) # Geo. Add (province_distribution, type_=ChartType.EFFECT_SCATTER) geo. Render (" geo.Copy the code
2. Distribution of hot pot restaurants in Sichuan
In order to draw the distribution map of cities, Sichuan Province was selected as an example (if all the cities in China were to be drawn, the resulting map would be dense and not beautiful).
XLS 'data = pd. Read_excel (datafile) city = data[' datafile '].tolist() values2 = data[' datafile '].tolist() ### sichuan name = [] value = [] flag = 0 for I in range(0,len(city)): if city[I] ==" miyangyang ": flag = 1 if flag: Append (city[I]) value.append(int(values2[I])) if city[I] ==" ": Name.append(city[I]) value.append(int(values2[I])) break c = (Map().add(" 中 华 hotpot 中 的 数 据 ", [list(z) for z in zip(name, Value), "sichuan ").set_global_opts(title_opts= opts.titleopts (title_opts= opts.titleopts (title=" sichuan hotpot number distribution "), Visualmap_opts = opts.visualMapopts ()).render(" sichuan hotpan restaurant number distribution.html "))Copy the code
Write at the end (with practical study materials)
The content of this article is intended to do a simple example analysis of visual actual combat operation, without too much in-depth discussion. The original intention of writing this article is to introduce others to the world. To truly understand the world, there is no doubt that deeper learning is required.Here I have compiled some reference learning materials and videos for Python visualization practice (here is only a simple demonstration), so that you can learn and play better. If you need a friend, you can send me a private message. Keywords: Visualization I will share with you:
Finally, if you have any good visualization practice, you can leave a comment below. If you have any help, you may as well give a thumbs up and favorites. Thank you!