“This is the 26th day of my participation in the November Gwen Challenge. See details of the event: The Last Gwen Challenge 2021”.
preface
Use Python to realize data visualization of NetEase cloud music playlist. Without further ado.
Let’s have a good time
The development tools
Python version: 3.6.4
Related modules:
Requests the module
Pandas module
Matplotlib module;
And some modules that come with Python.
Environment set up
Install Python and add it to the environment variables. PIP installs the required related modules.
This time through the acquisition of NetEase cloud Music Chinese playlist data, the Chinese playlist data visualization analysis.
Use the Matplotlib visualization library to make use of the underlying library for visualization.
Web analytics
Playlist index page
Select Chinese hot songs list page.
Get playlist number, name, and author, and playlist details page link.
A total of 1,302 Chinese songs were collected.
Playlist details page
Get playlist details page information, more information.
There are single names, favorites, comments, tags, introductions, total number of songs, number of plays, title of songs included.
Here the length of the song, artist, album information in the web page in the iframe.
If you want to access information, you can use Selenium
To get the data
Playlist index page
from bs4 import BeautifulSoup
import requests
import time
headers = {
'User-Agent': 'the Mozilla / 5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36'
}
for i in range(0.1330.35) :print(i)
time.sleep(2)
url = 'https://music.163.com/discover/playlist/?cat=, Europe and the United States & order = hot&limit = 35 & offset =' + str(i)
response = requests.get(url=url, headers=headers)
html = response.text
soup = BeautifulSoup(html, 'html.parser')
Get the tag that contains the playlist details page URL
ids = soup.select('.dec a')
Get the tag that contains the playlist index page information
lis = soup.select('#m-pl-container li')
print(len(lis))
for j in range(len(lis)):
Get the playlist details page address
url = ids[j]['href']
Get the playlist title
title = ids[j]['title']
Get playlist number
play = lis[j].select('.nb') [0].get_text()
Get the playlist contributor name
user = lis[j].select('p') [1].select('a') [0].get_text()
# output playlist index page information
print(url, title, play, user)
Write the information to a CSV file
with open('playlist.csv'.'a+', encoding='utf-8-sig') as f:
f.write(url + ', ' + title + ', ' + play + ', ' + user + '\n')
Copy the codeThrough the above code we get the playlist index page information
Playlist details page
Part of the code
from bs4 import BeautifulSoup
import pandas as pd
import requests
import time
df = pd.read_csv('playlist.csv', header=None, error_bad_lines=False, names=['url'.'title'.'play'.'user'])
headers = {
'User-Agent': 'the Mozilla / 5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36'
}
for i in df['url']:
time.sleep(2)
url = 'https://music.163.com' + i
response = requests.get(url=url, headers=headers)
html = response.text
soup = BeautifulSoup(html, 'html.parser')
Get the playlist title
title = soup.select('h2') [0].get_text().replace(', '.', ')
# fetch tag
tags = []
tags_message = soup.select('.u-tag i')
for p in tags_message:
tags.append(p.get_text())
Format the tag
if len(tags) > 1:
tag = The '-'.join(tags)
else:
tag = tags[0]
Copy the codeDetails of 1302 Chinese playlists obtained
Data visualization
TOP10 songs
Of the ten songs on the list, except “Mercury”, little F listened to many times.
Playlist contributed UP to TOP10
TOP10 playlists
Top 10 playlists with more than 70 million streams.
TOP10 in playlist collection
Recommended collection
TOP10 comments on playlist
The playlist “Goodbye Warrior: The death of Martial arts novel master Jin Yong” received the most comments.
Playlist collection quantity distribution
Mainly distributed between 0 and 150,000 (ln(150000)=12).
Playlist number distribution
The number of playlists is mainly distributed between 0 and 10 million.
Playlist label diagram
Since the selection is the Chinese song list, so the Chinese word is indispensable
Playlist introduction word cloud map,
Playlist introduces word cloud map, hope you can find a song you like