Download address: download.csdn.net/download/qq…

Project introduction

Design and Implementation of Python Scrapy Django WeUI News Collection and subscription System based on Web crawler

System specifications

With the rapid development of the Internet, the Internet has greatly enhanced the generation and dissemination of information on the network every day

A lot of content is generated, and how to efficiently discover and gather the information you need from this clutter becomes more and more important

Want to. The news content in the network is the same, news is distributed on different websites, and there are repeated content, we tend to only

Concerned about a part of the news, the news page in the network is often filled with a lot of information unrelated to the news, the impact

Given our reading efficiency and reading experience, how can we get the news we care about more conveniently, timely and efficiently

Tong can help us do that. This system uses the web crawler we can do the news website on the network for timing

To analyze and collect, and then the collected data for deduplication, classification and other operations into the database, and finally to provide personalized

Use Python with scrapy and other frameworks to write crawlers, use specific content extraction algorithm to extract target data, and finally

Use Django and WeUI to provide news subscription background and news content display page, and use wechat to push information to users. with

Users can subscribe to the specified keyword through this system. When the crawler system crawls to the content containing the specified keyword, it will push the news

Send to the user.