Huawei mobile search AspiegelBot crawler was also found by the webmaster recently after toutiao search violently crawled website information, just like Webmaster Toutiao.
We learned that this AspiegelBot crawler mainly simulates Android to catch and crawl, and the UA identifier does not mark the crawler’s company information, so it obviously does not want to be discovered. Later, the webmaster dug deeply and found that this crawler belongs to AspiegelBot, a company affiliated to Huawei Search. And AspiegelBot’s crawler ultra-high frequency crawler even comparable to the violent crawler last time headlines, many were crawled by some small websites many directly paralyzed, began to shield Huawei search AspiegelBot crawler access.
Huawei mobile search AspiegelBot crawler high frequency crawling with the webmaster headlines about, is also a large number of crawling site information, general search engines only began to build a large number of index library will play so, with just online when the headlines search a pattern, it seems that Huawei search is ready to enter the arena!
Source: Lu Songsong blog, welcome to share
Source: https://lusongsong.com/blog/post/13191.html