Issue in 2014

scrapy/scrapyd/issues/43

Pull request in 2019

scrapy/scrapyd/pull/326

The trial

  1. Installation:pip install -U git+https://github.com/my8100/scrapyd.git@add_basic_auth
  2. Updating the Configuration Filescrapyd.confFor details about other configuration items, seeThe official documentation
[scrapyd]
username = yourusername
password = yourpassword
Copy the code
  1. Activation:scrapyd
In [1]: import requests

In [2]: requests.get('http://127.0.0.1:6800/').status_code
Out[2]: 401

In [3]: requests.get('http://127.0.0.1:6800/', auth=('admin'.'admin')).status_code
Out[3]: 401

In [4]: requests.get('http://127.0.0.1:6800/', auth=('yourusername'.'yourpassword')).status_code
Out[4]: 200
Copy the code
  1. Since the latest GitHub submission for Scrapyd has reconfigured the Jobs page, if you are using ScrapydWeb to manage Scrapyd, you need to update your ScrapydWeb synchronously:

pip install -U git+https://github.com/my8100/scrapydweb.git

GitHub

my8100/scrapyd