关于安装scrapy-redis分布式常见问题

直接运行其实免Python 安装  使用命令pip install ***


sudo pip install scrapy


sudo pip install scrapy-redis


sudo apt-get install redis-server


sudo vi /etc/redis/redis.conf  配置文件下的bind 127.0.0.1,将其注释掉就行了


ImportError: No module named requests
  sudo pip install requests


 ImportError: No module named bs4
sudo pip install bs4




ImportError: No module named selenium
sudo pip install selenium


ImportError: No module named PIL
sudo pip install image


ImportError: No module named MySQLdb


sudo apt-get install python-mysqldb


OperationalError: (1130, "Host '10.21.20.47' is not allowed to connect to this MySQL server")


mysql>  GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'mypassword' WITH GRANT OPTION;
mysql>  FLUSH   PRIVILEGES;


重启mysql服务


再次尝试可能遇到问题：


"Access denied for user 'root'@'10.21.20.47' (using password: YES)


 执行授权命令
mysql> grant all privileges on *.* to root@'%' identified by '123';
Query OK, 0 rows affected (0.07 sec)
 退出再试


 问题3：
ConnectionError: Error 111 connecting to 192.168.76.128:6379. Connection refused
因为 etc/redis/redis.conf 配置文件下的bind 127.0.0.1,将其注释掉就行了


sudo vi /etc/redis/redis.conf     




在一个主机安装完后，用scp传送文件夹  


例：hadoop@master:~/weibo$  scp -r weibo slave1:~/weibo