1 Star 0 Fork 1

wangyang191/proxy_pool爬虫代理IP池

加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
文件
克隆/下载
setting.py 2.71 KB
一键复制 编辑 原始数据 按行查看 历史
liumengqi 提交于 2024-07-26 14:59 +08:00 . 增加齐云代理
# -*- coding: utf-8 -*-
"""
-------------------------------------------------
File Name: setting.py
Description : 配置文件
Author : JHao
date: 2019/2/15
-------------------------------------------------
Change Activity:
2019/2/15:
-------------------------------------------------
"""
BANNER = r"""
****************************************************************
*** ______ ********************* ______ *********** _ ********
*** | ___ \_ ******************** | ___ \ ********* | | ********
*** | |_/ / \__ __ __ _ __ _ | |_/ /___ * ___ | | ********
*** | __/| _// _ \ \ \/ /| | | || __// _ \ / _ \ | | ********
*** | | | | | (_) | > < \ |_| || | | (_) | (_) || |___ ****
*** \_| |_| \___/ /_/\_\ \__ |\_| \___/ \___/ \_____/ ****
**** __ / / *****
************************* /___ / *******************************
************************* ********************************
****************************************************************
"""
VERSION = "2.4.0"
# ############### server config ###############
HOST = "0.0.0.0"
PORT = 5010
# ############### database config ###################
# db connection uri
# example:
# Redis: redis://:password@ip:port/db
# Ssdb: ssdb://:password@ip:port
DB_CONN = 'redis://:pwd@127.0.0.1:6379/0'
# proxy table name
TABLE_NAME = 'use_proxy'
# ###### config the proxy fetch function ######
PROXY_FETCHER = [
"freeProxy01",
"freeProxy02",
"freeProxy03",
"freeProxy04",
"freeProxy05",
"freeProxy06",
"freeProxy07",
"freeProxy08",
"freeProxy09",
"freeProxy10",
"freeProxy11"
# , "freeProxy11", # 如果使用齐云代理就放开
]
# ############# proxy validator #################
# 代理验证目标网站
HTTP_URL = "http://httpbin.org"
HTTPS_URL = "https://www.qq.com"
# 代理验证时超时时间
VERIFY_TIMEOUT = 10
# 近PROXY_CHECK_COUNT次校验中允许的最大失败次数,超过则剔除代理
MAX_FAIL_COUNT = 0
# 近PROXY_CHECK_COUNT次校验中允许的最大失败率,超过则剔除代理
# MAX_FAIL_RATE = 0.1
# proxyCheck时代理数量少于POOL_SIZE_MIN触发抓取
POOL_SIZE_MIN = 20
# ############# proxy attributes #################
# 是否启用代理地域属性
PROXY_REGION = True
# ############# scheduler config #################
# Set the timezone for the scheduler forcely (optional)
# If it is running on a VM, and
# "ValueError: Timezone offset does not match system offset"
# was raised during scheduling.
# Please uncomment the following line and set a timezone for the scheduler.
# Otherwise it will detect the timezone from the system automatically.
TIMEZONE = "Asia/Shanghai"
Loading...
马建仓 AI 助手
尝试更多
代码解读
代码找茬
代码优化
Python
1
https://gitee.com/wangyang191/proxy_pool.git
git@gitee.com:wangyang191/proxy_pool.git
wangyang191
proxy_pool
proxy_pool爬虫代理IP池
master

搜索帮助