# python-noaa-data-use-selenium **Repository Path**: xujason/python-noaa-data-use-selenium ## Basic Information - **Project Name**: python-noaa-data-use-selenium - **Description**: 使用Python的selenium爬取noaa上面的指定数据 - **Primary Language**: Python - **License**: Not specified - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 1 - **Created**: 2022-06-30 - **Last Updated**: 2022-06-30 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README ## 数据爬取 ### 变量介绍 ```python # 网络较慢时,设置大一些,中间可以有些停顿等待 lower_network = 3 # type 是对应的检测类型 type = 'MSL' # units 单位(0表示 Feet, 1表示 Meters) units = 0 # epoch 时间(0表示 Present, 1表示 Superseded) epoch = 0 # main_url 是主站点,后面会用到 url 的拼接 main_url = 'https://tidesandcurrents.noaa.gov/datums.html?datum=' + type + '&units=' + str(units) + '&epoch=' + str(epoch) # path 是要本地配置的chromedriver的路径,网络搜索chromedriver # 如果非Windows系统,注意用户权限 path = '/usr/bin/chromedriver' ``` 安装chromedriver,参考:[https://npm.taobao.org/mirrors/chromedriver](https://npm.taobao.org/mirrors/chromedriver) ### 安装环境依赖 ```python # 安装Python和pip,建议3.x版本(使用的3.6版本) # 安装依赖 selenium # 如果非Windows系统,注意用户权限;安装了多个版本的Python,注意正确使用命令行 pip install selenium ``` 安装多版本Python,参考:[https://www.jianshu.com/p/810f3be23431](https://www.jianshu.com/p/810f3be23431) ### 工程运行 ```python # cmd到工程目录下,执行 python noaa.py # 或者 - 根据系统安装配置执行 python3 noaa.py # 如果不报错,会打开chrome浏览器,不要关闭这个弹窗,数据会开始爬取,控制台中可以看到爬取的简要内容 # 会在工程目录下生成MSL.txt文件,每行为一条记录,由id;name;state;value;url组成,使用excel打开,选择分号';'分列 ``` ### 运行结果示例 ```txt # 前15行的数据 1611347;PORT ALLEN, HANAPEPE BAY, KAUAI ISLAND;HI;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1611347&name=PORT%20ALLEN,%20HANAPEPE%20BAY,%20KAUAI%20ISLAND&state=HI 1611400;Nawiliwili;HI;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1611400&name=Nawiliwili&state=HI 1612340;Honolulu;HI;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1612340&name=Honolulu&state=HI 1612366;FORT KAMEHAMEHA, BISHOP POINT, PEARL HBR;HI;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1612366&name=FORT%20KAMEHAMEHA,%20BISHOP%20POINT,%20PEARL%20HBR&state=HI 1612404;FORD ISLAND, FERRY DOCK, PEARL HARBOR;HI;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1612404&name=FORD%20ISLAND,%20FERRY%20DOCK,%20PEARL%20HARBOR&state=HI 1612480;Mokuoloe;HI;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1612480&name=Mokuoloe&state=HI 1612668;HALEIWA, WAIALUA BAY, OAHU IS;HI;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1612668&name=HALEIWA,%20WAIALUA%20BAY,%20OAHU%20IS&state=HI 1612702;LAIEMALOO, OAHU ISLAND;HI;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1612702&name=LAIEMALOO,%20OAHU%20ISLAND&state=HI 1613198;Kaunakakai Harbor;HI;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1613198&name=Kaunakakai%20Harbor&state=HI 1614465;KAUMALAPAU HARBOR, LANAI ISLAND;HI;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1614465&name=KAUMALAPAU%20HARBOR,%20LANAI%20ISLAND&state=HI 1615680;Kahului, Kahului Harbor;HI;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1615680&name=Kahului,%20Kahului%20Harbor&state=HI 1617433;Kawaihae;HI;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1617433&name=Kawaihae&state=HI 1617760;Hilo, Hilo Bay, Kuhio Bay;HI;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1617760&name=Hilo,%20Hilo%20Bay,%20Kuhio%20Bay&state=HI 1619000;Johnston Atoll;United States of America;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1619000&name=Johnston%20Atoll&state=United%20States%20of%20America 1619910;Sand Island, Midway Islands;;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1619910&name=Sand%20Island,%20Midway%20Islands&state= ```