python-noaa-data-use-selenium: 使用Python的selenium爬取noaa上面的指定数据

## 数据爬取

### 变量介绍

```python
   # 网络较慢时，设置大一些，中间可以有些停顿等待
   lower_network = 3
   # type 是对应的检测类型
   type = 'MSL'
   # units 单位（0表示 Feet， 1表示 Meters）
   units = 0
   # epoch 时间（0表示 Present， 1表示 Superseded）
   epoch = 0
   # main_url 是主站点，后面会用到 url 的拼接
   main_url = 'https://tidesandcurrents.noaa.gov/datums.html?datum=' + type + '&units=' + str(units) + '&epoch=' + str(epoch)
   # path 是要本地配置的chromedriver的路径，网络搜索chromedriver
   # 如果非Windows系统，注意用户权限
   path = '/usr/bin/chromedriver'
   ```

安装chromedriver，参考：[https://npm.taobao.org/mirrors/chromedriver](https://npm.taobao.org/mirrors/chromedriver)

### 安装环境依赖

```python
   # 安装Python和pip，建议3.x版本（使用的3.6版本）
   # 安装依赖 selenium
   # 如果非Windows系统，注意用户权限；安装了多个版本的Python，注意正确使用命令行
   pip install selenium
   ```

安装多版本Python，参考：[https://www.jianshu.com/p/810f3be23431](https://www.jianshu.com/p/810f3be23431)

### 工程运行

```python
   # cmd到工程目录下，执行
   python noaa.py
   # 或者 - 根据系统安装配置执行
   python3 noaa.py
   # 如果不报错，会打开chrome浏览器，不要关闭这个弹窗，数据会开始爬取，控制台中可以看到爬取的简要内容
   # 会在工程目录下生成MSL.txt文件，每行为一条记录，由id;name;state;value;url组成，使用excel打开，选择分号';'分列
   ```

### 运行结果示例

```txt
   # 前15行的数据
    1611347;PORT ALLEN, HANAPEPE BAY, KAUAI ISLAND;HI;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1611347&name=PORT%20ALLEN,%20HANAPEPE%20BAY,%20KAUAI%20ISLAND&state=HI
    1611400;Nawiliwili;HI;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1611400&name=Nawiliwili&state=HI
    1612340;Honolulu;HI;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1612340&name=Honolulu&state=HI
    1612366;FORT KAMEHAMEHA, BISHOP POINT, PEARL HBR;HI;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1612366&name=FORT%20KAMEHAMEHA,%20BISHOP%20POINT,%20PEARL%20HBR&state=HI
    1612404;FORD ISLAND, FERRY DOCK, PEARL HARBOR;HI;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1612404&name=FORD%20ISLAND,%20FERRY%20DOCK,%20PEARL%20HARBOR&state=HI
    1612480;Mokuoloe;HI;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1612480&name=Mokuoloe&state=HI
    1612668;HALEIWA, WAIALUA BAY, OAHU IS;HI;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1612668&name=HALEIWA,%20WAIALUA%20BAY,%20OAHU%20IS&state=HI
    1612702;LAIEMALOO, OAHU ISLAND;HI;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1612702&name=LAIEMALOO,%20OAHU%20ISLAND&state=HI
    1613198;Kaunakakai Harbor;HI;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1613198&name=Kaunakakai%20Harbor&state=HI
    1614465;KAUMALAPAU HARBOR, LANAI ISLAND;HI;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1614465&name=KAUMALAPAU%20HARBOR,%20LANAI%20ISLAND&state=HI
    1615680;Kahului, Kahului Harbor;HI;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1615680&name=Kahului,%20Kahului%20Harbor&state=HI
    1617433;Kawaihae;HI;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1617433&name=Kawaihae&state=HI
    1617760;Hilo, Hilo Bay, Kuhio Bay;HI;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1617760&name=Hilo,%20Hilo%20Bay,%20Kuhio%20Bay&state=HI
    1619000;Johnston Atoll;United States of America;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1619000&name=Johnston%20Atoll&state=United%20States%20of%20America
    1619910;Sand Island, Midway Islands;;0.00;https://tidesandcurrents.noaa.gov/datums.html?datum=MSL&units=0&epoch=0&id=1619910&name=Sand%20Island,%20Midway%20Islands&state=

```

Jason/python-noaa-data-use-selenium

简介

发行版

贡献者

近期动态

Jason/python-noaa-data-use-selenium .gitee-modal { width: 500px !important; }

简介

发行版

贡献者

近期动态

搜索帮助

Jason/python-noaa-data-use-selenium