# elk-docker

**Repository Path**: atompi/elk-docker

## Basic Information

- **Project Name**: elk-docker
- **Description**: elk stack Dockerfile
- **Primary Language**: Docker
- **License**: MIT
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 2
- **Forks**: 1
- **Created**: 2017-04-22
- **Last Updated**: 2020-12-19

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# ELK stack

ELK stack 是以 Elasticsearch、Logstash、Kibana 三个开源软件为主的数据处理工具链。

在实时数据检索和分析场合，三者通常是配合使用，而且又先后归于Elastic.io公司名下，故有Elastic Stack简称。

---

# ELK stack部署 (elasticsearch 2.4.4、Logstash 2.4.1、Kibana 4.6.4)

分为Linux Server Host部署方式和Docker部署方式。

初始化Linux Server Host
```
修改时区：
> sudo cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
> sudo dpkg-reconfigure -f noninteractive tzdata

Clone本仓库
> cd ~/
> git clone https://git.oschina.net/atompi/elk-docker.git
> cd ~/elk-docker

修改apt源为阿里云：
> sudo cp sources.list /etc/apt/sources.list
> sudo apt-get update
```

## elasticsearch部署 (单节点)
### Linux Server Host部署方式 (Ubuntu 16.04)

安装openjdk-8
```
> sudo apt install -y openjdk-8-jdk
```
修改内核参数
```
> echo "vm.max_map_count=262144" | sudo tee -a /etc/sysctl.conf
> sudo sysctl -w vm.max_map_count=262144
```
下载elasticsearch deb包
```
> cd /usr/local/src
> sudo wget https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/deb/elasticsearch/2.4.4/elasticsearch-2.4.4.deb
```
安装elasticsearch
```
> cd /usr/local/src
> sudo dpkg -i elasticsearch-2.4.4.deb
```
配置elasticsearch
```
echo "network.host: $ELS_HOST" >> /etc/elasticsearch/elasticsearch.yml      # $ELS_HOST为elasticsearch部署的节点host ip 
```
启动elasticsearch
```
sudo service elasticsearch start
```
查看elasticsearch日志
```
tail -f /var/log/elasticsearch/elasticsearch.log
```

## Logstash部署
### Linux Server Host部署方式 (Ubuntu 16.04)

安装redis-3.2.8
```
> cd /usr/local/src
> sudo wget http://download.redis.io/releases/redis-3.2.8.tar.gz /usr/local/src/
> cd /usr/local/src && sudo tar -zxf redis-3.2.8.tar.gz
> cd /usr/local/src/redis-3.2.8
> make && sudo make install
> cd utils && (echo;echo;echo;echo;echo;echo)|sudo ./install_server.sh      # 默认方式安装redis server 也可以去掉 '|' 前面的命令组 (echo;echo;echo;echo;echo;echo) ，部署redis server时手动设置参数
> sudo sed -i 's/127.0.0.1/0.0.0.0/g' /etc/redis/6379.conf       # 通过redis手机shipper日志，所以要配置远程可访问
```
安装Logstash
```
> cd /usr/local/src
> sudo wget https://download.elastic.co/logstash/logstash/packages/debian/logstash-2.4.1_all.deb
> sudo dpkg -i logstash-2.4.1_all.deb
```
配置Logstash indexer (收集nginx access log)
```
> sudo mkdir -p /opt/logstash/patterns
> sudo cp elk/logstash-2.4.1/nginx /opt/logstash/patterns/
> sudo cp elk/logstash-2.4.1/nginx_access_indexer.conf /etc/logstash/conf.d/
```
配置Logstash shipper (收集nginx access log)
```
>sudo cp elk/logstash-2.4.1/nginx_access_shipper.conf /etc/logstash/conf.d/
```
启动Logstash
```
> sudo service logstash start
```

nginx log format (源码位于elk/logstash-2.4.1/nginx_logformat.conf)
```
log_format access_log    '$remote_addr - $remote_user [$time_local] "$request" '
                         '$status $body_bytes_sent "$http_referer" '
                         '"$http_user_agent" $http_x_forwarded_for '
                         '$request_time $upstream_response_time ';

access_log  logs/access.log access_log;

```

## Kibana部署
### Linux Server Host部署方式 (Ubuntu 16.04)

安装nginx、apache2-utils
```
> sudo apt install -y nginx apache2-utils
```
配置nginx、htpasswd验证
```
> sudo cp kibana.conf /etc/nginx/conf.d/
> sudo htpasswd -cb /opt/kibana/htpasswd.users kibana $KIBANA_PWD        # $KIBANA_PWD kibana用户密码
```
安装Kibana
```
> cd /usr/local/src
> sudo wget https://download.elastic.co/kibana/kibana/kibana-4.6.4-linux-x86_64.tar.gz
> sudo tar -zxf kibana-4.6.4-linux-x86_64.tar.gz
> sudo mv kibana-4.6.4-linux-x86_64 /opt/kibana
```
配置Kibana
```
> sudo echo "server.host: \"localhost\"" >> /opt/kibana/conf/kibana.conf
> sudo echo "elasticsearch.url: \"http:\/\/$ELS_HOST:9200\"
```
启动Kibana
```
> sudo mkdir /var/log/kibana
> su
> nohup /opt/kibana/bin/kibana > /var/log/kibana/kibana.log &
```
启动nginx
```
> sudo nginx -t
> sudo nginx -s reload
```
查看Kibala日志
```
> /var/log/kibana/kibana.log
```
浏览器访问
```
修改host，添加条目：$KIBANA_HOST elk.oschina.cc
浏览器访问地址：http://elk.oschina.cc
用户名：kibana
密码：$KIBANA_PWD设置的密码
```