elasticsearch

安装

优化

• 关闭swap

  1swapoff -a
  2# 注释 /etc/fstab swap
  3sysctl -p

• unlimit调整

  1sysctl -w vm.max_map_count=262144
  2echo 'vm.max_map_count=262144' >> /etc/sysctl.conf

• 使用ssd

重启

 1curl -X PUT "localhost:9200/_cluster/settings" -H 'Content-Type: application/json' -d'
 2{
 3  "persistent": {
 4    "cluster.routing.allocation.enable": "none"
 5  }
 6}
 7'
 8curl -X POST "localhost:9200/_flush/synced"
 9supervisorctl restart elasticsearch
10curl -X GET "localhost:9200/_cat/health"
11curl -X PUT "localhost:9200/_cluster/settings" -H 'Content-Type: application/json' -d'
12{
13  "persistent": {
14    "cluster.routing.allocation.enable": null
15  }
16}
17'

踩坑

• 滚动重启 重启前

  1curl -XPUT http://192.168.67.14:9200/_cluster/settings -d '{
  2    "transient" : {
  3        "cluster.routing.allocation.enable" : "none"
  4    }
  5}'

  重启后

  1curl -XPUT http://192.168.67.14:9200/_cluster/settings -d '{
  2    "transient" : {
  3        "cluster.routing.allocation.enable" : "all"
  4    }
  5}'

• 修改index.number_of_shards 需要设置模块, 已经存在的索引无法修改分片数量

  1curl -XPUT "http://elasticsearch-master.efk:9200/_template/local" -H 'Content-Type: application/json' -d'
  2       {
  3         "index_patterns": ["local-*"],
  4         "settings": {
  5           "index.number_of_replicas": 0,
  6           "index.number_of_shards": 3
  7         }
  8       }'

• 磁盘损坏

   1curl -XPOST 'http://192.168.60.32:9200/_cluster/reroute' -d '{
   2        "commands" : [ {
   3              "allocate" : {
   4                  "index" : ".marvel-2018.10.10",
   5                  "shard" : 0,
   6                  "node" : "1.7.5-192.168.67.14",
   7                  "allow_primary" : true
   8              }
   9            }
  10        ]
  11    }'
  

• unassigned shards

  1curl http://127.0.0.1:9200/_cat/shards?v 2>&1 | grep UNASSIGNED

  1curl http://127.0.0.1:9200/_nodes/process?pretty

   1curl -XPOST 'http://192.168.60.32:9200/_cluster/reroute' -d '{
   2        "commands" : [ {
   3                "allocate" : {
   4                    "index" : "logstash-xxx-up-2019.01.29",
   5                    "shard" : 9,
   6                    "node" : "kzI0751OSJBcx_wmLQ",
   7                    "allow_primary" : true
   8                }
   9            }
  10        ]
  11    }'

• 修改@timestamp类型

   1curl -XPUT "http://127.0.0.1:9200/_template/uplog" -H 'Content-Type: application/json' -H 'Host: elasticsearch.morgans' -d'
   2{
   3  "index_patterns": ["uplog-*"],
   4  "settings": {
   5    "index.number_of_replicas": 0
   6  },
   7  "mappings": {
   8    "_default_": {
   9        "properties": {
  10            "@timestamp": {
  11                "type": "date",
  12                "doc_values": true
  13            }
  14        }
  15    }
  16  }
  17}'

kibana

安装

优化

踩坑

• 保存时kibana报错 Visualization Editor: blocked by: [FORBIDDEN/12/index read-only 参考 https://github.com/elastic/kibana/issues/13685

  1curl -XPUT -H "Content-Type: application/json" http://127.0.0.1:9200/.kibana/_settings -d '{"index.blocks.read_only_allow_delete": null}'

• Timelion每次查询得到的数据不一样 参考 https://github.com/elastic/kibana/issues/23428 和 https://discuss.elastic.co/t/timelion-gives-different-result-when-it-auto-refreshes/155182 修改kibana.yml配置

  1elasticsearch.shardTimeout = 30000