elasticsearch

安装

优化

• 关闭swap

  swapoff -a
  # 注释 /etc/fstab swap
  sysctl -p

• unlimit调整

  sysctl -w vm.max_map_count=262144
  echo 'vm.max_map_count=262144' >> /etc/sysctl.conf

• 使用ssd

重启

curl -X PUT "localhost:9200/_cluster/settings" -H 'Content-Type: application/json' -d'
{
  "persistent": {
    "cluster.routing.allocation.enable": "none"
  }
}
'
curl -X POST "localhost:9200/_flush/synced"
supervisorctl restart elasticsearch
curl -X GET "localhost:9200/_cat/health"
curl -X PUT "localhost:9200/_cluster/settings" -H 'Content-Type: application/json' -d'
{
  "persistent": {
    "cluster.routing.allocation.enable": null
  }
}
'

踩坑

• 滚动重启 重启前

  curl -XPUT http://192.168.67.14:9200/_cluster/settings -d '{
      "transient" : {
          "cluster.routing.allocation.enable" : "none"
      }
  }'

  重启后

  curl -XPUT http://192.168.67.14:9200/_cluster/settings -d '{
      "transient" : {
          "cluster.routing.allocation.enable" : "all"
      }
  }'

• 修改index.number_of_shards 需要设置模块, 已经存在的索引无法修改分片数量

  curl -XPUT "http://elasticsearch-master.efk:9200/_template/local" -H 'Content-Type: application/json' -d'
         {
           "index_patterns": ["local-*"],
           "settings": {
             "index.number_of_replicas": 0,
             "index.number_of_shards": 3
           }
         }'

• 磁盘损坏

  curl -XPOST 'http://192.168.60.32:9200/_cluster/reroute' -d '{
          "commands" : [ {
                "allocate" : {
                    "index" : ".marvel-2018.10.10",
                    "shard" : 0,
                    "node" : "1.7.5-192.168.67.14",
                    "allow_primary" : true
                }
              }
          ]
      }'
  

• unassigned shards

  curl http://127.0.0.1:9200/_cat/shards?v 2>&1 | grep UNASSIGNED

  curl http://127.0.0.1:9200/_nodes/process?pretty

  curl -XPOST 'http://192.168.60.32:9200/_cluster/reroute' -d '{
          "commands" : [ {
                  "allocate" : {
                      "index" : "logstash-xxx-up-2019.01.29",
                      "shard" : 9,
                      "node" : "kzI0751OSJBcx_wmLQ",
                      "allow_primary" : true
                  }
              }
          ]
      }'

• 修改@timestamp类型

  curl -XPUT "http://127.0.0.1:9200/_template/uplog" -H 'Content-Type: application/json' -H 'Host: elasticsearch.morgans' -d'
  {
    "index_patterns": ["uplog-*"],
    "settings": {
      "index.number_of_replicas": 0
    },
    "mappings": {
      "_default_": {
          "properties": {
              "@timestamp": {
                  "type": "date",
                  "doc_values": true
              }
          }
      }
    }
  }'

kibana

安装

优化

踩坑

• 保存时kibana报错 Visualization Editor: blocked by: [FORBIDDEN/12/index read-only 参考 https://github.com/elastic/kibana/issues/13685

  curl -XPUT -H "Content-Type: application/json" http://127.0.0.1:9200/.kibana/_settings -d '{"index.blocks.read_only_allow_delete": null}'

• Timelion每次查询得到的数据不一样 参考 https://github.com/elastic/kibana/issues/23428 和 https://discuss.elastic.co/t/timelion-gives-different-result-when-it-auto-refreshes/155182 修改kibana.yml配置

  elasticsearch.shardTimeout = 30000