Elasticsearch搭建
2024-06-29 22:39:25
elasticsearch搭建，以及数据备份
- 一、环境
- 二、单机部署
- 1、解压安装包，创建elasticsearch用户，赋权
- 2、修改配置文件
- 3、启动
- 报错1 没有jdk环境
- 报错2 都是因未对操作系统做优化导致
- 三、集群部署
- 1、安装es（同上，配置文件加入集群配置）
- 2、启动，验证
- 3、开启用户认证功能
- 3.1 生成证书
- 3.2 赋权，传到集群内的其他服务器相同目录
- 3.3 配置文件中开启xpack
- 3.4 重启es，添加密码
- 3.5 修改密码
- 3.5.1 已知现在密码修改
- 3.5.2 忘记密码
- 四、ES升级
- 1 备份旧版本
- 2 解压新版本，替换旧版本
- 3 修改新版本的配置文件（拷贝旧版本的配置）
- 4 修改新版本的属主
- 5 启动
- 6 测试
- 五、es备份和恢复
- 1、 基于快照的方式
- 1.1 创建镜像仓库和镜像（所有节点）
- 1.1.1 新建目录作为镜像仓库并修改权限为es用户
- 1.1.2 修改所有es的配置文件，添加仓库地址，重启es
- 1.1.3 创建镜像仓库
- 1.1.4 创建索引
- 1.1.5 创建镜像快照
- 1.1.6 删除索引
- 1.1.7 恢复单个索引
- 1.1.8 异机恢复索引
- 2、基于esdump的方式
一、环境
| 主机名 | IP地址 | 操作系统 | es版本 |
|---|---|---|---|
| es1 | 192.168.10.180 | Centos 7 | elasticsearch7.8.0 |
| es2 | 192.168.10.181 | Centos 7 | elasticsearch7.8.0 |
二、单机部署
1、解压安装包，创建elasticsearch用户，赋权
[root@es1 ~]# ls
elasticsearch-7.8.0-linux-x86_64.tar.gz
[root@es1 ~]# ls
elasticsearch-7.8.0-linux-x86_64.tar.gz
[root@es1 ~]# tar xf elasticsearch-7.8.0-linux-x86_64.tar.gz
[root@es1 ~]# useradd es && passwd es
Changing password for user es.
New password:
BAD PASSWORD: The password is shorter than 8 characters
Retype new password:
passwd: all authentication tokens updated successfully.
[root@es1 ~]# mv elasticsearch-7.8.0 elasticsearch
[root@es1 ~]# chown -R es:es elasticsearch
[root@es1 ~]#
2、修改配置文件
[root@es1 ~]# mv elasticsearch /data/
[root@es1 ~]# cd /data/
[root@es1 data]# ls
elasticsearch
[root@es1 data]# cd /data/elasticsearch/config/
[root@es1 config]# vim elasticsearch.yml [root@es1 config]# cat elasticsearch.yml
node.name: node-1 ##节点名称
path.data: /data/elasticsearch/data ##数据存放路径
path.logs: /data/elasticsearch/logs ##日志存放路径
bootstrap.memory_lock: true ##避免es使用swap交换分区
indices.requests.cache.size: 5% ##缓存配置
indices.queries.cache.size: 10% ##缓存配置
network.host: 192.168.10.180 ##本机IP
http.port: 9200 ##默认端口
cluster.initial_master_nodes: ["node-1"] ##设置符合主节点条件的节点的主机名或 IP 地址来引导启动集群
http.cors.enabled: true ##跨域
http.cors.allow-origin: "*" ##跨域
3、启动
报错1 没有jdk环境
[root@es1 config]# su es
[es@es1 config]$ cd ../bin/
[es@es1 bin]$ ./elasticsearch -d
could not find java in JAVA_HOME at /usr/local/java/bin/java
解决方法：
方法1：配置文件中添加jdk判断（将JAVA_HOME改为ES_JAVA_HOME）
[es@es1 bin]$ grep "JAVA_HOME" elasticsearch-env
ES_JAVA_HOME="/data/elasticsearch/jdk/"
if [ ! -z "$ES_JAVA_HOME" ]; thenJAVA="$ES_JAVA_HOME/bin/java"JAVA_TYPE="ES_JAVA_HOME"
方法2：在环境变量中添加es自带的jdk
[root@es2 ~]# grep "JAVA_HOME" /etc/profile
export JAVA_HOME=/data/elasticsearch/jdk/
export PATH=$JAVA_HOME/bin:$PATH
报错2 都是因未对操作系统做优化导致
[es@es1 bin]$ ./elasticsearch -d
[es@es1 bin]$ ERROR: [3] bootstrap checks failed
[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]
[2]: memory locking requested for elasticsearch process but memory is not locked
[3]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
ERROR: Elasticsearch did not exit normally - check the logs at /data/elasticsearch/logs/elasticsearch.log
解决方法：优化（只给结果，想了解自行百度）
[root@es1 config]# grep -v "#" /etc/security/limits.conf* hard memlock unlimited
* soft memlock unlimited
* hard nofile 1024000
* soft nofile 1024000
* hard nproc 1024000
* soft nproc 1024000
[root@es1 config]# grep -v "#" /etc/systemd/system.conf[Manager]
DefaultLimitNOFILE=65536
DefaultLimitNPROC=32000
DefaultLimitMEMLOCK=infinity
[root@es1 config]# grep -v "#" /etc/sysctl.conf
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_tw_reuse = 1
net.core.somaxconn = 65535
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_fin_timeout = 5
fs.inotify.max_user_watches = 1048576
fs.inotify.max_user_instances = 256
net.ipv4.tcp_keepalive_time = 30
net.ipv4.ip_local_port_range = 1024 65535
net.ipv4.tcp_max_syn_backlog = 262144
net.ipv4.tcp_max_tw_buckets = 256000
vm.max_map_count = 262144
net.core.netdev_max_backlog = 262144
net.ipv4.tcp_max_orphans = 262144
net.ipv4.tcp_synack_retries = 1
net.ipv4.tcp_syn_retries = 1##加载配置（ulimit修改完需要断开连接，重开终端）
[root@es1 config]# /bin/systemctl daemon-reload
[root@es1 config]# /sbin/sysctl -p
启动成功
[root@es1 ~]# su es
[es@es1 root]$ cd /data/elasticsearch/bin/
[es@es1 bin]$ ./elasticsearch -d
[es@es1 bin]$ ps -ef | grep elas
es 9544 1 99 16:27 pts/0 00:00:24 /data/elasticsearch/jdk//bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -XX:+ShowCodeDetailsInExceptionMessages -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dio.netty.allocator.numDirectArenas=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.locale.providers=SPI,COMPAT -Xms1g -Xmx1g -XX:+UseG1GC -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -Djava.io.tmpdir=/tmp/elasticsearch-12986234859914472880 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=logs/hs_err_pid%p.log -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m -XX:MaxDirectMemorySize=536870912 -Des.path.home=/data/elasticsearch -Des.path.conf=/data/elasticsearch/config -Des.distribution.flavor=default -Des.distribution.type=tar -Des.bundled_jdk=true -cp /data/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch -d
es 9655 9544 0 16:27 pts/0 00:00:00 /data/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller
es 10179 9050 0 16:27 pts/0 00:00:00 grep --color=auto elas
[es@es1 bin]$ 三、集群部署
1、安装es（同上，配置文件加入集群配置）
[root@es1 ~]# cat /data/elasticsearch/config/elasticsearch.yml
cluster.name: es-cluster ##集群名称，所有集群下应用名称需一致，若名称一致则会自动加入集群
node.name: node-1
path.data: /data/elasticsearch/data
path.logs: /data/elasticsearch/logs
bootstrap.memory_lock: true
indices.requests.cache.size: 5%
indices.queries.cache.size: 10%
network.host: 192.168.10.180
http.port: 9200
transport.tcp.port: 9300
discovery.zen.ping.unicast.hosts: ["192.168.10.180:9300", "192.168.10.181:9300"] ##集群内部监听的tcp端口号，默认9300
cluster.initial_master_nodes: ["node-1","node-2"]
http.cors.enabled: true
http.cors.allow-origin: "*"
[root@es2 config]# cat /data/elasticsearch/config/elasticsearch.yml
cluster.name: es-cluster
node.name: node-2
path.data: /data/elasticsearch/data
path.logs: /data/elasticsearch/logs
network.host: 192.168.10.181
http.port: 9200
transport.tcp.port: 9300
discovery.zen.ping.unicast.hosts: ["192.168.10.180:9300", "192.168.10.181:9300"]
cluster.initial_master_nodes: ["node-1", "node-2"]
bootstrap.memory_lock: true
indices.requests.cache.size: 5%
indices.queries.cache.size: 10%
http.cors.enabled: true
http.cors.allow-origin: "*"
[root@es2 config]# 2、启动，验证
[es@es1 bin]$ ps -ef | grep elas
es 16253 1 10 16:50 pts/0 00:00:42 /data/elasticsearch/jdk//bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -XX:+ShowCodeDetailsInExceptionMessages -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dio.netty.allocator.numDirectArenas=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.locale.providers=SPI,COMPAT -Xms1g -Xmx1g -XX:+UseG1GC -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -Djava.io.tmpdir=/tmp/elasticsearch-8972888888703564096 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=logs/hs_err_pid%p.log -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m -XX:MaxDirectMemorySize=536870912 -Des.path.home=/data/elasticsearch -Des.path.conf=/data/elasticsearch/config -Des.distribution.flavor=default -Des.distribution.type=tar -Des.bundled_jdk=true -cp /data/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch -d
es 16353 16253 0 16:50 pts/0 00:00:00 /data/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller
es 31076 12154 0 16:56 pts/0 00:00:00 grep --color=auto elas
[es@es1 bin]$ kill -9 16253
[es@es1 bin]$ ./elasticsearch -d
[es@es1 bin]$ curl http://192.168.10.180:9200/_cat/nodes?v
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
192.168.10.180 36 98 7 0.31 0.15 0.14 dilmrt * node-1
[es@es1 bin]$ ^C
[es@es1 bin]$ curl http://192.168.10.180:9200/_cat/nodes?v
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
192.168.10.180 38 98 3 0.19 0.15 0.14 dilmrt * node-1
192.168.10.181 36 78 4 0.29 0.10 0.07 dilmrt - node-2
3、开启用户认证功能
3.1 生成证书
[es@es1 bin]$ ./elasticsearch-certutil cert -out config/elastic-certificates.p12 -pass ""
This tool assists you in the generation of X.509 certificates and certificate
signing requests for use with SSL/TLS in the Elastic stack.The 'cert' mode generates X.509 certificate and private keys.* By default, this generates a single certificate and key for useon a single instance.* The '-multiple' option will prompt you to enter details for multipleinstances and will generate a certificate and key for each one* The '-in' option allows for the certificate generation to be automated by describingthe details of each instance in a YAML file* An instance is any piece of the Elastic Stack that requires an SSL certificate.Depending on your configuration, Elasticsearch, Logstash, Kibana, and Beatsmay all require a certificate and private key.* The minimum required value for each instance is a name. This can simply be thehostname, which will be used as the Common Name of the certificate. A fulldistinguished name may also be used.* A filename value may be required for each instance. This is necessary when thename would result in an invalid file or directory name. The name provided hereis used as the directory name (within the zip) and the prefix for the key andcertificate files. The filename is required if you are prompted and the nameis not displayed in the prompt.* IP addresses and DNS names are optional. Multiple values can be specified as acomma separated string. If no IP addresses or DNS names are provided, you maydisable hostname verification in your SSL configuration.* All certificates generated by this tool will be signed by a certificate authority (CA).* The tool can automatically generate a new CA for you, or you can provide your own with the-ca or -ca-cert command line options.By default the 'cert' mode produces a single PKCS#12 output file which holds:* The instance certificate* The private key for the instance certificate* The CA certificateIf you specify any of the following options:* -pem (PEM formatted output)* -keep-ca-key (retain generated CA key)* -multiple (generate multiple certificates)* -in (generate certificates from an input file)
then the output will be be a zip file containing individual certificate/key filesCertificates written to /data/elasticsearch/config/elastic-certificates.p12This file should be properly secured as it contains the private key for
your instance.This file is a self contained file and can be copied and used 'as is'
For each Elastic product that you wish to configure, you should copy
this '.p12' file to the relevant configuration directory
and then follow the SSL configuration instructions in the product guide.
3.2 赋权，传到集群内的其他服务器相同目录
[es@es1 bin]$ cd ../config/
[es@es1 config]$ chmod 755 elastic-certificates.p12
[es@es1 config]$ scp elastic-certificates.p12 192.168.10.181:/data/elasticsearch/config/
3.3 配置文件中开启xpack
[es@es1 config]$ cat /data/elasticsearch/config/elasticsearch.yml
cluster.name: es-cluster
node.name: node-1
path.data: /data/elasticsearch/data
path.logs: /data/elasticsearch/logs
bootstrap.memory_lock: true
indices.requests.cache.size: 5%
indices.queries.cache.size: 10%
network.host: 192.168.10.180
http.port: 9200
transport.tcp.port: 9300
discovery.zen.ping.unicast.hosts: ["192.168.10.180:9300","192.168.10.181:9300"]
cluster.initial_master_nodes: ["node-1","node-2"]
http.cors.enabled: true
http.cors.allow-origin: "*"xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12
3.4 重启es，添加密码
[es@es1 config]$ ps -ef | grep elas
es 4477 12154 0 17:13 pts/0 00:00:00 grep --color=auto elas
es 31755 1 4 16:57 pts/0 00:00:49 /data/elasticsearch/jdk//bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -XX:+ShowCodeDetailsInExceptionMessages -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dio.netty.allocator.numDirectArenas=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.locale.providers=SPI,COMPAT -Xms1g -Xmx1g -XX:+UseG1GC -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -Djava.io.tmpdir=/tmp/elasticsearch-12887036096412054470 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=logs/hs_err_pid%p.log -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m -XX:MaxDirectMemorySize=536870912 -Des.path.home=/data/elasticsearch -Des.path.conf=/data/elasticsearch/config -Des.distribution.flavor=default -Des.distribution.type=tar -Des.bundled_jdk=true -cp /data/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch -d
es 31854 31755 0 16:57 pts/0 00:00:00 /data/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller
[es@es1 config]$ kill -9 31755
[es@es1 config]$ cd ../bin/
[es@es1 bin]$ ./elasticsearch -d
创建密码（最少6位）
[es@es1 bin]$ ./elasticsearch-setup-passwords interactive
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,kibana_system,logstash_system,beats_system,remote_monitoring_user.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]yEnter password for [elastic]:
Reenter password for [elastic]:
Enter password for [apm_system]:
Reenter password for [apm_system]:
Enter password for [kibana_system]:
Reenter password for [kibana_system]:
Enter password for [logstash_system]:
Reenter password for [logstash_system]:
Enter password for [beats_system]:
Reenter password for [beats_system]:
Enter password for [remote_monitoring_user]:
Reenter password for [remote_monitoring_user]:
Changed password for user [apm_system]
Changed password for user [kibana_system]
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [beats_system]
Changed password for user [remote_monitoring_user]
Changed password for user [elastic]
测试
[es@es1 bin]$ curl -u "elastic:123456" http://192.168.10.180:9200/_cat/nodes?v
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
192.168.10.181 13 79 5 0.82 0.88 0.65 dilmrt - node-2
192.168.10.180 14 98 5 0.10 0.14 0.12 dilmrt * node-1
3.5 修改密码
3.5.1 已知现在密码修改
-u 是现在的密码
-d 是将要修改成的密码
[es@es1 bin]$ curl -u "elastic:123456" http://192.168.10.180:9200/_cat/nodes?v
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
192.168.10.181 13 79 5 0.82 0.88 0.65 dilmrt - node-2
192.168.10.180 14 98 5 0.10 0.14 0.12 dilmrt * node-1[es@es1 bin]$ curl -H "Content-Type:application/json" -XPOST -u elastic:123456 'http://192.168.10.180:9200/_xpack/security/user/elastic/_password' -d '{ "password" : "1234567" }'
{}[es@es1 bin]$ curl -u "elastic:123456" http://192.168.10.180:9200/_cat/nodes?v{"error":{"root_cause":[{"type":"security_exception","reason":"unable to authenticate user [elastic] for REST request [/_cat/nodes?v]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"unable to authenticate user [elastic] for REST request [/_cat/nodes?v]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}[es@es1 bin]$ curl -u "elastic:1234567" http://192.168.10.180:9200/_cat/nodes?v
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
192.168.10.181 33 79 3 0.06 0.48 0.57 dilmrt - node-2
192.168.10.180 34 98 3 0.45 0.19 0.15 dilmrt * node-1
3.5.2 忘记密码
创建超级用户
[es@es1 bin]$ ./elasticsearch-users useradd myname -p mypassword -r superuser[es@es1 bin]$ curl -u myname:mypassword -XPUT 'http://192.168.10.180:9200/_xpack/security/user/elastic/_password?pretty' -H 'Content-Type: application/json' -d '{"password" : "12345678"}'
{ }[es@es1 bin]$ curl -u "elastic:1234567" http://192.168.10.180:9200/_cat/nodes?v{"error":{"root_cause":[{"type":"security_exception","reason":"unable to authenticate user [elastic] for REST request [/_cat/nodes?v]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"unable to authenticate user [elastic] for REST request [/_cat/nodes?v]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}[es@es1 bin]$ curl -u "elastic:12345678" http://192.168.10.180:9200/_cat/nodes?vip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
192.168.10.181 50 79 3 0.00 0.18 0.41 dilmrt - node-2
192.168.10.180 50 98 3 0.95 0.65 0.36 dilmrt * node-1
[es@es1 bin]$
四、ES升级
1 备份旧版本
[root@localhost data]# ps -ef | grep elas
es 26795 1 4 15:21 ? 00:02:00 /usr/local/java/bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dio.netty.allocator.numDirectArenas=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.locale.providers=SPI,JRE -Xms3g -Xmx3g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Djava.io.tmpdir=/tmp/elasticsearch-3796414330494870773 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=logs/hs_err_pid%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:logs/gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=32 -XX:GCLogFileSize=64m -XX:MaxDirectMemorySize=1610612736 -Des.path.home=/data/elasticsearch -Des.path.conf=/data/elasticsearch/config -Des.distribution.flavor=default -Des.distribution.type=tar -Des.bundled_jdk=true -cp /data/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch -d
es 26823 26795 0 15:21 ? 00:00:00 /data/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller
root 29695 29495 0 16:05 pts/1 00:00:00 grep --color=auto elas
[root@localhost data]# pkill -9 java
[root@localhost data]# ps -ef | grep elas
root 29752 29495 0 16:06 pts/1 00:00:00 grep --color=auto elas2 解压新版本，替换旧版本
[root@localhost ~]# tar xf elasticsearch-7.16.2-linux-x86_64.tar.gz
[root@localhost ~]# mv elasticsearch-7.16.2 /data/elasticsearch
3 修改新版本的配置文件（拷贝旧版本的配置）
[root@localhost data]# cat /data/elasticsearch7.8.0/config/elasticsearch.yml > /data/elasticsearch/config/elasticsearch.yml
[root@localhost data]# cp -a /data/elasticsearch7.8.0/data /data/elasticsearch/
[root@localhost data]# cp -a /data/elasticsearch7.8.0/es_backup /data/elasticsearch/
[root@localhost config]# cp -a /data/elasticsearch7.8.0/config/elastic-certificates.p12 /data/elasticsearch/config/4 修改新版本的属主
[root@localhost data]# chown -R es:es /data/elasticsearch
5 启动
[root@localhost config]# su es
[es@localhost config]$ cd /data/elasticsearch/bin/
[es@localhost bin]$ ./elasticsearch -d
6 测试
[es@localhost bin]$ curl -u elastic:123456 http://192.168.10.180:9200
{"name" : "node-1","cluster_name" : "elasticsearch","cluster_uuid" : "JFPQN70-RLaRCUIGYS5VuQ","version" : {"number" : "7.16.2","build_flavor" : "default","build_type" : "tar","build_hash" : "2b937c44140b6559905130a8650c64dbd0879cfb","build_date" : "2021-12-18T19:42:46.604893745Z","build_snapshot" : false,"lucene_version" : "8.10.1","minimum_wire_compatibility_version" : "6.8.0","minimum_index_compatibility_version" : "6.0.0-beta1"},"tagline" : "You Know, for Search"
}
[es@localhost bin]$ curl -u elastic:123456 http://192.168.10.180:9200/_cat/indices?v
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open .geoip_databases 1D0788VhT6u_Vg9G-22T5g 1 0 44 0 41.5mb 41.5mb
green open .security-7 StBKKdhJQ6ySV-2lH6yJsg 1 0 7 0 23.9kb 23.9kb
五、es备份和恢复
1、 基于快照的方式
1.1 创建镜像仓库和镜像（所有节点）
1.1.1 新建目录作为镜像仓库并修改权限为es用户
##该目录使用共享的方式（nfs，glusterfs）
[root@localhost data]# mkdir -p /data/elasticsearch/es_backup
[root@localhost data]# chown -R es:es /data/elasticsearch/es_backup
1.1.2 修改所有es的配置文件，添加仓库地址，重启es
1.1.3 创建镜像仓库
[root@localhost data]# curl -XPUT -u elastic:123456 'http://192.168.10.180:9200/_snapshot/my_backup' -H "Content-Type: application/json" -d '{"type": "fs","settings": {"location": "/data/elasticsearch/es_backup","compress": true}
}'
{"acknowledged":true}
curl -XGET -u elastic:'123456' 'http://192.168.10.180:9200/_snapshot/_all?
pretty' #查看所有仓库
curl -XGET -u elastic:'123456' 'http://192.168.10.180:9200/_cat/repositories?v'
#查看仓库列表
curl -XGET -u elastic:'123456'
'http://192.168.10.180:9200/_snapshot/my_backup' #查看指定仓库
curl -XDELETE -u elastic:'123456'
'http://192.168.10.180:9200/_snapshot/my_backup?pretty' #删除仓库
1.1.4 创建索引
[root@localhost ~]# curl -XPUT -u elastic:123456 '192.168.10.180:9200/index-demo/test/1?pretty' -H 'content-Type:application/json' -d '{"user":"zhangsan","mesg":"hello world"}'
{"_index" : "index-demo","_type" : "test","_id" : "1","_version" : 1,"result" : "created","_shards" : {"total" : 2,"successful" : 1,"failed" : 0},"_seq_no" : 0,"_primary_term" : 1
}
[root@localhost ~]# curl -u elastic:'123456' "192.168.10.180:9200/_cat/indices?v"health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open .geoip_databases 1D0788VhT6u_Vg9G-22T5g 1 0 44 0 41.5mb 41.5mb
green open .security-7 StBKKdhJQ6ySV-2lH6yJsg 1 0 7 0 23.9kb 23.9kb
yellow open index-demo l9gCn03zQ0qR7kbU3XEA9Q 1 1 1 0 4.5kb 4.5kb
1.1.5 创建镜像快照
[root@localhost ~]# curl -X PUT -u elastic:'123456' "192.168.10.180:9200/_snapshot/my_backup/es_backup-20220330?wait_for_completion=true&pretty"
{"snapshot" : {"snapshot" : "es_backup-20220330","uuid" : "Tv1hwsGRQpS5duLRYap39w","repository" : "my_backup","version_id" : 7160299,"version" : "7.16.2","indices" : [".ds-.logs-deprecation.elasticsearch-default-2022.03.30-000001",".ds-ilm-history-5-2022.03.30-000001",".security-7","index-demo",".geoip_databases"],"data_streams" : ["ilm-history-5",".logs-deprecation.elasticsearch-default"],"include_global_state" : true,"state" : "SUCCESS","start_time" : "2022-03-30T02:51:29.910Z","start_time_in_millis" : 1648608689910,"end_time" : "2022-03-30T02:51:30.111Z","end_time_in_millis" : 1648608690111,"duration_in_millis" : 201,"failures" : [ ],"shards" : {"total" : 5,"failed" : 0,"successful" : 5},"feature_states" : [{"feature_name" : "geoip","indices" : [".geoip_databases"]},{"feature_name" : "security","indices" : [".security-7"]}]}
}
## 监视任何当前正在运行的快照
curl -XGET -u elastic:'123456'
"http://192.168.10.180:9200/_snapshot/my_backup/_current?pretty"
## 要获得参与任何当前正在运行的快照的每个分片的完整细分
curl -XGET -u elastic:'123456' 'http://192.168.10.180:9200/_snapshot/_status'
## 查看仓库中所有的镜像
curl -XGET -u elastic:'123456'
'http://192.168.10.180:9200/_snapshot/my_backup/_all?pretty'
## 获取特定存储库中的快照列表，这也会返回每个快照的内容。
curl -X GET -u elastic:'123456' "192.168.10.180:9200/_snapshot/my_backup/*?
verbose=false&pretty"
## 删除指定镜像
curl -X DELETE -u elastic:'123456'
"192.168.10.180:9200/_snapshot/my_backup/es_backup-20220330"
1.1.6 删除索引
[root@localhost ~]# curl -X DELETE -u elastic:123456 "192.168.10.180:9200/index-demo"
{"acknowledged":true}
[root@localhost ~]# curl -u elastic:'123456' "192.168.10.180:9200/_cat/indices?v"
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open .geoip_databases 1D0788VhT6u_Vg9G-22T5g 1 0 44 0 41.5mb 41.5mb
green open .security-7 StBKKdhJQ6ySV-2lH6yJsg 1 0 7 0 23.9kb 23.9kb1.1.7 恢复单个索引
[root@localhost ~]# curl -X GET -u elastic:'123456' "192.168.10.180:9200/_snapshot/my_backup/*?verbose=false&pretty"
{"snapshots" : [{"snapshot" : "es_backup-20220329","uuid" : "vfgjBu5tScCQzdNlP0swaA","repository" : "my_backup","indices" : [".geoip_databases",".security-7"],"data_streams" : [ ],"state" : "SUCCESS"},{"snapshot" : "es_backup-20220330","uuid" : "Tv1hwsGRQpS5duLRYap39w","repository" : "my_backup","indices" : [".ds-.logs-deprecation.elasticsearch-default-2022.03.30-000001",".ds-ilm-history-5-2022.03.30-000001",".geoip_databases",".security-7","index-demo"],"data_streams" : [ ],"state" : "SUCCESS"}],"total" : 2,"remaining" : 0
}
[root@localhost ~]# curl -X POST -u elastic:'123456' "192.168.10.180:9200/_snapshot/my_backup/es_backup-20220330/_restore?pretty" -H 'Content-Type: application/json' -d' { "indices": "index-demo" }'
{"accepted" : true
}
[root@localhost ~]# curl -u elastic:'123456' "192.168.10.180:9200/_cat/indices?v" health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open .geoip_databases 1D0788VhT6u_Vg9G-22T5g 1 0 44 0 41.5mb 41.5mb
green open .security-7 StBKKdhJQ6ySV-2lH6yJsg 1 0 7 0 23.9kb 23.9kb
yellow open index-demo j9Y6KTE8QiSuUCMdW6pn1g 1 1 1 0 4.5kb 4.5kb
1.1.8 异机恢复索引
异机需要创建镜像仓库和复制一份文件到仓库底下
[root@mysql2 indices]# curl -XPUT -u elastic:123456 'http://192.168.10.181:9200/_snapshot/my_backup' -H "Content-Type: application/json" -d '{ "type": "fs","settings": { "location": "/data/elasticsearch/es_backup","compress": true}
}'
{"acknowledged":true}[root@mysql2 indices]#
[root@mysql2 indices]#
[root@mysql2 indices]#
[root@mysql2 indices]# curl -X GET -u elastic:'123456' "192.168.10.181:9200/_snapshot/my_backup/*?verbose=false&pretty"{"snapshots" : [{"snapshot" : "es_backup-20220329","uuid" : "vfgjBu5tScCQzdNlP0swaA","repository" : "my_backup","indices" : [".geoip_databases",".security-7"],"data_streams" : [ ],"state" : "SUCCESS"},{"snapshot" : "es_backup-20220330","uuid" : "Tv1hwsGRQpS5duLRYap39w","repository" : "my_backup","indices" : [".ds-.logs-deprecation.elasticsearch-default-2022.03.30-000001",".ds-ilm-history-5-2022.03.30-000001",".geoip_databases",".security-7","index-demo"],"data_streams" : [ ],"state" : "SUCCESS"}],"total" : 2,"remaining" : 0
}
[root@mysql2 ~]# curl -u elastic:'123456' "192.168.10.181:9200/_cat/indices?v"
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open .geoip_databases jrIHBN1KTfqDAgKKeFZP8w 1 0 40 0 38mb 38mb
green open .security-7 Ahj0JDaARPq-OslsHxhohw 1 0 7 0 25.7kb 25.7kb
[root@mysql2 ~]# curl -X POST -u elastic:'123456' "192.168.10.181:9200/_snapshot/my_backup/es_backup-20220330/_restore?pretty" -H 'Content-Type: application/json' -d' { "indices": "index-demo" }'
{"accepted" : true
}
[root@mysql2 ~]# curl -u elastic:'123456' "192.168.10.181:9200/_cat/indices?v"
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open .geoip_databases jrIHBN1KTfqDAgKKeFZP8w 1 0 40 0 38mb 38mb
green open .security-7 Ahj0JDaARPq-OslsHxhohw 1 0 7 0 25.7kb 25.7kb
yellow open index-demo 5-bajJbvSYyjODxAxFntQA 1 1 1 0 4.5kb 4.5kb
[root@mysql2 ~]#
2、基于esdump的方式
Elasticsearch搭建相关推荐
- Python分布式爬虫-elasticsearch搭建搜索引擎
Python分布式爬虫-elasticsearch搭建搜索引擎 一.elasticsearch使用 1.elasticsearch介绍 ElasticSearch是一个基于Lucene的搜索服务器.它 ...
- 用ElasticSearch搭建自己的搜索和分析引擎
导语:互联网产品中的检索功能随处可见.当你的项目规模是百度大搜|商搜或者微信公众号搜索这种体量的时候,自己开发一个搜索引擎,加入各种定制的需求和优化,是非常自然的事情.但如果只是普通的中小型项目甚至创 ...
- elasticsearch搭建与java应用实例
0.学习目标 独立安装Elasticsearch 会使用Rest的API操作索引 会使用Rest的API查询数据 会使用Rest的API聚合数据 掌握Spring Data Elasticsearch ...
- ElasticSearch搭建集群
查看集群状态:GET _cat/health?v epoch timestamp cluster status node.total node.data shards pri relo init un ...
- 重构实践：基于腾讯云Elasticsearch搭建QQ邮箱全文检索
导语 | 随着用户邮件数量越来越多,邮件搜索已是邮箱的基本功能.QQ 邮箱于 2008 年推出的自研搜索引擎面临着存储机器逐渐老化,存储机型面临淘汰的境况.因此,需要搭建一套新的全文检索服务,迁移存储 ...
- 使用 Elasticsearch 搭建自己的搜索系统，真心强大！
什么是elasticsearch Elasticsearch 是一个开源的高度可扩展的全文搜索和分析引擎,拥有查询近实时的超强性能. 大名鼎鼎的Lucene 搜索引擎被广泛用于搜索领域,但是操作复杂繁 ...
- 使用elasticSearch搭建本地以图搜图服务
使用elasticSearch的向量检索实现以图搜图 文章目录 使用elasticSearch的向量检索实现以图搜图 前言 快速开始 安装elasticsearch和kibana 创建索引 下载项目 ...
- ElasticSearch搭建集群详解
Elasticsearch如果做集群的话Master节点至少三台服务器或者三个Master实例加入相同集群,三个Master节点最多只能故障一台Master节点,如果故障两个Master节点,Elas ...
- elasticsearch搭建遇到的问题整理合集1
es遇到的问题 1.防火墙 #开放9200端口允许访问 iptables -I INPUT -p tcp --dport 9200 -j ACCEPT #查看已有的端口 iptables -L -n ...
最新文章
- 《OpenGL编程指南》一第2章 着色器基础
- Ext JS 4 Beta 3 今天可以下载了
- python如何读取公共盘的文档_如何使用 Sphinx 给 Python 代码写文档 | Linux 中国
- JUC多线程：创建线程的四种方式
- Flink 零基础实战教程：如何计算实时热门商品
- Unity 和腾讯游戏成立联合创新实验室：从技术创新探索游戏产品新模式和概念
- 安卓网络连接全解：包括网络连接状态的监听、网络数据使用状态的监听、获取当前网络连接情况、启动wifi、获取当前连接wifi的网络情况、扫描wifi热点
- bzoj4567: [Scoi2016]背单词
- BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment阅读笔记
- 你真的了解牛顿第一定律吗
- nao机器人学习笔记3
- ctfshow 萌xin赛misc
- C语言中alarm的应用
- 布袋除尘器过滤风速多少_布袋除尘器过滤风速一般多大?
- 【机器学习】之 主成分分析PCA
- 计算机网络基础知识——（一）
- 不通过twitter API获取Twitter数据的方法
- 关于组装微型计算机的叙述不正确的是,计算机与交互式白板通过USB数据线物理连接后,还要正确安装(),这样才能正常使用。...
- 东信EST-100系列身份证读卡器安卓开发包，Android Studio环境编译
- 网站风格变黑白的方法，用css或javascript方法将网站改为灰色
热门文章
- 计算机系统仪表特点,一个大型计算机系统的特点
- 记我拿到一台MAC电脑后的学习研究心得
- HUAWEI华为笔记本电脑 荣耀MagicBook 16锐龙5 5600H(HYM-W56)原装出厂Windows10系统恢复原厂OEM系统21H1
- 带你学开源项目：Meizhi Android之RxJava Retrofit最佳实践
- UG模具设计系统全套视频教程-2D排位+3D分模
- oracle两张表数据匹配,Oracle-left join两表关联只取B表匹配到的第一条记录
- B-Tree Summing 记录一种用来处理复杂输入的好方法
- python编程amd和intel哪个好_2020年12月，深度学习推荐使用AMD还是INTEL的CPU？
- matlab对外部导入的数据进行三维曲面绘制
- 腾讯云Linux服务器搭建(一) 服务器选择