ES(elasticsearch)备份还原脚本

yunvn

　　在作es backup和es restore（导入）时，需要停es集群中的data节点，只保留master节点。
以下图说明，星号标识的为master节点，其他为data节点。
停两个节点之前，可以在页面上确认红色标示的docs值，再停两个data节点，检查运行着的节点其docs值与停之前一致。

　　备份脚本es_backup.sh

#!/bin/bash
filename=`date +%Y%m%d%H`
backesFile=es$filename.tar.gz
mkdir -p  /xor/elasticsearch/es_snapshot
mkdir -p  /xor/elasticsearch/es_backup/es_dump
es_backup_dir=/xor/elasticsearch/es_backup
es_snapshot_dir=/xor/elasticsearch/es_snapshot
chmod -R 777 $es_snapshot_dir
curl -XPUT localhost:9200/_snapshot/es_snapshot -d '{"type":"fs","settings":{"location":"/xor/elasticsearch/es_snapshot"}}'
cd $es_backup_dir/es_dump
curl -XDELETE localhost:9200/_snapshot/es_snapshot/$filename?
echo -e '\nsleep 5\n'
sleep 10
curl -XPUT localhost:9200/_snapshot/es_snapshot/$filename?wait_for_completion=true
echo -e '\nsleep 5\n'
sleep 5
cp $es_snapshot_dir/* $es_backup_dir/es_dump -rf
cd ..
tar czf $backesFile  es_dump/
rm es_dump -rf
cd $es_snapshot_dir
rm  * -rf
echo -e '\nes data backup succeed!\n'
　　还原脚本es_restore.sh

#!/bin/bash
filename=$1
backesFile=es$filename.tar.gz
mkdir -p  /xor/data2/packages/aqua/backup/es/es_snapshot
mkdir -p  /xor/data2/packages/aqua/backup/es/es_backup/es_dump
es_backup_dir=/xor/data2/packages/aqua/backup/es/es_backup
es_snapshot_dir=/xor/data2/packages/aqua/backup/es/es_snapshot
chmod -R 777 $es_snapshot_dir
curl -XPUT localhost:9200/_snapshot/es_snapshot -d '{"type":"fs","settings":{"location":"/xor/data2/packages/aqua/backup/es/es_snapshot"}}'
cd $es_backup_dir
tar zxvf $backesFile
rm $es_snapshot_dir/* -rf
cp $es_backup_dir/es_dump/* $es_snapshot_dir -rf
curl -XPOST localhost:9200/_all/_close
echo -e '\nsleep 5\n'
sleep 5
curl -XPOST localhost:9200/_snapshot/es_snapshot/$filename/_restore?
rm es_dump -rf
echo -e '\n es data restore succeed!'
　　es_delete_by_query.sh
用于删除es的数据。脚本需要传四个参数,参数依次为表名、时间字段名、开始时间、结束时间,示例如下:
./es_delete_by_query.sh vhsession view_begin_time 2018-01-10 00:00:00
2018-01-10 18:00:00
，时间区间包含上下界，时间字段也可以只传日期

#!/bin/sh
# example: sh  ./es_delete_by_query.sh "bundle" "licensing_window_start" "2018-01-05 15:30:00" "2018-01-05 15:30:00"
# example: sh  ./es_delete_by_query.sh "bundle" "licensing_window_start" "2018-01-05" "2018-01-06"
doc_type=$1
datefield=$2
start_time=$3
end_time=$4
if [ ! -n "$start_time" ]; then
echo "the start_time is empty,please input again...."
exit 1
else
tt=`date -d "$start_time" +"%FT%T+0800"`
if [ 0 -eq $? ]
then
start_time=$tt
else
exit 1
fi      
fi
if [ ! -n "$end_time" ]; then
echo "the end_time is empty,please input again...."
exit 1
else
tt=`date -d "$end_time" +"%FT%T+0800"`
if [ 0 -eq $? ]
then
end_time=$tt
else
exit 1
fi   
fi
echo "args: doc_type = ${doc_type},datefield = ${datefield}, start_time = ${start_time}, end_time = ${end_time}"
echo "query count response : "
curl  -XPOST http://localhost:9200/aquapaassearch/${doc_type}/_count?pretty -d '
{
"query": {
"filtered": {
"filter": {
"bool": {
"must": {
"range": {
"'${datefield}'": {
"from": "'${start_time}'",
"to": "'${end_time}'",
"include_lower": true,
"include_upper": true
}
}
}
}
}
}
}
}'
read -p "please input '1'(otherwise not) to confirm if you need delete:" flag
if [ "1" = "$flag" ]; then
echo "delete response : "
curl  -XDELETE http://localhost:9200/aquapaassearch/${doc_type}/_query?pretty -d '
{
"query": {
"filtered": {
"filter": {
"bool": {
"must": {
"range": {
"'${datefield}'": {
"from": "'${start_time}'",
"to": "'${end_time}'",
"include_lower": true,
"include_upper": true
}
}
}
}
}
}
}
}'
echo "delete sucess!!!"
else
echo "confirmed just query count!!!"
fi