Mirrored_Repos/gravity-sync
1
0
Fork 0
mirror of https://github.com/vmstan/gravity-sync.git synced 2024-08-30 18:22:11 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #7 from vmstan/development

Browse Source 
v1.3.0
This commit is contained in:
Michael Stanclift 2020-05-24 22:21:41 -05:00 committed by GitHub
commit 9bd9d628e1
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
4 changed files with 326 additions and 87 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored
View File

@ -1,4 +1,5 @@
gravity-sync.log
gravity-sync.cron
gravity.db.last
gravity-sync.conf
backup/*.last

14
README.md
View File

@ -2,7 +2,7 @@
For more information visit [https://vmstan.com/gravity-sync/](https://vmstan.com/gravity-sync/)
![Pull execution](https://user-images.githubusercontent.com/3002053/82719284-c396d880-9c6e-11ea-859c-b316357a495c.png)
![Pull execution](https://user-images.githubusercontent.com/3002053/82774990-f88c6200-9e0b-11ea-97e5-23c8b38f32e3.png)
## Background
@ -12,7 +12,7 @@ The script assumes you have one "primary" PH as the place you make all your conf
It will **not** overwrite device specific settings such as local network configuration, admin/API passwords/keys, upstream DNS resolvers, etc. It will also **not** keep DHCP settings or device leases synchronized.
## Prereqs
## Prerequisites
You will need to designate one Pi-Hole as primary and one as secondary. The designation is purely at your discretion and depends on your desired use case:
@ -68,9 +68,9 @@ Download the latest release from [GitHub](https://github.com/vmstan/gravity-sync
```
cd ~
wget https://github.com/vmstan/gravity-sync/archive/v1.2.5.zip
unzip v1.2.5.zip
mv ~/gravity-sync-1.2.5 ~/gravity-sync
wget https://github.com/vmstan/gravity-sync/archive/v1.3.0.zip
unzip v1.3.0.zip
mv ~/gravity-sync-1.3.0 ~/gravity-sync
cd gravity-sync
```
@ -108,7 +108,7 @@ The script, when functioning in `pull` mode, will not prompt for user input afte
./gravity-sync.sh pull
```
If the execution completes, you will now have overwritten your running gravity.db on the secondary PH after creating a copy (`gravity.db.backup`) in the `backup` directory located with your script. The script will also keep a copy of the last sync'd gravity.db from the master, in the `backup` folder called `gravity.db.last` should you need it for some reason.
If the execution completes, you will now have overwritten your running gravity.db on the secondary PH after creating a copy (`gravity.db.backup`) in the `backup` directory located with your script. The script will also keep a copy of the last sync'd gravity.db from the master, in the `backup` folder called `gravity.db.pull` should you need it for some reason.
Finally, a file called `gravity-sync.log` will be created in the `gravity-sync` folder along side the script, with the date the script was last executed appended to the bottom. Over time I intend for this logging function to become more helpful.
@ -144,7 +144,7 @@ I've automated my synchronization using Crontab. If you'd like to keep this a ma
```
crontab -e
*/30 * * * * /home/USER/gravity-sync/gravity-sync.sh pull >/dev/null 2>&1
*/30 * * * * /bin/bash /home/USER/gravity-sync/gravity-sync.sh pull > /home/USER/gravity-sync/gravity-sync.cron
```
Now, make another small adjustment to your primary settings. Now just wait until the annointed hour, and see if your changes have been synchronized. If so, profit!

13
gravity-sync.conf.example
View File

@ -1,3 +1,10 @@
# REQUIRED SETTINGS
REMOTE_HOST='192.168.1.10' # your master Pihole IP or DNS
REMOTE_USER='pi' # user account with sudo rights on master Pihole
# REQUIRED SETTINGS ######################
# Pi-hole Primary Node in IP or DNS
REMOTE_HOST='192.168.1.10'
# User on Primary PH with SUDO rights
REMOTE_USER='pi'
# Password for REMOTE_USER account
# Leave blank to use key-pair SSH auth
REMOTE_PASS=''

385
gravity-sync.sh
View File

@ -1,7 +1,8 @@
#!/bin/bash
# GRAVITY SYNC BY VMSTAN #####################
VERSION='1.2.5'
PROGRAM='Gravity Sync'
VERSION='1.3.0'
# Must execute from a location in the home folder of the user who own's it (ex: /home/pi/gravity-sync)
# Configure certificate based SSH authentication between the Pi-hole HA nodes - it does not use passwords
@ -11,7 +12,7 @@ VERSION='1.2.5'
# REQUIRED SETTINGS ##########################
# You MUST define REMOTE_HOST and REMOTE_USER in a file called 'gravity-sync.conf'
# You MUST define REMOTE_HOST and REMOTE_USER in a file called 'gravity-sync.conf' OK
# You can copy the 'gravity-sync.conf.example' file in the script directory to get started
# STANDARD VARIABLES #########################
@ -20,12 +21,17 @@ VERSION='1.2.5'
LOCAL_FOLDR='gravity-sync' # must exist in running user home folder
CONFIG_FILE='gravity-sync.conf' # must exist as explained above
SYNCING_LOG='gravity-sync.log' # will be created in above folder
CRONJOB_LOG='gravity-sync.cron'
BACKUP_FOLD='backup'
# PH Folder/File Locations
PIHOLE_DIR='/etc/pihole' # default install directory
GRAVITY_FI='gravity.db' # this should not change
##############################################
### DO NOT CHANGE ANYTHING BELOW THIS LINE ###
##############################################
# Script Colors
RED='\033[0;91m'
GREEN='\033[0;92m'
@ -34,22 +40,33 @@ YELLOW='\033[0;93m'
PURPLE='\033[0;95m'
NC='\033[0m'
##############################################
### DO NOT CHANGE ANYTHING BELOW THIS LINE ###
##############################################
# Message Codes
FAIL="[${RED}FAIL${NC}]"
WARN="[${PURPLE}WARN${NC}]"
GOOD="[${GREEN}GOOD${NC}]"
STAT="[${CYAN}EXEC${NC}]"
INFO="[${YELLOW}INFO${NC}]"
# FUNCTION DEFINITIONS #######################
# Import Settings
function import_gs {
echo -e "[${CYAN}STAT${NC}] Importing ${CONFIG_FILE} Settings"
MESSAGE="Importing ${CONFIG_FILE} Settings"
echo -e "${STAT} $MESSAGE"
if [ -f $HOME/${LOCAL_FOLDR}/${CONFIG_FILE} ]
then
source $HOME/${LOCAL_FOLDR}/${CONFIG_FILE}
echo -e "[${GREEN}GOOD${NC}] Using ${REMOTE_USER}@${REMOTE_HOST}"
error_validate
MESSAGE="Using ${REMOTE_USER}@${REMOTE_HOST}"
echo -e "${INFO} ${MESSAGE}"
else
echo -e "[${RED}FAIL${NC}] Required ${CONFIG_FILE} Missing"
echo -e "Please review installation documentation for more information"
echo -e "${FAIL} ${MESSAGE}"
MESSAGE="${CONFIG_FILE} Missing"
echo -e "${INFO} ${MESSAGE}"
exit_nochange
fi
}
@ -58,58 +75,123 @@ function import_gs {
function update_gs {
TASKTYPE='UPDATE'
logs_export # dumps log prior to execution because script stops after successful pull
echo -e "[${PURPLE}WARN${NC}] Requires GitHub Installation"
MESSAGE="Requires GitHub Installation"
echo -e "${INFO} ${MESSAGE}"
git reset --hard
git pull
exit
}
# Developer Build Update
function beta_gs {
TASKTYPE='BETA'
logs_export # dumps log prior to execution because script stops after successful pull
MESSAGE="Requires GitHub Installation"
echo -e "${INFO} ${MESSAGE}"
git reset --hard
git pull
git checkout origin/development
exit
}
# Pull Function
function pull_gs {
TASKTYPE='PULL'
echo -e "[${CYAN}STAT${NC}] Pulling ${GRAVITY_FI} from ${REMOTE_HOST}"
echo -e "${INFO} ${TASKTYPE} Requested"
md5_compare
MESSAGE="Pulling ${GRAVITY_FI} from ${REMOTE_HOST}"
echo -e "${STAT} ${MESSAGE}"
rsync -v -e 'ssh -p 22' ${REMOTE_USER}@${REMOTE_HOST}:${PIHOLE_DIR}/${GRAVITY_FI} $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${GRAVITY_FI}.pull
echo -e "[${CYAN}STAT${NC}] Backing Up Current ${GRAVITY_FI} on $HOSTNAME"
error_validate
MESSAGE="Backing Up ${GRAVITY_FI} on $HOSTNAME"
echo -e "${STAT} ${MESSAGE}"
cp -v ${PIHOLE_DIR}/${GRAVITY_FI} $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${GRAVITY_FI}.backup
echo -e "[${CYAN}STAT${NC}] Replacing ${GRAVITY_FI} on $HOSTNAME"
error_validate
MESSAGE="Replacing ${GRAVITY_FI} on $HOSTNAME"
echo -e "${STAT} ${MESSAGE}"
sudo cp -v $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${GRAVITY_FI}.pull ${PIHOLE_DIR}/${GRAVITY_FI}
error_validate
MESSAGE="Setting Permissions on ${GRAVITY_FI}"
echo -e "${STAT} ${MESSAGE}"
sudo chmod 644 ${PIHOLE_DIR}/${GRAVITY_FI}
error_validate
MESSAGE="Setting Ownership on ${GRAVITY_FI}"
echo -e "${STAT} ${MESSAGE}"
sudo chown pihole:pihole ${PIHOLE_DIR}/${GRAVITY_FI}
echo -e "${GRAVITY_FI} ownership and file permissions reset"
echo -e "[${CYAN}STAT${NC}] Reloading FTLDNS Configuration"
error_validate
sleep 3
MESSAGE="Updating FTLDNS Configuration"
echo -e "${STAT} ${MESSAGE}"
pihole restartdns reloadlists
error_validate
MESSAGE="Reloading FTLDNS Services"
echo -e "${STAT} ${MESSAGE}"
pihole restartdns
logs_export
error_validate
logs_export
exit_withchange
}
# Push Function
function push_gs {
TASKTYPE='PUSH'
echo -e "[${PURPLE}WARN${NC}] DATA LOSS IS POSSIBLE"
echo -e "The standard use of this script is to ${YELLOW}PULL${NC} data from the"
echo -e "primary PH server to the secondary. By issuing a ${YELLOW}PUSH${NC}, we"
echo -e "will instead overwrite the ${GRAVITY_FI} on ${YELLOW}${REMOTE_HOST}${NC}"
echo -e "with ${YELLOW}$HOSTNAME${NC} server data. A copy of the remote ${GRAVITY_FI}"
echo -e "will be saved to this server at:"
echo -e "${YELLOW}$HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${GRAVITY_FI}.push${NC}"
echo -e ""
echo -e "Are you sure you want to overwrite the primary node configuration on ${REMOTE_HOST}?"
echo -e "${INFO} ${TASKTYPE} Requested"
md5_compare
echo -e "${WARN} Are you sure you want to overwrite the primary node configuration on ${REMOTE_HOST}?"
select yn in "Yes" "No"; do
case $yn in
Yes )
# echo "Replacing gravity.db on primary"
echo -e "[${CYAN}STAT${NC}] Backing Up ${GRAVITY_FI} from ${REMOTE_HOST}"
MESSAGE="Backing Up ${GRAVITY_FI} from ${REMOTE_HOST}"
echo -e "${STAT} ${MESSAGE}"
rsync -v -e 'ssh -p 22' ${REMOTE_USER}@${REMOTE_HOST}:${PIHOLE_DIR}/${GRAVITY_FI} $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${GRAVITY_FI}.push
echo -e "[${CYAN}STAT${NC}] Pushing ${GRAVITY_FI} to ${REMOTE_HOST}"
error_validate
MESSAGE="Pushing ${GRAVITY_FI} to ${REMOTE_HOST}"
echo -e "${STAT} ${MESSAGE}"
rsync --rsync-path="sudo rsync" -v -e 'ssh -p 22' ${PIHOLE_DIR}/${GRAVITY_FI} ${REMOTE_USER}@${REMOTE_HOST}:${PIHOLE_DIR}/${GRAVITY_FI}
echo -e "[${CYAN}STAT${NC}] Applying Permissions to Remote ${GRAVITY_FI}"
error_validate
MESSAGE="Setting Permissions on ${GRAVITY_FI}"
echo -e "${STAT} ${MESSAGE}"
ssh ${REMOTE_USER}@${REMOTE_HOST} "sudo chmod 644 ${PIHOLE_DIR}/${GRAVITY_FI}"
error_validate
MESSAGE="Setting Ownership on ${GRAVITY_FI}"
echo -e "${STAT} ${MESSAGE}"
ssh ${REMOTE_USER}@${REMOTE_HOST} "sudo chown pihole:pihole ${PIHOLE_DIR}/${GRAVITY_FI}"
echo -e "[${CYAN}STAT${NC}] Reloading FTLDNS Configuration"
error_validate
sleep 3
MESSAGE="Updating FTLDNS Configuration"
echo -e "${STAT} ${MESSAGE}"
ssh ${REMOTE_USER}@${REMOTE_HOST} 'pihole restartdns reloadlists'
error_validate
MESSAGE="Reloading FTLDNS Services"
echo -e "${STAT} ${MESSAGE}"
ssh ${REMOTE_USER}@${REMOTE_HOST} 'pihole restartdns'
logs_export
error_validate
logs_export
exit_withchange
;;
@ -123,15 +205,28 @@ function push_gs {
# Logging Functions
## Check Log Function
function logs_gs {
echo -e "Recent ${YELLOW}PULL${NC} attempts"
echo -e "========================================================"
echo -e "Recent Complete ${YELLOW}PULL${NC} Executions"
tail -n 10 ${SYNCING_LOG} | grep PULL
echo -e "Recent ${YELLOW}UPDATE${NC} attempts"
echo -e "Recent Complete ${YELLOW}UPDATE${NC} Executions"
tail -n 10 ${SYNCING_LOG} | grep UPDATE
echo -e "Recent ${YELLOW}PUSH${NC} attempts"
echo -e "Recent Complete ${YELLOW}PUSH${NC} Executions"
tail -n 10 ${SYNCING_LOG} | grep PUSH
echo -e "========================================================"
exit_nochange
}
## Check Last Crontab
function logs_crontab {
echo -e "========================================================"
echo -e "========================================================"
echo -e ""
cat ${CRONJOB_LOG}
echo -e ""
echo -e "========================================================"
echo -e "========================================================"
}
## Log Out
function logs_export {
echo -e "[${CYAN}STAT${NC}] Logging Timestamps to ${SYNCING_LOG}"
@ -142,32 +237,58 @@ function logs_export {
# Validate Functions
## Validate GS Folders
function validate_gs_folders {
if [ -d $HOME/${LOCAL_FOLDR} ]
then
echo -e "[${GREEN}GOOD${NC}] Required $HOME/${LOCAL_FOLDR} Located"
else
echo -e "[${RED}FAIL${NC}] Required $HOME/${LOCAL_FOLDR} Missing"
exit_nochange
fi
MESSAGE="Locating $HOME/${LOCAL_FOLDR}"
echo -e "${STAT} ${MESSAGE}"
if [ -d $HOME/${LOCAL_FOLDR} ]
then
echo -e "${GOOD} ${MESSAGE}"
else
echo -e "${FAIL} ${MESSAGE}"
exit_nochange
fi
if [ -d $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD} ]
then
echo -e "[${GREEN}GOOD${NC}] Required $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD} Located"
else
echo -e "[${RED}FAIL${NC}] Required $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD} Missing"
exit_nochange
fi
MESSAGE="Locating $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}"
echo -e "${STAT} ${MESSAGE}"
if [ -d $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD} ]
then
echo -e "${GOOD} ${MESSAGE}"
else
echo -e "${FAIL} ${MESSAGE}"
exit_nochange
fi
}
## Validate PH Folders
function validate_ph_folders {
if [ -d ${PIHOLE_DIR} ]
then
echo -e "[${GREEN}GOOD${NC}] Required ${PIHOLE_DIR} Located"
else
echo -e "[${RED}FAIL${NC}] Required ${PIHOLE_DIR} Missing"
exit_nochange
fi
MESSAGE="Locating ${PIHOLE_DIR}"
echo -e "${STAT} ${MESSAGE}"
if [ -d ${PIHOLE_DIR} ]
then
echo -e "${GOOD} ${MESSAGE}"
else
echo -e "${FAIL} ${MESSAGE}"
exit_nochange
fi
}
## Validate SSHPASS
function validate_os_sshpass {
if hash sshpass 2>/dev/null
then
if test -z "$REMOTE_PASS"
then
sshpassword=''
MESSAGE="Using SSH Key-Pair Authentication"
else
sshpassword="sshpass -p ${REMOTE_PASS} "
MESSAGE="Using SSH Password Authentication"
fi
else
sshpassword=''
MESSAGE="Using SSH Key-Pair Authentication"
fi
echo -e "$INFO $MESSAGE"
}
# List GS Arguments
@ -178,11 +299,16 @@ function list_gs_arguments {
echo -e "Replication Options:"
echo -e " ${YELLOW}pull${NC} Sync the ${GRAVITY_FI} configuration on primary PH to this server"
echo -e " ${YELLOW}push${NC} Force any changes made on this server back to the primary PH"
echo -e " ${YELLOW}compare${NC} Check to see if there is any variance between primary and secondary"
echo -e ""
echo -e "Debugging Options:"
echo -e "Update Options:"
echo -e " ${YELLOW}update${NC} Use GitHub to update this script to the latest version available"
echo -e " ${YELLOW}beta${NC} Use GitHub to update this script to the latest beta version available"
echo -e ""
echo -e "Debug Options:"
echo -e " ${YELLOW}version${NC} Display the version of the current installed script"
echo -e " ${YELLOW}logs${NC} Show recent successful jobs"
echo -e " ${YELLOW}cron${NC} Display output of last crontab execution"
echo -e ""
exit_nochange
}
@ -190,79 +316,184 @@ function list_gs_arguments {
# Exit Codes
## No Changes Made
function exit_nochange {
echo -e "[${CYAN}STAT${NC}] Exiting Without Making Changes"
exit
echo -e "${INFO} ${PROGRAM} ${TASKTYPE} Aborting"
exit 0
}
## Changes Made
function exit_withchange {
echo -e "[${CYAN}STAT${NC}] ${GRAVITY_FI} ${TASKTYPE} Completed"
exit
echo -e "${INFO} ${PROGRAM} ${TASKTYPE} Completed"
exit 0
}
# Error Validation
function error_validate {
if [ "$?" != "0" ]; then
echo -e "${FAIL} ${MESSAGE}"
exit 1
else
echo -e "${GOOD} ${MESSAGE}"
fi
}
# Output Version
function show_version {
echo -e "${INFO} ${PROGRAM} ${VERSION}"
}
# Look for Changes
function md5_compare {
echo -e "${INFO} Comparing ${GRAVITY_FI} Changes"
MESSAGE="Analyzing Remote ${GRAVITY_FI}"
echo -e "${STAT} ${MESSAGE}"
primaryMD5=$(ssh ${REMOTE_USER}@${REMOTE_HOST} 'md5sum /etc/pihole/gravity.db')
error_validate
MESSAGE="Analyzing Local ${GRAVITY_FI}"
echo -e "${STAT} ${MESSAGE}"
secondMD5=$(md5sum ${PIHOLE_DIR}/${GRAVITY_FI})
error_validate
if [ "$primaryMD5" == "$secondMD5" ]
then
echo -e "${INFO} No Changes in ${GRAVITY_FI}"
exit_nochange
else
echo -e "${INFO} Changes Detected in ${GRAVITY_FI}"
fi
}
# SCRIPT EXECUTION ###########################
echo -e "[${CYAN}STAT${NC}] Evaluating Script Arguments"
MESSAGE="Evaluating Script Arguments"
echo -e "${STAT} ${MESSAGE}"
case $# in
0)
echo -e "[${RED}FAIL${NC}] Missing Required Arguments"
echo -e "${FAIL} ${MESSAGE}"
list_gs_arguments
;;
1)
case $1 in
pull)
echo -e "[${GREEN}GOOD${NC}] Pull Requested"
import_gs
echo -e "[${CYAN}STAT${NC}] Validating Folder Configuration"
echo -e "${GOOD} ${MESSAGE}"
import_gs
MESSAGE="Validating Folder Configuration"
echo -e "${INFO} ${MESSAGE}"
validate_gs_folders
validate_ph_folders
validate_os_sshpass
pull_gs
exit
;;
push)
echo -e "[${GREEN}GOOD${NC}] Push Requested"
import_gs
echo -e "${GOOD} ${MESSAGE}"
import_gs
echo -e "[${CYAN}STAT${NC}] Validating Folder Configuration"
echo -e "${INFO} Validating Folder Configuration"
validate_gs_folders
validate_ph_folders
validate_os_sshpass
push_gs
exit
;;
version)
echo -e "[ ${PURPLE}INFO${NC} ] Gravity Sync ${VERSION}"
TASKTYPE='VERSION'
show_version
exit_nochange
;;
update)
echo -e "[${GREEN}GOOD${NC}] Update Requested"
# TASKTYPE='UPDATE'
echo -e "${GOOD} ${MESSAGE}"
echo -e "${INFO} Update Requested"
update_gs
exit_nochange
;;
beta)
# TASKTYPE='BETA'
echo -e "${GOOD} ${MESSAGE}"
echo -e "${INFO} Beta Update Requested"
beta_gs
exit_nochange
;;
logs)
echo -e "[${GREEN}GOOD${NC}] Logs Requested"
TASKTYPE='LOGS'
MESSAGE="Logs Requested"
echo -e "${GOOD} ${MESSAGE}"
logs_gs
;;
compare)
TASKTYPE='COMPARE'
echo -e "${GOOD} ${MESSAGE}"
import_gs
echo -e "${INFO} Validating Folder Configuration"
validate_gs_folders
validate_ph_folders
validate_os_sshpass
md5_compare
;;
cron)
TASKTYPE='CRON'
echo -e "${GOOD} ${MESSAGE}"
CRONPATH="$HOME/${LOCAL_FOLDR}/${CRONJOB_LOG}"
MESSAGE="Replaying Last Cronjob"
echo -e "${STAT} ${MESSAGE}"
if [ -f ${CRONPATH} ]
then
if [ -s ${CRONPATH} ]
echo -e "${GOOD} ${MESSAGE}"
logs_crontab
exit_nochange
then
echo -e "${FAIL} ${MESSAGE}"
echo -e "${INFO} ${CRONPATH} appears empty"
exit_nochange
fi
else
echo -e "${FAIL} ${MESSAGE}"
echo -e "${YELLOW}${CRONPATH}${NC} cannot be located"
exit_nochange
fi
;;
*)
echo -e "[${RED}FAIL${NC}] '${YELLOW}$1${NC}' is Invalid Argument"
MESSAGE="'${YELLOW}$1${NC}' is an Invalid Argument"
echo -e "${FAIL} ${MESSAGE}"
list_gs_arguments
exit_nochange
;;
esac
;;
*)
echo -e "[${RED}FAIL${NC}] Too Many Arguments"
list_gs_arguments
exit_nochange
MESSAGE="Too Many Arguments"
echo -e "${FAIL} ${MESSAGE}"
list_gs_arguments
exit_nochange
;;
esac