mirror of
https://github.com/vmstan/gravity-sync.git
synced 2024-08-30 18:22:11 +00:00
Michael Stanclift
GitHub
commit
ef68174fd2
19
README.md
19
README.md
@ -6,9 +6,9 @@ That's Gravity Sync.
|
||||
|
||||
|
||||
|
||||
At it's core, Gravity Sync is maybe a handful of core bash commands, that uses rsync to reach out to a remote host, copy the running `gravity.db` file that contains the Pi-hole blocklist, and then replaces the copy on the local system. What Gravity Sync provides is an easy way to keep this happening in the background. Ideally you set it and forget it. In the long term, it would be awesome if the Pi-hole team made this entire script unncessary.
|
||||
At it's core, Gravity Sync is maybe a handful of core bash commands, that uses rsync to reach out to a remote host, copy the running `gravity.db` and `custom.list` files that contains the Pi-hole blocklist, as well as the `custom.list` file that contains local DNS enteries, and then replaces the copy on the local system. What Gravity Sync provides is an easy way to keep this happening in the background. Ideally you set it and forget it. In the long term, it would be awesome if the Pi-hole team made this entire script unncessary.
|
||||
|
||||
Gravity Sync will **not** overwrite device specific settings such as local network configuration, admin/API passwords/keys, local hostfiles, upstream DNS resolvers, etc. It will also **not** keep DHCP settings or device leases synchronized.
|
||||
Gravity Sync will **not** overwrite device specific settings such as device network configuration, admin/API passwords/keys, upstream DNS resolvers, etc. It will also **not** keep DHCP settings or device leases synchronized.
|
||||
|
||||
## Prerequisites
|
||||
Gravity Sync **requires** Pi-hole 5.0 or higher.
|
||||
@ -20,6 +20,7 @@ The designation of primary and secondary is purely at your discretion and depend
|
||||
Additionally, some things to consider:
|
||||
|
||||
- Gravity Sync is regularly tested during development with Ubuntu and Raspberry Pi OS (previously, Raspbian). As Gravity Sync is just an (admittedly) long bash script, it will likely work on other Linux distributions that have the `bash` shell installed. But please file an Issue if you're unable to run it on another platform.
|
||||
- Gravity Sync uses SUDO to elevate permissions for itself during execution. You will need to make sure that you have passwordless SUDO enabled for the accounts on both the primary and secondary Pi-hole that will be performing the work. Most of the pre-built images available for the Raspberry Pi already have this configured, but if you have your Pi-hole running in a virtual machine, you may need to adjust this manually. [This tutorial](https://linuxize.com/post/how-to-run-sudo-command-without-password/) may be helpful in this respect.
|
||||
- Gravity Sync has not been tested with Docker container deployments of Pi-hole, and is not expected to work there without major modifications. You will need Pi-hole setup with a "traditional" install directly in the base operating system.
|
||||
|
||||
## Installation
|
||||
@ -44,9 +45,9 @@ Download the latest release from [GitHub](https://github.com/vmstan/gravity-sync
|
||||
|
||||
```bash
|
||||
cd ~
|
||||
wget https://github.com/vmstan/gravity-sync/archive/v1.6.0zip
|
||||
unzip v1.6.0.zip
|
||||
mv ~/gravity-sync-1.6.0 ~/gravity-sync
|
||||
wget https://github.com/vmstan/gravity-sync/archive/v1.7.0.zip
|
||||
unzip v1.7.0.zip
|
||||
mv ~/gravity-sync-1.7.0 ~/gravity-sync
|
||||
cd gravity-sync
|
||||
```
|
||||
|
||||
@ -95,7 +96,7 @@ Gravity Sync uses SSH to run commands on the primary Pi-hole, and sync the two s
|
||||
#### Key-Pair Authentication
|
||||
This is the preferred option, as it's more reliable and less dependant on third party plugins.
|
||||
|
||||
You'll need to generate an SSH key for your secondary Pi-hole user and copy it to your primary Pi-hole. This will allow you to connect to and copy the gravity.db file without needing a password each time. When generating the SSH key, accept all the defaults and do not put a passphrase on your key file.
|
||||
You'll need to generate an SSH key for your secondary Pi-hole user and copy it to your primary Pi-hole. This will allow you to connect to and copy the necessary files without needing a password each time. When generating the SSH key, accept all the defaults and do not put a passphrase on your key file.
|
||||
|
||||
*Note: If you already have this setup on your systems for other purposes, you can skip this step.*
|
||||
|
||||
@ -142,7 +143,7 @@ The Gravity Sync Pull, is the standard method of sync operation, and will not pr
|
||||
./gravity-sync.sh pull
|
||||
```
|
||||
|
||||
If the execution completes, you will now have overwritten your running gravity.db on the secondary Pi-hole after creating a copy of the running database (`gravity.db.backup`) in the `backup` subfolder located with your script. Gravity Sync will also keep a copy of the last sync'd gravity.db from the primary (in the `backup` folder identified as `gravity.db.pull`) for future use.
|
||||
If the execution completes, you will now have overwritten your running `gravity.db` and `custom.list` on the secondary Pi-hole after creating a copy of the running files (with `.backup` appended) in the `backup` subfolder located with your script. Gravity Sync will also keep a copy of the last sync'd files from the primary (in the `backup` folder appended with `.pull`) for future use.
|
||||
|
||||
Finally, a file called `gravity-sync.log` will be created in the `gravity-sync` folder along side the script with the date the script was last executed appended to the bottom.
|
||||
|
||||
@ -155,7 +156,7 @@ Gravity Sync includes the ability to `push` from the secondary Pi-hole back to t
|
||||
./gravity-sync.sh push
|
||||
```
|
||||
|
||||
Before executing, this will make a copy of the remote database under `backup/gravity.db.push` then sync the local configuration to the primary Pi-hole.
|
||||
Before executing, this will make a copy of the remote database under `backup/gravity.db.push` and `backup/custom.list.push` then sync the local configuration to the primary Pi-hole.
|
||||
|
||||
This function purposefuly asks for user interaction to avoid being accidentally automated.
|
||||
|
||||
@ -166,7 +167,7 @@ Gravity Sync can also `restore` the database on the secondary Pi-hole in the eve
|
||||
./gravity-sync.sh restore
|
||||
```
|
||||
|
||||
This will copy your last `gravity.db.backup` to the running copy on the secondary Pi-hole.
|
||||
This will copy your last `gravity.db.backup` and `custom.list.backup` to the running copy on the secondary Pi-hole.
|
||||
|
||||
This function purposefuly asks for user interaction to avoid being accidentally automated.
|
||||
|
||||
|
||||
@ -26,4 +26,7 @@ REMOTE_PASS=''
|
||||
# LOG_PATH=''
|
||||
# SYNCING_LOG=''
|
||||
# CRONJOB_LOG=''
|
||||
# VERIFY_PASS='' # 0=Verify 1=Bypass
|
||||
|
||||
# VERIFY_PASS='' # 0=Verify 1=Bypass
|
||||
# SKIP_CUSTOM='' # 0=SyncIt 1=SkipIt
|
||||
# DATE_OUTPUT='' $ 0=NoShow 1=ShowIt (na)
|
||||
209
gravity-sync.sh
209
gravity-sync.sh
@ -2,7 +2,7 @@
|
||||
|
||||
# GRAVITY SYNC BY VMSTAN #####################
|
||||
PROGRAM='Gravity Sync'
|
||||
VERSION='1.6.0'
|
||||
VERSION='1.7.0'
|
||||
|
||||
# Execute from the home folder of the user who owns it (ex: 'cd ~/gravity-sync')
|
||||
# For documentation or downloading updates visit https://github.com/vmstan/gravity-sync
|
||||
@ -26,10 +26,13 @@ CRONJOB_LOG='gravity-sync.cron' # replace in gravity-sync.conf to overwrite
|
||||
|
||||
# Interaction Customization
|
||||
VERIFY_PASS='0' # replace in gravity-sync.conf to overwrite
|
||||
SKIP_CUSTOM='0' # replace in gravity-sync.conf to overwrite
|
||||
DATE_OUTPUT='0' # replace in gravity-sync.conf to overwrite
|
||||
|
||||
# Pi-hole Folder/File Locations
|
||||
PIHOLE_DIR='/etc/pihole' # default Pi-hole data directory
|
||||
GRAVITY_FI='gravity.db' # default Pi-hole database file
|
||||
CUSTOM_DNS='custom.list' # default Pi-hole local DNS lookups
|
||||
PIHOLE_BIN='/usr/local/bin/pihole' # default Pi-hole binary directory
|
||||
|
||||
# OS Settings
|
||||
@ -172,7 +175,66 @@ function pull_gs {
|
||||
sudo chmod 664 ${PIHOLE_DIR}/${GRAVITY_FI} >/dev/null 2>&1
|
||||
error_validate
|
||||
fi
|
||||
|
||||
|
||||
if [ "$SKIP_CUSTOM" != '1' ]
|
||||
then
|
||||
if [ "$REMOTE_CUSTOM_DNS" == "1" ]
|
||||
then
|
||||
MESSAGE="Backing Up ${CUSTOM_DNS} on $HOSTNAME"
|
||||
echo_stat
|
||||
cp ${PIHOLE_DIR}/${CUSTOM_DNS} $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${CUSTOM_DNS}.backup >/dev/null 2>&1
|
||||
error_validate
|
||||
|
||||
MESSAGE="Pulling ${CUSTOM_DNS} from ${REMOTE_HOST}"
|
||||
echo_stat
|
||||
${SSHPASSWORD} rsync -e "ssh -p ${SSH_PORT} -i $HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST}:${PIHOLE_DIR}/${CUSTOM_DNS} $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${CUSTOM_DNS}.pull >/dev/null 2>&1
|
||||
error_validate
|
||||
|
||||
MESSAGE="Replacing ${CUSTOM_DNS} on $HOSTNAME"
|
||||
echo_stat
|
||||
sudo cp $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${CUSTOM_DNS}.pull ${PIHOLE_DIR}/${CUSTOM_DNS} >/dev/null 2>&1
|
||||
error_validate
|
||||
|
||||
MESSAGE="Validating Ownership on ${CUSTOM_DNS}"
|
||||
echo_stat
|
||||
|
||||
CUSTOMLS_OWN=$(ls -ld ${PIHOLE_DIR}/${CUSTOM_DNS} | awk '{print $3 $4}')
|
||||
if [ $CUSTOMLS_OWN == "rootroot" ]
|
||||
then
|
||||
echo_good
|
||||
else
|
||||
echo_fail
|
||||
|
||||
MESSAGE="Attempting to Compensate"
|
||||
echo_info
|
||||
|
||||
MESSAGE="Setting Ownership on ${CUSTOM_DNS}"
|
||||
echo_stat
|
||||
sudo chown root:root ${PIHOLE_DIR}/${CUSTOM_DNS} >/dev/null 2>&1
|
||||
error_validate
|
||||
fi
|
||||
|
||||
MESSAGE="Validating Permissions on ${CUSTOM_DNS}"
|
||||
echo_stat
|
||||
|
||||
CUSTOMLS_RWE=$(namei -m ${PIHOLE_DIR}/${CUSTOM_DNS} | grep -v f: | grep ${CUSTOM_DNS} | awk '{print $1}')
|
||||
if [ $CUSTOMLS_RWE == "-rw-r--r--" ]
|
||||
then
|
||||
echo_good
|
||||
else
|
||||
echo_fail
|
||||
|
||||
MESSAGE="Attempting to Compensate"
|
||||
echo_info
|
||||
|
||||
MESSAGE="Setting Ownership on ${CUSTOM_DNS}"
|
||||
echo_stat
|
||||
sudo chmod 644 ${PIHOLE_DIR}/${CUSTOM_DNS} >/dev/null 2>&1
|
||||
error_validate
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
MESSAGE="Inverting Tachyon Pulse"
|
||||
echo_info
|
||||
sleep 1
|
||||
@ -217,6 +279,32 @@ function push_gs {
|
||||
${SSHPASSWORD} ssh -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "sudo chown pihole:pihole ${PIHOLE_DIR}/${GRAVITY_FI}" >/dev/null 2>&1
|
||||
error_validate
|
||||
|
||||
if [ "$SKIP_CUSTOM" != '1' ]
|
||||
then
|
||||
if [ "$REMOTE_CUSTOM_DNS" == "1" ]
|
||||
then
|
||||
MESSAGE="Backing Up ${CUSTOM_DNS} from ${REMOTE_HOST}"
|
||||
echo_stat
|
||||
${SSHPASSWORD} rsync -e "ssh -p ${SSH_PORT} -i $HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST}:${PIHOLE_DIR}/${CUSTOM_DNS} $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${CUSTOM_DNS}.push >/dev/null 2>&1
|
||||
error_validate
|
||||
|
||||
MESSAGE="Pushing ${CUSTOM_DNS} to ${REMOTE_HOST}"
|
||||
echo_stat
|
||||
${SSHPASSWORD} rsync --rsync-path="sudo rsync" -e "ssh -p ${SSH_PORT} -i $HOME/${SSH_PKIF}" ${PIHOLE_DIR}/${CUSTOM_DNS} ${REMOTE_USER}@${REMOTE_HOST}:${PIHOLE_DIR}/${CUSTOM_DNS} >/dev/null 2>&1
|
||||
error_validate
|
||||
|
||||
MESSAGE="Setting Permissions on ${CUSTOM_DNS}"
|
||||
echo_stat
|
||||
${SSHPASSWORD} ssh -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "sudo chmod 644 ${PIHOLE_DIR}/${CUSTOM_DNS}" >/dev/null 2>&1
|
||||
error_validate
|
||||
|
||||
MESSAGE="Setting Ownership on ${CUSTOM_DNS}"
|
||||
echo_stat
|
||||
${SSHPASSWORD} ssh -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "sudo chown root:root ${PIHOLE_DIR}/${CUSTOM_DNS}" >/dev/null 2>&1
|
||||
error_validate
|
||||
fi
|
||||
fi
|
||||
|
||||
MESSAGE="Contacting Borg Collective"
|
||||
echo_info
|
||||
sleep 1
|
||||
@ -233,6 +321,7 @@ function push_gs {
|
||||
|
||||
logs_export
|
||||
exit_withchange
|
||||
|
||||
}
|
||||
|
||||
function restore_gs {
|
||||
@ -283,7 +372,56 @@ function restore_gs {
|
||||
sudo chmod 664 ${PIHOLE_DIR}/${GRAVITY_FI} >/dev/null 2>&1
|
||||
error_validate
|
||||
fi
|
||||
|
||||
|
||||
if [ "$SKIP_CUSTOM" != '1' ]
|
||||
then
|
||||
if [ "$REMOTE_CUSTOM_DNS" == "1" ]
|
||||
then
|
||||
MESSAGE="Restoring ${CUSTOM_DNS} on $HOSTNAME"
|
||||
echo_stat
|
||||
cp $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${CUSTOM_DNS}.backup ${PIHOLE_DIR}/${CUSTOM_DNS} >/dev/null 2>&1
|
||||
error_validate
|
||||
|
||||
MESSAGE="Validating Ownership on ${CUSTOM_DNS}"
|
||||
echo_stat
|
||||
|
||||
CUSTOMLS_OWN=$(ls -ld ${PIHOLE_DIR}/${CUSTOM_DNS} | awk '{print $3 $4}')
|
||||
if [ $CUSTOMLS_OWN == "rootroot" ]
|
||||
then
|
||||
echo_good
|
||||
else
|
||||
echo_fail
|
||||
|
||||
MESSAGE="Attempting to Compensate"
|
||||
echo_info
|
||||
|
||||
MESSAGE="Setting Ownership on ${CUSTOM_DNS}"
|
||||
echo_stat
|
||||
sudo chown root:root ${PIHOLE_DIR}/${CUSTOM_DNS} >/dev/null 2>&1
|
||||
error_validate
|
||||
fi
|
||||
|
||||
MESSAGE="Validating Permissions on ${CUSTOM_DNS}"
|
||||
echo_stat
|
||||
|
||||
CUSTOMLS_RWE=$(namei -m ${PIHOLE_DIR}/${CUSTOM_DNS} | grep -v f: | grep ${CUSTOM_DNS} | awk '{print $1}')
|
||||
if [ $CUSTOMLS_RWE == "-rw-r--r--" ]
|
||||
then
|
||||
echo_good
|
||||
else
|
||||
echo_fail
|
||||
|
||||
MESSAGE="Attempting to Compensate"
|
||||
echo_info
|
||||
|
||||
MESSAGE="Setting Ownership on ${CUSTOM_DNS}"
|
||||
echo_stat
|
||||
sudo chmod 644 ${PIHOLE_DIR}/${CUSTOM_DNS} >/dev/null 2>&1
|
||||
error_validate
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
MESSAGE="Evacuating Saucer Section"
|
||||
echo_info
|
||||
sleep 1
|
||||
@ -324,6 +462,8 @@ function logs_gs {
|
||||
tail -n 7 "${LOG_PATH}/${SYNCING_LOG}" | grep PULL
|
||||
echo -e "Recent Complete ${YELLOW}PUSH${NC} Executions"
|
||||
tail -n 7 "${LOG_PATH}/${SYNCING_LOG}" | grep PUSH
|
||||
echo -e "Recent Complete ${YELLOW}RESTORE${NC} Executions"
|
||||
tail -n 7 "${LOG_PATH}/${SYNCING_LOG}" | grep RESTORE
|
||||
echo -e "========================================================"
|
||||
|
||||
exit_nochange
|
||||
@ -456,24 +596,79 @@ function md5_compare {
|
||||
MESSAGE="Comparing ${GRAVITY_FI} Changes"
|
||||
echo_info
|
||||
|
||||
HASHMARK='0'
|
||||
|
||||
MESSAGE="Analyzing Remote ${GRAVITY_FI}"
|
||||
echo_stat
|
||||
primaryMD5=$(${SSHPASSWORD} ssh -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "md5sum ${PIHOLE_DIR}/${GRAVITY_FI}")
|
||||
primaryDBMD5=$(${SSHPASSWORD} ssh -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "md5sum ${PIHOLE_DIR}/${GRAVITY_FI}")
|
||||
error_validate
|
||||
|
||||
MESSAGE="Analyzing Local ${GRAVITY_FI}"
|
||||
echo_stat
|
||||
secondMD5=$(md5sum ${PIHOLE_DIR}/${GRAVITY_FI})
|
||||
secondDBMD5=$(md5sum ${PIHOLE_DIR}/${GRAVITY_FI})
|
||||
error_validate
|
||||
|
||||
if [ "$primaryMD5" == "$secondMD5" ]
|
||||
if [ "$primaryDBMD5" == "$secondDBMD5" ]
|
||||
then
|
||||
MESSAGE="No Differences in ${GRAVITY_FI}"
|
||||
echo_info
|
||||
exit_nochange
|
||||
HASHMARK=$((HASHMARK+0))
|
||||
else
|
||||
MESSAGE="Changes Detected in ${GRAVITY_FI}"
|
||||
echo_info
|
||||
HASHMARK=$((HASHMARK+1))
|
||||
fi
|
||||
|
||||
if [ "$SKIP_CUSTOM" != '1' ]
|
||||
then
|
||||
if [ -f ${PIHOLE_DIR}/${CUSTOM_DNS} ]
|
||||
then
|
||||
MESSAGE="Comparing ${CUSTOM_DNS} Changes"
|
||||
echo_info
|
||||
|
||||
if ${SSHPASSWORD} ssh -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} test -e ${PIHOLE_DIR}/${CUSTOM_DNS}
|
||||
then
|
||||
REMOTE_CUSTOM_DNS="1"
|
||||
MESSAGE="Analyzing Remote ${CUSTOM_DNS}"
|
||||
echo_stat
|
||||
|
||||
primaryCLMD5=$(${SSHPASSWORD} ssh -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "md5sum ${PIHOLE_DIR}/${CUSTOM_DNS}")
|
||||
error_validate
|
||||
|
||||
MESSAGE="Analyzing Local ${CUSTOM_DNS}"
|
||||
echo_stat
|
||||
secondCLMD5=$(md5sum ${PIHOLE_DIR}/${CUSTOM_DNS})
|
||||
error_validate
|
||||
|
||||
if [ "$primaryCLMD5" == "$secondCLMD5" ]
|
||||
then
|
||||
MESSAGE="No Differences in ${CUSTOM_DNS}"
|
||||
echo_info
|
||||
HASHMARK=$((HASHMARK+0))
|
||||
else
|
||||
MESSAGE="Changes Detected in ${CUSTOM_DNS}"
|
||||
echo_info
|
||||
HASHMARK=$((HASHMARK+1))
|
||||
fi
|
||||
else
|
||||
MESSAGE="No Remote ${CUSTOM_DNS} Detected"
|
||||
echo_info
|
||||
fi
|
||||
else
|
||||
MESSAGE="No Local ${CUSTOM_DNS} Detected"
|
||||
echo_info
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ "$HASHMARK" != "0" ]
|
||||
then
|
||||
MESSAGE="Replication Suggested"
|
||||
echo_info
|
||||
HASHMARK=$((HASHMARK+0))
|
||||
else
|
||||
MESSAGE="No Replication Required"
|
||||
echo_info
|
||||
exit_nochange
|
||||
fi
|
||||
}
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user