diff --git a/ADVANCED.md b/ADVANCED.md index 2b019e4..318f5a6 100644 --- a/ADVANCED.md +++ b/ADVANCED.md @@ -14,8 +14,8 @@ Download the latest release from [GitHub](https://github.com/vmstan/gravity-sync ```bash cd ~ -wget https://github.com/vmstan/gravity-sync/archive/v2.0.2.zip -unzip v2.0.2.zip -d gravity-sync +wget https://github.com/vmstan/gravity-sync/archive/v2.1.0.zip +unzip v2.1.0.zip -d gravity-sync cd gravity-sync ``` @@ -169,6 +169,11 @@ The `./gravity-sync.sh config` function will attempt to ping the remote host to Default setting in Gravity Sync is 0, change to 1 to skip this network test. +#### `BACKUP_RETAIN=''` +The `./gravity-sync.sh backup` function will retain a defined number of days worth of `gravity.db` and `custom.list` backups. + +Default setting in Gravity Sync is 7, adjust as resired. + ## Execution If you are just straight up unable to run the `gravity-sync.sh` file, make sure it's marked as an executable by Linux. diff --git a/CHANGELOG.md b/CHANGELOG.md index 400d621..750e6a7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,24 @@ # The Changelog +## 2.1 +### The Backup Release + +A new function `./gravity-sync.sh backup` will now perform a `SQLITE3` operated backup of the `gravity.db` on the local Pi-hole. This can be run at any time you wish, but can also be automated by the `./gravity-sync.sh automate` function to run once a day. New and existing users will be prompted to configure both during this task. If can also disable both using the automate function, or just automate one or the other, by setting the value to `0` during setup. + +New users will automatically have their local settings backed up after completion of the initial setup, before the first run of any sync tasks. + +By default, 7 days worth of backups will be retained in the `backup` folder. You can adjust the retention length by changing the `BACKUP_RETAIN` function in your `gravity-sync.conf` file. See the `ADVANCED.md` file for more information on setting these custom configuration options. + +There are also enhancements to the `./gravity-sync.sh restore` function, where as previously this task would only restore the previous copy of the database that is made during sync operations, now this will ask you to select a previous backup copy (by date) and will use that file to restore. This will stop the Pi-hole services on the local server while the task is completed. After a successful restoration, you will now also be prompted to perform a `push` operation of the restored database to the primary Pi-hole server. + +It's suggested to make sure your local restore was successful before completing the `restore` operation with the `push` job. + +#### Deprecation + +Support for the the Dropbear SSH client/server (which was added in 1.7.6) will be removed in an upcoming version of Gravity Sync. If you are using this instead of OpenSSH (common with DietPi) please reconfigure your installation to use OpenSSH. You will want to delete your existing `~/.ssh/id_rsa` and `~/.ssh/id_rsa.pub` files and run `./gravity-sync.sh configure` again to generate a new key and copy it to the primary Pi-hole. + +The `./gravity-sync.sh update` and `version` functions will look for the `dbclient` binary on the local system and warn users to change. + ## 2.0 ### The Smart Release diff --git a/README.md b/README.md index 3105c58..49628d8 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ What is better than a [Pi-hole](https://github.com/pi-hole/pi-hole) blocking ads via DNS on your network? That's right, Two Pi-hole! (Redundency is key in any network infrastucture.) But if you have more than one Pi-hole in your network you'll want a simple way to keep the list configurations and local DNS settings identical between the two. That's where Gravity Sync comes in. Gravity Sync will: -- Sync that the allow/blocklist configurations stored in `gravity.db` between two Pi-hole +- Sync the allow/blocklist configurations stored in `gravity.db` between two Pi-hole - Sync the local DNS settings stored in `custom.list` between two Pi-hole. - Provide an easy way to keep this happening in the background. @@ -16,7 +16,10 @@ Gravity Sync will **not**: It is suggested that you use an external DHCP server on your network (such as your router) when using multiple Pi-hole. -### Requirements +### Disclaimer +Gravity Sync is not developed by or affiliated with the Pi-hole project. This is a community effort that seeks to implement replication, which is currently not a part of the core Pi-hole product. The code has been well tested across multiple user environments but there always is an element of risk involved with running any arbitrary software you find on the Internet. + +## Requirements - Pi-hole 5.0 (or higher) be installed on at least two systems, using any of the Linux distros that Pi-hole is [certified to run on](https://docs.pi-hole.net/main/prerequesites/#supported-operating-systems). Docker deployments of Pi-hole are not supported. - You will need to make sure that you have passwordless `SUDO` enabled for the accounts on both the primary and secondary Pi-hole that will be performing the work. Most of the pre-built images available for the Raspberry Pi already have this configured, but if you have your Pi-hole running in a virtual machine built from a generic ISO, you may need to [adjust this manually](https://linuxize.com/post/how-to-run-sudo-command-without-password/). - Make sure `SSH` and `RSYNC` are installed on both the primary and secondary Pi-hole prior to installation. This is what does the leavy lifting between your Pi-hole nodes. OpenSSH is reccomended but if you're using a ultra-lightweight Pi distrbution (such as DietPi) that uses Dropbear by default, it should work as well. Other SSH client/server combonations are not supported at this time. diff --git a/VERSION b/VERSION index f93ea0c..50aea0e 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -2.0.2 \ No newline at end of file +2.1.0 \ No newline at end of file diff --git a/gravity-sync.conf.example b/gravity-sync.conf.example index c125d38..1e55653 100644 --- a/gravity-sync.conf.example +++ b/gravity-sync.conf.example @@ -33,3 +33,5 @@ REMOTE_PASS='' # SKIP_CUSTOM='' # DATE_OUTPUT='' # PING_AVOID='' + +# BACKUP_RETAIN='' \ No newline at end of file diff --git a/gravity-sync.sh b/gravity-sync.sh index 1611444..4cf0328 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -3,7 +3,7 @@ SCRIPT_START=$SECONDS # GRAVITY SYNC BY VMSTAN ##################### PROGRAM='Gravity Sync' -VERSION='2.0.2' +VERSION='2.1.0' # Execute from the home folder of the user who owns it (ex: 'cd ~/gravity-sync') # For documentation or downloading updates visit https://github.com/vmstan/gravity-sync @@ -32,6 +32,9 @@ SKIP_CUSTOM='0' # replace in gravity-sync.conf to overwrite DATE_OUTPUT='0' # replace in gravity-sync.conf to overwrite PING_AVOID='0' # replace in gravity-sync.conf to overwrite +# Backup Customization +BACKUP_RETAIN='7' # replace in gravity-sync.conf to overwrite + # Pi-hole Folder/File Locations PIHOLE_DIR='/etc/pihole' # default Pi-hole data directory GRAVITY_FI='gravity.db' # default Pi-hole database file @@ -526,14 +529,36 @@ function smart_gs { } function restore_gs { - MESSAGE="This will restore ${GRAVITY_FI} on $HOSTNAME with the previous version!" + MESSAGE="This will restore your settings on $HOSTNAME with a previous version!" echo_warn + MESSAGE="PREVIOUS BACKUPS" + echo_info + ls $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD} | grep $(date +%Y) | grep ${GRAVITY_FI} | colrm 18 + + MESSAGE="Enter the date you want to restore from" + echo_need + read INPUT_BACKUP_DATE + + if [ -f $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${INPUT_BACKUP_DATE}-${GRAVITY_FI}.backup ] + then + MESSAGE="Backup File Located" + echo_info + else + MESSAGE="Invalid Requested" + fi + intent_validate + MESSAGE="Stopping Pi-hole Services" + echo_stat + + sudo service pihole-FTL stop >/dev/null 2>&1 + error_validate + MESSAGE="Restoring ${GRAVITY_FI} on $HOSTNAME" echo_stat - sudo cp $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${GRAVITY_FI}.backup ${PIHOLE_DIR}/${GRAVITY_FI} >/dev/null 2>&1 + sudo cp $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${INPUT_BACKUP_DATE}-${GRAVITY_FI}.backup ${PIHOLE_DIR}/${GRAVITY_FI} >/dev/null 2>&1 error_validate MESSAGE="Validating Ownership on ${GRAVITY_FI}" @@ -576,11 +601,11 @@ function restore_gs { if [ "$SKIP_CUSTOM" != '1' ] then - if [ -f $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${CUSTOM_DNS}.backup ] + if [ -f $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${INPUT_BACKUP_DATE}-${CUSTOM_DNS}.backup ] then MESSAGE="Restoring ${CUSTOM_DNS} on $HOSTNAME" echo_stat - sudo cp $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${CUSTOM_DNS}.backup ${PIHOLE_DIR}/${CUSTOM_DNS} >/dev/null 2>&1 + sudo cp $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${INPUT_BACKUP_DATE}-${CUSTOM_DNS}.backup ${PIHOLE_DIR}/${CUSTOM_DNS} >/dev/null 2>&1 error_validate MESSAGE="Validating Ownership on ${CUSTOM_DNS}" @@ -626,19 +651,36 @@ function restore_gs { MESSAGE="Evacuating Saucer Section" echo_info sleep 1 + + MESSAGE="Restarting FTLDNS Services" + echo_stat + sudo service pihole-FTL start >/dev/null 2>&1 + error_validate MESSAGE="Updating FTLDNS Configuration" echo_stat ${PIHOLE_BIN} restartdns reloadlists >/dev/null 2>&1 error_validate - MESSAGE="Reloading FTLDNS Services" - echo_stat - ${PIHOLE_BIN} restartdns >/dev/null 2>&1 - error_validate - - logs_export - exit_withchange + + MESSAGE="Do you want to push the restored configuration to the primary Pi-hole? (yes/no)" + echo_need + read PUSH_TO_PRIMARY + + if [ "${PUSH_TO_PRIMARY}" == "Yes" ] || [ "${PUSH_TO_PRIMARY}" == "yes" ] || [ "${PUSH_TO_PRIMARY}" == "Y" ] || [ "${PUSH_TO_PRIMARY}" == "y" ] + then + push_gs + elif [ "${PUSH_TO_PRIMARY}" == "No" ] || [ "${PUSH_TO_PRIMARY}" == "no" ] || [ "${PUSH_TO_PRIMARY}" == "N" ] || [ "${PUSH_TO_PRIMARY}" == "n" ] + then + logs_export + exit_withchange + else + MESSAGE="Invalid Selection - Defaulting No" + echo_warn + + logs_export + exit_withchange + fi } # Logging Functions @@ -1335,6 +1377,8 @@ function config_generate { validate_os_sshpass detect_remotersync + + task_backup exit_withchange } @@ -1431,13 +1475,23 @@ function show_version { else if [ "$GITVERSION" != "$VERSION" ] then - MESSAGE="Upgrade Available: ${PURPLE}${GITVERSION}${NC}" + MESSAGE="Update Available: ${PURPLE}${GITVERSION}${NC}" else MESSAGE="Latest Version: ${GREEN}${GITVERSION}${NC}" fi fi echo_info echo -e "========================================================" + + dbclient_warning +} + +function dbclient_warning { + if hash dbclient 2>/dev/null + then + MESSAGE="Dropbear support has been deprecated - please convert to OpenSSH" + echo_warn + fi } # Task Stack @@ -1458,6 +1512,9 @@ function task_automate { CRON_EXIST='1' fi + MESSAGE="Configuring Hourly Smart Sync" + echo_info + MESSAGE="Sync Frequency in Minutes (1-30) or 0 to Disable" echo_need read INPUT_AUTO_FREQ @@ -1473,12 +1530,11 @@ function task_automate { then clear_cron - MESSAGE="Automation Disabled" - echo_info + MESSAGE="Sync Automation Disabled" + echo_warn else - MESSAGE="No Automation Scheduled" - echo_info - exit_nochange + MESSAGE="No Sync Automation Scheduled" + echo_warn fi else if [ $CRON_EXIST == 1 ] @@ -1486,11 +1542,35 @@ function task_automate { clear_cron fi - MESSAGE="Saving New Automation" + MESSAGE="Saving New Sync Automation" echo_stat - (crontab -l 2>/dev/null; echo "*/${INPUT_AUTO_FREQ} * * * * ${BASH_PATH} $HOME/${LOCAL_FOLDR}/${GS_FILENAME} > ${LOG_PATH}/${CRONJOB_LOG}") | crontab - + (crontab -l 2>/dev/null; echo "*/${INPUT_AUTO_FREQ} * * * * ${BASH_PATH} $HOME/${LOCAL_FOLDR}/${GS_FILENAME} smart > ${LOG_PATH}/${CRONJOB_LOG}") | crontab - error_validate fi + + MESSAGE="Configuring Daily Backup Frequency" + echo_info + + MESSAGE="Hour of Day to Backup (1-24) or 0 to Disable" + echo_need + read INPUT_AUTO_BACKUP + + if [ $INPUT_AUTO_BACKUP -gt 24 ] + then + MESSAGE="Invalid Frequency Range" + echo_fail + exit_nochange + elif [ $INPUT_AUTO_BACKUP -lt 1 ] + then + MESSAGE="No Backup Automation Scheduled" + echo_warn + else + MESSAGE="Saving New Backup Automation" + echo_stat + (crontab -l 2>/dev/null; echo "* ${INPUT_AUTO_BACKUP} * * * ${BASH_PATH} $HOME/${LOCAL_FOLDR}/${GS_FILENAME} backup >/dev/null 2>&1") | crontab - + error_validate + fi + exit_withchange } @@ -1554,6 +1634,8 @@ function task_update { TASKTYPE='UPDATE' MESSAGE="${MESSAGE}: ${TASKTYPE} Requested" echo_good + + dbclient_warning update_gs } @@ -1605,6 +1687,35 @@ function task_invalid { list_gs_arguments } +## Backup Task +function task_backup { + TASKTYPE='BACKUP' + MESSAGE="${MESSAGE}: ${TASKTYPE} Requested" + echo_good + + BACKUPTIMESTAMP=$(date +%F-%H%M%S) + + MESSAGE="Performing Backup of ${GRAVITY_FI}" + echo_stat + + sqlite3 ${PIHOLE_DIR}/${GRAVITY_FI} ".backup '$HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${BACKUPTIMESTAMP}-${GRAVITY_FI}.backup'" + error_validate + + MESSAGE="Performing Backup Up ${CUSTOM_DNS}" + echo_stat + + cp ${PIHOLE_DIR}/${CUSTOM_DNS} $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${BACKUPTIMESTAMP}-${CUSTOM_DNS}.backup + error_validate + + MESSAGE="Cleaning Up Old Backups" + echo_stat + + find $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/$(date +%Y)*.backup -mtime +${BACKUP_RETAIN} -type f -delete + error_validate + + exit_withchange +} + # Echo Stack ## Informative function echo_info { @@ -1800,6 +1911,10 @@ case $# in task_automate ;; + backup) + task_backup + ;; + *) task_invalid ;;