From 0bc11572d7a386b03c6c395aab61bb0429076076 Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 08:44:31 -0500 Subject: [PATCH 01/35] header test --- ADVANCED.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ADVANCED.md b/ADVANCED.md index bb73990..910764b 100644 --- a/ADVANCED.md +++ b/ADVANCED.md @@ -76,7 +76,7 @@ Save. Keep calm, carry on. ### Hidden Figures There are a series of advanced configuration options that you may need to change to better adapt Gravity Sync to your environment. They are referenced at the end of the `gravity-sync.conf` file. It is suggested that you make any necessary variable changes to this file, as they will superceed the ones located in the core script. If you want to revert back to the Gravity Sync default for any of these settings, just apply a `#` to the beginning of the line to comment it out. -#### SSH_PORT='' +#### `SSH_PORT=''` Gravity Sync is configured by default to use the standard SSH port (22) but if you need to change this, such as if you're traversing a NAT/firewall for a multi-site deployment, to use a non-standard port. Default setting in Gravity Sync is 22. From a3c66affbe19c49f8622c9cd11275d3930375442 Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 09:30:53 -0500 Subject: [PATCH 02/35] Update Troubleshooting --- ADVANCED.md | 26 ++++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-) diff --git a/ADVANCED.md b/ADVANCED.md index 910764b..a5bae85 100644 --- a/ADVANCED.md +++ b/ADVANCED.md @@ -81,46 +81,46 @@ Gravity Sync is configured by default to use the standard SSH port (22) but if y Default setting in Gravity Sync is 22. -#### SSH_PKIF='' +#### `SSH_PKIF=''` Gravity Sync is configured by default to use the `.ssh/id_rsa` keyfile that is generated using the `ssh-keygen` command. If you have an existing keyfile stored somewhere else that you'd like to use, you can configure that here. The keyfile will still need to be in the users $HOME directory. At this time Gravity Sync does not support passphrases in RSA keyfiles. If you have a passphrase applied to your standard `.ssh/id_rsa` either remove it, or generate a new file and specify that key for use only by Gravity Sync. Default setting for Gravity Sync is `.ssh/id_rsa` -#### LOG_PATH='' +#### `LOG_PATH=''` Gravity Sync will place logs in the same folder as the script (identified as .cron and .log) but if you'd like to place these in a another location, you can do that by identifying the full path to the directory. (ex: `/full/path/to/logs`) Default setting for Gravity Sync is `$HOME/${LOCAL_FOLDR}` -#### SYNCING_LOG='' +#### `SYNCING_LOG=''` Gravity Sync will write a timestamp for any completed pull, push or restore job to this file. If you want to change the name of this file, you will also need to adjust the LOG_PATH variable above, otherwise your file will be remove during `update` operations. Default setting for Gravity Sync is `gravity-sync.log` -#### CRONJOB_LOG='' +#### `CRONJOB_LOG=''` Gravity Sync will log the execution history of the previous automation task via Cron to this file. If you want to change the name of this file, you will also need to adjust the LOG_PATH variable above, otherwise your file will be remove during `update` operations. This will have an impact to both the `./gravity-sync.sh automate` function and the `./gravity-sync.sh cron` functions. If you need to change this after running the automate function, either modify your crontab manually or delete the entry and re-run the automate function. Default setting for Gravity Sync is `gravity-sync.cron` -#### VERIFY_PASS='' +#### `VERIFY_PASS=''` Gravity Sync will prompt to verify user interactivity during push, restore, or config operations (that overwrite an existing configuration) with the intention that it prevents someone from accidentally automating in the wrong direction or overwriting data intentionally. If you'd like to automate a push function, or just don't like to be asked twice to do something distructive, then you can opt-out. Default setting in Gravity Sync is 0, change to 1 to bypass this check. -#### SKIP_CUSTOM='' +#### `SKIP_CUSTOM=''` Starting in v1.7.0, Gravity Sync manages the `custom.list` file that contains the "Local DNS Records" function within the Pi-hole interface. If you do not want to sync this setting, perhaps if you're doing a mutli-site deployment with differing local DNS settings, then you can opt-out of this sync. Default setting in Gravity Sync is 0, change to 1 to exempt `custom.list` from replication. -#### DATE_OUTPUT='' +#### `DATE_OUTPUT=''` *This feature has not been fully implemented, but the intent is to provide the ability to add timestamped output to each status indicator in the script output (ex: [2020-05-28 19:46:54] [EXEC] $MESSAGE).* Default setting in Gravity Sync is 0, change to 1 to print timestamped output. -#### BASH_PATH='' +#### `BASH_PATH=''` If you need to adjust the path to bash that is identified for automated execution via Crontab, you can do that here. This will only have an impact if changed before generating the crontab via the `./gravity-sync.sh automate` function. If you need to change this after the fact, either modify your crontab manually or delete the entry and re-run the automate function. ## Updates @@ -141,6 +141,16 @@ Delete the `dev` file and update again to revert back to the stable/master branc This method for implementation is decidedly different than the configuration flags in the .conf file, as explained above, to make it easier to identify development systems. +### Updater Troubleshooting +If the built in updater doesn't function as expected, you can manually run the git commands that operate under the covers. +``` +git fetch --all +git reset --hard origin/master +``` +Use `origin/development` to pull that branch instead. + +If your code is still not updating after this, reinstallation is suggested rather than spending all your time troubleshooting `git` commands. + ## Automation There are many automation methods available to run scripts on a regular basis of a Linux system. The one built into all of them is cron, but if you'd like to utilize something different then the principles are still the same. From 0928a7a0be336349aca1e5dfe91bf46feb9a2ae9 Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 10:00:01 -0500 Subject: [PATCH 03/35] Testing for dbclient --- ADVANCED.md | 21 +++++++++---------- README.md | 12 ++++++++++- gravity-sync.sh | 54 ++++++++++++++++++++++++++----------------------- 3 files changed, 49 insertions(+), 38 deletions(-) diff --git a/ADVANCED.md b/ADVANCED.md index a5bae85..963cd7f 100644 --- a/ADVANCED.md +++ b/ADVANCED.md @@ -123,6 +123,14 @@ Default setting in Gravity Sync is 0, change to 1 to print timestamped output. #### `BASH_PATH=''` If you need to adjust the path to bash that is identified for automated execution via Crontab, you can do that here. This will only have an impact if changed before generating the crontab via the `./gravity-sync.sh automate` function. If you need to change this after the fact, either modify your crontab manually or delete the entry and re-run the automate function. +## Execution +If you are just straight up unable to run the `gravity-sync.sh` file, make sure it's marked as an executable by Linux. + +```bash +chmod +x gravity-sync.sh +``` + + ## Updates If you manually installed Gravity Sync via .zip or .tar.gz you will need to download and overwrite the `gravity-sync.sh` file with a newer version. If you've chosen this path, I won't lay out exactly what you'll need to do every time, but you should at least review the contents of the script bundle (specifically the example configuration file) to make sure there are no new additional files or required settings. @@ -162,18 +170,7 @@ crontab -e ``` ## Troubleshooting -If you are just straight up unable to run the `gravity-sync.sh` file, make sure it's marked as an executable by Linux. - -```bash -chmod +x gravity-sync.sh -``` - -If you are getting errors about missing SSH or RSYNC when you run your first `compare` or `pull` operation, and you're using an ultra-lightweight distro like DietPi, make sure they are installed on the base operating system. - -- If your script prompts for a password on the remote system, make sure that your user account is setup not to require passwords in the sudoers file. -- If you use a non-standard SSH port to connect to your primary Pi-hole, you can add `SSH_PORT='123'` to the bottom of your `gravity-sync.conf` file. (Subsitute 123 for your non-standard port.) This will overwrite the `SSH_PORT=22` at the top of the script as it is imported later in the execution. -- If you'd like to know what version of the script you have running by running `./gravity-sync.sh version` -- If the update script fails, make sure you did your original deployment via `git clone` and not a manual install. + - If it doesn't kick off, you can manually execute a `git pull` while in the `gravity-sync` directory. If all else fails, delete the entire `gravity-sync` folder from your system and re-deploy. This will have no impact on your replicated databases. diff --git a/README.md b/README.md index 4c33498..4670bb8 100644 --- a/README.md +++ b/README.md @@ -86,6 +86,8 @@ Before executing, this will make a copy of the remote database under `backup/gra This function purposefuly asks for user interaction to avoid being accidentally automated. +- If your script prompts for a password on the remote system, make sure that your remote user account is setup not to require passwords in the sudoers file. + ### The Restore Function Graviy Sync can also `restore` the database on the secondary Pi-hole in the event you've overwritten it accidentally. This might happen in the above scenario where you've had your primary Pi-hole down for an extended period, made changes to the secondary, but perhaps didn't get a chance to perform a `push` of the changes back to the primary, before your automated sync ran. @@ -98,7 +100,13 @@ This will copy your last `gravity.db.backup` and `custom.list.backup` to the ru This function purposefuly asks for user interaction to avoid being accidentally automated. ## Updates -You can run the built-in updater to get the latest version of all the files. +If you'd like to know what version of the script you have running the built in version checker. It will notify you if there are updates available. + + ``` + ./gravity-sync.sh version + ``` + +You can then run the built-in updater to get the latest version of all the files. Both the `version` and `update` commands reach out to GitHub, so outbound access to github.com is required. ```bash ./gravity-sync.sh update @@ -108,6 +116,8 @@ Your copy of the `gravity-sync.conf` file, logs and backups should not be be imp You can run a `./gravity-sync.sh config` at any time to generate a new configuration file if you're concerned that you're missing something. +- If the update script fails, make sure you did your original deployment via `git clone` and not a manual install. Refer to [ADVANCED.md](https://github.com/vmstan/gravity-sync/blob/master/ADVANCED.md) for more details. + ## Automation Automation of sync is accomplished by adding an execution of the script to the user's crontab file. As Gravity Sync won't make any changes if it doesn't detect a difference to sync, then the impact should be minor to your systems. diff --git a/gravity-sync.sh b/gravity-sync.sh index 595454f..0d31586 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -2,7 +2,7 @@ # GRAVITY SYNC BY VMSTAN ##################### PROGRAM='Gravity Sync' -VERSION='1.7.5' +VERSION='1.7.6' # Execute from the home folder of the user who owns it (ex: 'cd ~/gravity-sync') # For documentation or downloading updates visit https://github.com/vmstan/gravity-sync @@ -142,7 +142,7 @@ function pull_gs { MESSAGE="Pulling ${GRAVITY_FI} from ${REMOTE_HOST}" echo_stat - ${SSHPASSWORD} rsync -e "ssh -p ${SSH_PORT} -i $HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST}:${PIHOLE_DIR}/${GRAVITY_FI} $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${GRAVITY_FI}.pull >/dev/null 2>&1 + ${SSHPASSWORD} rsync -e "${SSH_CMD} -p ${SSH_PORT} -i $HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST}:${PIHOLE_DIR}/${GRAVITY_FI} $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${GRAVITY_FI}.pull >/dev/null 2>&1 error_validate MESSAGE="Replacing ${GRAVITY_FI} on $HOSTNAME" @@ -199,7 +199,7 @@ function pull_gs { MESSAGE="Pulling ${CUSTOM_DNS} from ${REMOTE_HOST}" echo_stat - ${SSHPASSWORD} rsync -e "ssh -p ${SSH_PORT} -i $HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST}:${PIHOLE_DIR}/${CUSTOM_DNS} $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${CUSTOM_DNS}.pull >/dev/null 2>&1 + ${SSHPASSWORD} rsync -e "${SSH_CMD} -p ${SSH_PORT} -i $HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST}:${PIHOLE_DIR}/${CUSTOM_DNS} $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${CUSTOM_DNS}.pull >/dev/null 2>&1 error_validate MESSAGE="Replacing ${CUSTOM_DNS} on $HOSTNAME" @@ -273,22 +273,22 @@ function push_gs { MESSAGE="Backing Up ${GRAVITY_FI} from ${REMOTE_HOST}" echo_stat - ${SSHPASSWORD} rsync -e "ssh -p ${SSH_PORT} -i $HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST}:${PIHOLE_DIR}/${GRAVITY_FI} $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${GRAVITY_FI}.push >/dev/null 2>&1 + ${SSHPASSWORD} rsync -e "${SSH_CMD} -p ${SSH_PORT} -i $HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST}:${PIHOLE_DIR}/${GRAVITY_FI} $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${GRAVITY_FI}.push >/dev/null 2>&1 error_validate MESSAGE="Pushing ${GRAVITY_FI} to ${REMOTE_HOST}" echo_stat - ${SSHPASSWORD} rsync --rsync-path="sudo rsync" -e "ssh -p ${SSH_PORT} -i $HOME/${SSH_PKIF}" ${PIHOLE_DIR}/${GRAVITY_FI} ${REMOTE_USER}@${REMOTE_HOST}:${PIHOLE_DIR}/${GRAVITY_FI} >/dev/null 2>&1 + ${SSHPASSWORD} rsync --rsync-path="sudo rsync" -e "${SSH_CMD} -p ${SSH_PORT} -i $HOME/${SSH_PKIF}" ${PIHOLE_DIR}/${GRAVITY_FI} ${REMOTE_USER}@${REMOTE_HOST}:${PIHOLE_DIR}/${GRAVITY_FI} >/dev/null 2>&1 error_validate MESSAGE="Setting Permissions on ${GRAVITY_FI}" echo_stat - ${SSHPASSWORD} ssh -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "sudo chmod 664 ${PIHOLE_DIR}/${GRAVITY_FI}" >/dev/null 2>&1 + ${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "sudo chmod 664 ${PIHOLE_DIR}/${GRAVITY_FI}" >/dev/null 2>&1 error_validate MESSAGE="Setting Ownership on ${GRAVITY_FI}" echo_stat - ${SSHPASSWORD} ssh -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "sudo chown pihole:pihole ${PIHOLE_DIR}/${GRAVITY_FI}" >/dev/null 2>&1 + ${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "sudo chown pihole:pihole ${PIHOLE_DIR}/${GRAVITY_FI}" >/dev/null 2>&1 error_validate if [ "$SKIP_CUSTOM" != '1' ] @@ -297,22 +297,22 @@ function push_gs { then MESSAGE="Backing Up ${CUSTOM_DNS} from ${REMOTE_HOST}" echo_stat - ${SSHPASSWORD} rsync -e "ssh -p ${SSH_PORT} -i $HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST}:${PIHOLE_DIR}/${CUSTOM_DNS} $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${CUSTOM_DNS}.push >/dev/null 2>&1 + ${SSHPASSWORD} rsync -e "${SSH_CMD} -p ${SSH_PORT} -i $HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST}:${PIHOLE_DIR}/${CUSTOM_DNS} $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${CUSTOM_DNS}.push >/dev/null 2>&1 error_validate MESSAGE="Pushing ${CUSTOM_DNS} to ${REMOTE_HOST}" echo_stat - ${SSHPASSWORD} rsync --rsync-path="sudo rsync" -e "ssh -p ${SSH_PORT} -i $HOME/${SSH_PKIF}" ${PIHOLE_DIR}/${CUSTOM_DNS} ${REMOTE_USER}@${REMOTE_HOST}:${PIHOLE_DIR}/${CUSTOM_DNS} >/dev/null 2>&1 + ${SSHPASSWORD} rsync --rsync-path="sudo rsync" -e "${SSH_CMD} -p ${SSH_PORT} -i $HOME/${SSH_PKIF}" ${PIHOLE_DIR}/${CUSTOM_DNS} ${REMOTE_USER}@${REMOTE_HOST}:${PIHOLE_DIR}/${CUSTOM_DNS} >/dev/null 2>&1 error_validate MESSAGE="Setting Permissions on ${CUSTOM_DNS}" echo_stat - ${SSHPASSWORD} ssh -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "sudo chmod 644 ${PIHOLE_DIR}/${CUSTOM_DNS}" >/dev/null 2>&1 + ${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "sudo chmod 644 ${PIHOLE_DIR}/${CUSTOM_DNS}" >/dev/null 2>&1 error_validate MESSAGE="Setting Ownership on ${CUSTOM_DNS}" echo_stat - ${SSHPASSWORD} ssh -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "sudo chown root:root ${PIHOLE_DIR}/${CUSTOM_DNS}" >/dev/null 2>&1 + ${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "sudo chown root:root ${PIHOLE_DIR}/${CUSTOM_DNS}" >/dev/null 2>&1 error_validate fi fi @@ -323,12 +323,12 @@ function push_gs { MESSAGE="Updating FTLDNS Configuration" echo_stat - ${SSHPASSWORD} ssh -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "${PIHOLE_BIN} restartdns reloadlists" >/dev/null 2>&1 + ${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "${PIHOLE_BIN} restartdns reloadlists" >/dev/null 2>&1 error_validate MESSAGE="Reloading FTLDNS Services" echo_stat - ${SSHPASSWORD} ssh -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "${PIHOLE_BIN} restartdns" >/dev/null 2>&1 + ${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "${PIHOLE_BIN} restartdns" >/dev/null 2>&1 error_validate logs_export @@ -589,7 +589,7 @@ function validate_os_sshpass { MESSAGE="Validating SSH Connection to ${REMOTE_HOST}" echo_stat - timeout 5 ${SSHPASSWORD} ssh -p ${SSH_PORT} -i '$HOME/${SSH_PKIF}' -o StrictHostKeyChecking=no ${REMOTE_USER}@${REMOTE_HOST} 'exit' >/dev/null 2>&1 + timeout 5 ${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i '$HOME/${SSH_PKIF}' -o StrictHostKeyChecking=no ${REMOTE_USER}@${REMOTE_HOST} 'exit' >/dev/null 2>&1 error_validate } @@ -651,23 +651,27 @@ function distro_check { ## Detect SSH & RSYNC function detect_ssh { - MESSAGE="Validating SSH Client on $HOSTNAME" + MESSAGE="Validating SSH Command on $HOSTNAME" echo_stat if hash ssh 2>/dev/null then echo_good + SSH_CMD='ssh' else echo_fail MESSAGE="Attempting to Compensate" echo_info - - distro_check - - MESSAGE="Installing SSH on $HOSTNAME" - echo_stat - ${PKG_INSTALL} ssh >/dev/null 2>&1 - error_validate + if hash dbclient > 2>/dev/null + then + SSH_CMD='dbclient' + MESSAGE="Using Dropbear Instead" + echo_info + else + MESSAGE="No Alternative Detected" + echo_info + exit_nochange + fi fi MESSAGE="Validating RSYNC Client on $HOSTNAME" @@ -709,7 +713,7 @@ function md5_compare { MESSAGE="Analyzing ${REMOTE_HOST} ${GRAVITY_FI}" echo_stat - primaryDBMD5=$(${SSHPASSWORD} ssh -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "md5sum ${PIHOLE_DIR}/${GRAVITY_FI}") + primaryDBMD5=$(${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "md5sum ${PIHOLE_DIR}/${GRAVITY_FI}") error_validate MESSAGE="Analyzing $HOSTNAME ${GRAVITY_FI}" @@ -735,13 +739,13 @@ function md5_compare { # MESSAGE="Comparing ${CUSTOM_DNS} Changes" # echo_info - if ${SSHPASSWORD} ssh -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} test -e ${PIHOLE_DIR}/${CUSTOM_DNS} + if ${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} test -e ${PIHOLE_DIR}/${CUSTOM_DNS} then REMOTE_CUSTOM_DNS="1" MESSAGE="Analyzing ${REMOTE_HOST} ${CUSTOM_DNS}" echo_stat - primaryCLMD5=$(${SSHPASSWORD} ssh -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "md5sum ${PIHOLE_DIR}/${CUSTOM_DNS}") + primaryCLMD5=$(${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "md5sum ${PIHOLE_DIR}/${CUSTOM_DNS}") error_validate MESSAGE="Analyzing $HOSTNAME ${CUSTOM_DNS}" From 62fc23220500b500e5a834097f3e0b636ef2fe05 Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 10:13:01 -0500 Subject: [PATCH 04/35] Attempt to install ssh-client if not found --- gravity-sync.sh | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/gravity-sync.sh b/gravity-sync.sh index 0d31586..8ec52c8 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -668,9 +668,10 @@ function detect_ssh { MESSAGE="Using Dropbear Instead" echo_info else - MESSAGE="No Alternative Detected" - echo_info - exit_nochange + MESSAGE="Installing SSH Client with ${PKG_MANAGER}" + echo_stat + ${PKG_INSTALL} ssh-client >/dev/null 2>&1 + error_validate fi fi @@ -688,6 +689,9 @@ function detect_ssh { distro_check MESSAGE="Attempting to Compensate" + echo_info + + MESSAGE="Installing RSYNC with ${PKG_MANAGER}" echo_stat ${PKG_INSTALL} rsync >/dev/null 2>&1 error_validate From e1180e7395ddf644d05dfdbbcfe9d711d96746cc Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 10:16:36 -0500 Subject: [PATCH 05/35] Removing a > --- CHANGELOG.md | 4 ++++ gravity-sync.sh | 2 +- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index aab5319..f900441 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,10 @@ **Known Issues** - No new Star Trek references. +#### 1.7.6 +- Detects `dbclient` install as alternative to OpenSSH Client. (DietPi) +- Attempts to install OpenSSH Client if not found, and Dropbear is not alternative. (DietPi) + #### 1.7.5 - No code changes! - Primary README now only reflect "The Easy Way" to install and configure Gravity Sync diff --git a/gravity-sync.sh b/gravity-sync.sh index 8ec52c8..88ec801 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -662,7 +662,7 @@ function detect_ssh { echo_fail MESSAGE="Attempting to Compensate" echo_info - if hash dbclient > 2>/dev/null + if hash dbclient 2>/dev/null then SSH_CMD='dbclient' MESSAGE="Using Dropbear Instead" From 34d43eb2386606b0012b4935466b44fbb7f2d4bd Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 10:26:45 -0500 Subject: [PATCH 06/35] Detect .ssh folder and create if needed --- gravity-sync.sh | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/gravity-sync.sh b/gravity-sync.sh index 88ec801..dae84bd 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -613,8 +613,13 @@ function detect_sshkeygen { then MESSAGE="Using DROPBEARKEY Instead" echo_info - KEYGEN_COMMAND="dropbearkey -t rsa -f" - + if [ -d $HOME/.ssh ] + then + KEYGEN_COMMAND="dropbearkey -t rsa -f" + else + mkdir $HOME/.ssh >/dev/null 2>&1 + KEYGEN_COMMAND="dropbearkey -t rsa -f" + fi else MESSAGE="No Alternatives Located" echo_info From d8ca2380ce4c46133f6c38581003b98889432562 Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 10:43:55 -0500 Subject: [PATCH 07/35] Use dropbearkey to send keyfile is needed --- CHANGELOG.md | 3 ++- gravity-sync.sh | 8 +++++++- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f900441..5d52c99 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,6 +14,7 @@ #### 1.7.6 - Detects `dbclient` install as alternative to OpenSSH Client. (DietPi) - Attempts to install OpenSSH Client if not found, and Dropbear is not alternative. (DietPi) +- Fix bug with `dropbearkey` not finding .ssh folder #### 1.7.5 - No code changes! @@ -31,7 +32,7 @@ - Fixes `custom.list` not being processed by `./gravity-sync.sh restore` function. - Detects absence of `ssh` client command on host OS (DietPi) - Detects absence of `rsync` client command on host OS (DietPi) -- Detects absence of `ssh-keygen` utility on host OS and will use dropbearkey as an alternative (DietPi) +- Detects absence of `ssh-keygen` utility on host OS and will use `dropbearkey` as an alternative (DietPi) - Changelog polarity reversed after heated discussions with marketing team. #### 1.7.3 diff --git a/gravity-sync.sh b/gravity-sync.sh index dae84bd..a96e04c 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -929,7 +929,13 @@ function config_generate { echo -e "========================================================" echo -e "========================================================" - ssh-copy-id -f -i $HOME/${SSH_PKIF}.pub ${REMOTE_USER}@${REMOTE_HOST} + if hash ssh-copy-id 2>/dev/null + then + ssh-copy-id -f -i $HOME/${SSH_PKIF}.pub ${REMOTE_USER}@${REMOTE_HOST} + elseif hash dropbearkey 2>/dev/null + then + dropbearkey -t ecdsa -f $HOME/${SSH_PKIF} | grep "^ecdsa" > $HOME/${SSH_PKIF}.pub + cat $HOME/${SSH_PKIF}.pub | dbclient ${REMOTE_USER}@${REMOTE_HOST} 'cat - >> .ssh/authorized_keys' echo -e "========================================================" echo -e "========================================================" else From 58e0b87001c38945ee0b9d83d73325c738137035 Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 10:45:37 -0500 Subject: [PATCH 08/35] fi --- gravity-sync.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/gravity-sync.sh b/gravity-sync.sh index a96e04c..d1ea40b 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -936,6 +936,7 @@ function config_generate { then dropbearkey -t ecdsa -f $HOME/${SSH_PKIF} | grep "^ecdsa" > $HOME/${SSH_PKIF}.pub cat $HOME/${SSH_PKIF}.pub | dbclient ${REMOTE_USER}@${REMOTE_HOST} 'cat - >> .ssh/authorized_keys' + fi echo -e "========================================================" echo -e "========================================================" else From 67de66aa23c5c7bc2dd4f0e6540a4bccc200421f Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 10:45:52 -0500 Subject: [PATCH 09/35] elif --- gravity-sync.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gravity-sync.sh b/gravity-sync.sh index d1ea40b..d721965 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -932,7 +932,7 @@ function config_generate { if hash ssh-copy-id 2>/dev/null then ssh-copy-id -f -i $HOME/${SSH_PKIF}.pub ${REMOTE_USER}@${REMOTE_HOST} - elseif hash dropbearkey 2>/dev/null + elif hash dropbearkey 2>/dev/null then dropbearkey -t ecdsa -f $HOME/${SSH_PKIF} | grep "^ecdsa" > $HOME/${SSH_PKIF}.pub cat $HOME/${SSH_PKIF}.pub | dbclient ${REMOTE_USER}@${REMOTE_HOST} 'cat - >> .ssh/authorized_keys' From 071332c77b0e73560a1ae239a636370124a05f7e Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 10:51:38 -0500 Subject: [PATCH 10/35] move dropbear pub file creation --- gravity-sync.sh | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/gravity-sync.sh b/gravity-sync.sh index d721965..5b8c59f 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -618,7 +618,7 @@ function detect_sshkeygen { KEYGEN_COMMAND="dropbearkey -t rsa -f" else mkdir $HOME/.ssh >/dev/null 2>&1 - KEYGEN_COMMAND="dropbearkey -t rsa -f" + KEYGEN_COMMAND="dropbearkey -t rsa -f $HOME/${SSH_PKIF} | grep "^rsa" > $HOME/${SSH_PKIF}.pub " fi else MESSAGE="No Alternatives Located" @@ -932,9 +932,8 @@ function config_generate { if hash ssh-copy-id 2>/dev/null then ssh-copy-id -f -i $HOME/${SSH_PKIF}.pub ${REMOTE_USER}@${REMOTE_HOST} - elif hash dropbearkey 2>/dev/null + elif hash dbclient 2>/dev/null then - dropbearkey -t ecdsa -f $HOME/${SSH_PKIF} | grep "^ecdsa" > $HOME/${SSH_PKIF}.pub cat $HOME/${SSH_PKIF}.pub | dbclient ${REMOTE_USER}@${REMOTE_HOST} 'cat - >> .ssh/authorized_keys' fi echo -e "========================================================" From e18a4f6fdc32dd5ea7b953592b963411c6c0af58 Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 10:53:24 -0500 Subject: [PATCH 11/35] ecdsa --- gravity-sync.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gravity-sync.sh b/gravity-sync.sh index 5b8c59f..5dfbbcc 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -618,7 +618,7 @@ function detect_sshkeygen { KEYGEN_COMMAND="dropbearkey -t rsa -f" else mkdir $HOME/.ssh >/dev/null 2>&1 - KEYGEN_COMMAND="dropbearkey -t rsa -f $HOME/${SSH_PKIF} | grep "^rsa" > $HOME/${SSH_PKIF}.pub " + KEYGEN_COMMAND="dropbearkey -t ecdsa -f $HOME/${SSH_PKIF} | grep "^ecdsa" > $HOME/${SSH_PKIF}.pub " fi else MESSAGE="No Alternatives Located" From 10c1c785457ac46c0a98fa6c7b659c002f410efd Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 12:29:08 -0500 Subject: [PATCH 12/35] Define method to extract .pub key for dropbear --- gravity-sync.sh | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/gravity-sync.sh b/gravity-sync.sh index 5dfbbcc..ffba533 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -618,7 +618,7 @@ function detect_sshkeygen { KEYGEN_COMMAND="dropbearkey -t rsa -f" else mkdir $HOME/.ssh >/dev/null 2>&1 - KEYGEN_COMMAND="dropbearkey -t ecdsa -f $HOME/${SSH_PKIF} | grep "^ecdsa" > $HOME/${SSH_PKIF}.pub " + KEYGEN_COMMAND="dropbearkey -t rsa -f $HOME/${SSH_PKIF}" fi else MESSAGE="No Alternatives Located" @@ -934,6 +934,7 @@ function config_generate { ssh-copy-id -f -i $HOME/${SSH_PKIF}.pub ${REMOTE_USER}@${REMOTE_HOST} elif hash dbclient 2>/dev/null then + dropbearkey -y -f $HOME/${SSH_PKIF} | grep "^ssh-rsa " > $HOME/${SSH_PKIF} cat $HOME/${SSH_PKIF}.pub | dbclient ${REMOTE_USER}@${REMOTE_HOST} 'cat - >> .ssh/authorized_keys' fi echo -e "========================================================" From da669349392e08c65ae4a51193c67e70e6456f97 Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 12:33:50 -0500 Subject: [PATCH 13/35] Output to .pub --- gravity-sync.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gravity-sync.sh b/gravity-sync.sh index ffba533..0e5f9e6 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -934,7 +934,7 @@ function config_generate { ssh-copy-id -f -i $HOME/${SSH_PKIF}.pub ${REMOTE_USER}@${REMOTE_HOST} elif hash dbclient 2>/dev/null then - dropbearkey -y -f $HOME/${SSH_PKIF} | grep "^ssh-rsa " > $HOME/${SSH_PKIF} + dropbearkey -y -f $HOME/${SSH_PKIF} | grep "^ssh-rsa " > $HOME/${SSH_PKIF}.pub cat $HOME/${SSH_PKIF}.pub | dbclient ${REMOTE_USER}@${REMOTE_HOST} 'cat - >> .ssh/authorized_keys' fi echo -e "========================================================" From df82e8d6d0e7ececc616603b3996e456b2e0a934 Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 13:08:18 -0500 Subject: [PATCH 14/35] Check for dbclient on validation --- gravity-sync.sh | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/gravity-sync.sh b/gravity-sync.sh index 0e5f9e6..d81e9cc 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -589,8 +589,15 @@ function validate_os_sshpass { MESSAGE="Validating SSH Connection to ${REMOTE_HOST}" echo_stat + if hash ssh 2>/dev/null + then timeout 5 ${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i '$HOME/${SSH_PKIF}' -o StrictHostKeyChecking=no ${REMOTE_USER}@${REMOTE_HOST} 'exit' >/dev/null 2>&1 error_validate + elif hash dbclient ssh 2>/dev/null + then + timeout 5 ${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i '$HOME/${SSH_PKIF}' ${REMOTE_USER}@${REMOTE_HOST} 'exit' >/dev/null 2>&1 + error_validate + fi } From 5cc7c305d9bbf97a3e7a3fd20fc21a8df37d8616 Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 13:29:50 -0500 Subject: [PATCH 15/35] extra echo for dbclient --- gravity-sync.sh | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/gravity-sync.sh b/gravity-sync.sh index d81e9cc..f397253 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -593,8 +593,9 @@ function validate_os_sshpass { then timeout 5 ${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i '$HOME/${SSH_PKIF}' -o StrictHostKeyChecking=no ${REMOTE_USER}@${REMOTE_HOST} 'exit' >/dev/null 2>&1 error_validate - elif hash dbclient ssh 2>/dev/null + elif hash dbclient 2>/dev/null then + if [ "$SSH_CMD" = "dbclient" ]; then; echo ''; fi timeout 5 ${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i '$HOME/${SSH_PKIF}' ${REMOTE_USER}@${REMOTE_HOST} 'exit' >/dev/null 2>&1 error_validate fi @@ -729,6 +730,7 @@ function md5_compare { MESSAGE="Analyzing ${REMOTE_HOST} ${GRAVITY_FI}" echo_stat + if [ "$SSH_CMD" = "dbclient" ]; then; echo ''; fi primaryDBMD5=$(${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "md5sum ${PIHOLE_DIR}/${GRAVITY_FI}") error_validate From 1f1aead6d4997fae12e5613241ff20e3a437e7b6 Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 13:30:43 -0500 Subject: [PATCH 16/35] fi; --- gravity-sync.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/gravity-sync.sh b/gravity-sync.sh index f397253..a5fab82 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -595,7 +595,7 @@ function validate_os_sshpass { error_validate elif hash dbclient 2>/dev/null then - if [ "$SSH_CMD" = "dbclient" ]; then; echo ''; fi + if [ "$SSH_CMD" = "dbclient" ]; then; echo ''; fi; timeout 5 ${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i '$HOME/${SSH_PKIF}' ${REMOTE_USER}@${REMOTE_HOST} 'exit' >/dev/null 2>&1 error_validate fi @@ -730,7 +730,7 @@ function md5_compare { MESSAGE="Analyzing ${REMOTE_HOST} ${GRAVITY_FI}" echo_stat - if [ "$SSH_CMD" = "dbclient" ]; then; echo ''; fi + if [ "$SSH_CMD" = "dbclient" ]; then; echo ''; fi; primaryDBMD5=$(${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "md5sum ${PIHOLE_DIR}/${GRAVITY_FI}") error_validate From 7a01524cb1b17ec9dcb6ea1a60b4f71d121b0315 Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 13:33:43 -0500 Subject: [PATCH 17/35] Less ; --- gravity-sync.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/gravity-sync.sh b/gravity-sync.sh index a5fab82..2ac95f7 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -595,7 +595,7 @@ function validate_os_sshpass { error_validate elif hash dbclient 2>/dev/null then - if [ "$SSH_CMD" = "dbclient" ]; then; echo ''; fi; + if [ "$SSH_CMD" = "dbclient" ]; then echo ''; fi; timeout 5 ${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i '$HOME/${SSH_PKIF}' ${REMOTE_USER}@${REMOTE_HOST} 'exit' >/dev/null 2>&1 error_validate fi @@ -730,7 +730,7 @@ function md5_compare { MESSAGE="Analyzing ${REMOTE_HOST} ${GRAVITY_FI}" echo_stat - if [ "$SSH_CMD" = "dbclient" ]; then; echo ''; fi; + if [ "$SSH_CMD" = "dbclient" ]; then echo ''; fi; primaryDBMD5=$(${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "md5sum ${PIHOLE_DIR}/${GRAVITY_FI}") error_validate From 8a62f790596dac36fbbc09427245079605232f5e Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 13:36:47 -0500 Subject: [PATCH 18/35] I hate bash --- gravity-sync.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gravity-sync.sh b/gravity-sync.sh index 2ac95f7..74a1cf8 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -595,7 +595,7 @@ function validate_os_sshpass { error_validate elif hash dbclient 2>/dev/null then - if [ "$SSH_CMD" = "dbclient" ]; then echo ''; fi; + # if [ "$SSH_CMD" = "dbclient" ]; then echo ''; fi; timeout 5 ${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i '$HOME/${SSH_PKIF}' ${REMOTE_USER}@${REMOTE_HOST} 'exit' >/dev/null 2>&1 error_validate fi From 9c35387106b84454422c78033fe4b082cddaa2e0 Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 13:45:16 -0500 Subject: [PATCH 19/35] Cleanup Dropbear Notice --- gravity-sync.sh | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/gravity-sync.sh b/gravity-sync.sh index 74a1cf8..4f7f983 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -669,22 +669,24 @@ function detect_ssh { if hash ssh 2>/dev/null then + MESSAGE="Validating SSH Command on $HOSTNAME (OpenSSH)" echo_good SSH_CMD='ssh' + elif hash dbclient 2>/dev/null + then + MESSAGE="Validating SSH Command on $HOSTNAME (Dropbear)" + echo_good + SSH_CMD='dbclient' else echo_fail + MESSAGE="Attempting to Compensate" echo_info - if hash dbclient 2>/dev/null - then - SSH_CMD='dbclient' - MESSAGE="Using Dropbear Instead" - echo_info - else - MESSAGE="Installing SSH Client with ${PKG_MANAGER}" - echo_stat - ${PKG_INSTALL} ssh-client >/dev/null 2>&1 - error_validate + MESSAGE="Installing SSH Client with ${PKG_MANAGER}" + echo_stat + + ${PKG_INSTALL} ssh-client >/dev/null 2>&1 + error_validate fi fi From 24c2425b7f1dfef3b5f584256013fd65f06b0a28 Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 13:45:37 -0500 Subject: [PATCH 20/35] then --- gravity-sync.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/gravity-sync.sh b/gravity-sync.sh index 4f7f983..b00956f 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -678,6 +678,7 @@ function detect_ssh { echo_good SSH_CMD='dbclient' else + then echo_fail MESSAGE="Attempting to Compensate" From eac8a746e2a1673f634a859a580e90238c1a1e4b Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 13:46:39 -0500 Subject: [PATCH 21/35] fi --- gravity-sync.sh | 1 - 1 file changed, 1 deletion(-) diff --git a/gravity-sync.sh b/gravity-sync.sh index b00956f..d14da7e 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -688,7 +688,6 @@ function detect_ssh { ${PKG_INSTALL} ssh-client >/dev/null 2>&1 error_validate - fi fi MESSAGE="Validating RSYNC Client on $HOSTNAME" From f1cac147131ebfbd1481055babac53862fa09aba Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 13:47:36 -0500 Subject: [PATCH 22/35] Extra then --- gravity-sync.sh | 1 - 1 file changed, 1 deletion(-) diff --git a/gravity-sync.sh b/gravity-sync.sh index d14da7e..aa67a17 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -678,7 +678,6 @@ function detect_ssh { echo_good SSH_CMD='dbclient' else - then echo_fail MESSAGE="Attempting to Compensate" From 4e4ddd56d1d2ce139d8ae118d954ed5b31e9cbd8 Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 13:50:58 -0500 Subject: [PATCH 23/35] Maybe " --- gravity-sync.sh | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/gravity-sync.sh b/gravity-sync.sh index aa67a17..31a2b15 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -596,7 +596,7 @@ function validate_os_sshpass { elif hash dbclient 2>/dev/null then # if [ "$SSH_CMD" = "dbclient" ]; then echo ''; fi; - timeout 5 ${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i '$HOME/${SSH_PKIF}' ${REMOTE_USER}@${REMOTE_HOST} 'exit' >/dev/null 2>&1 + timeout 5 ${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} 'exit' >/dev/null 2>&1 error_validate fi } @@ -664,17 +664,17 @@ function distro_check { ## Detect SSH & RSYNC function detect_ssh { - MESSAGE="Validating SSH Command on $HOSTNAME" + MESSAGE="Validating SSH on $HOSTNAME" echo_stat if hash ssh 2>/dev/null then - MESSAGE="Validating SSH Command on $HOSTNAME (OpenSSH)" + MESSAGE="Validating SSH on $HOSTNAME (OpenSSH)" echo_good SSH_CMD='ssh' elif hash dbclient 2>/dev/null then - MESSAGE="Validating SSH Command on $HOSTNAME (Dropbear)" + MESSAGE="Validating SSH on $HOSTNAME (Dropbear)" echo_good SSH_CMD='dbclient' else @@ -689,7 +689,7 @@ function detect_ssh { error_validate fi - MESSAGE="Validating RSYNC Client on $HOSTNAME" + MESSAGE="Validating RSYNC on $HOSTNAME" echo_stat if hash rsync 2>/dev/null From 3fe515a4e4651c4ba17fc1cfe636e822e2100bad Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 13:53:12 -0500 Subject: [PATCH 24/35] Fing " --- gravity-sync.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gravity-sync.sh b/gravity-sync.sh index 31a2b15..f620ec6 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -591,7 +591,7 @@ function validate_os_sshpass { echo_stat if hash ssh 2>/dev/null then - timeout 5 ${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i '$HOME/${SSH_PKIF}' -o StrictHostKeyChecking=no ${REMOTE_USER}@${REMOTE_HOST} 'exit' >/dev/null 2>&1 + timeout 5 ${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" -o StrictHostKeyChecking=no ${REMOTE_USER}@${REMOTE_HOST} 'exit' >/dev/null 2>&1 error_validate elif hash dbclient 2>/dev/null then From 2a578a62bd7378e2ed10383b2299d6bc7031f701 Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 13:54:14 -0500 Subject: [PATCH 25/35] Less iffy --- gravity-sync.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gravity-sync.sh b/gravity-sync.sh index f620ec6..3247f74 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -731,7 +731,7 @@ function md5_compare { MESSAGE="Analyzing ${REMOTE_HOST} ${GRAVITY_FI}" echo_stat - if [ "$SSH_CMD" = "dbclient" ]; then echo ''; fi; + # if [ "$SSH_CMD" = "dbclient" ]; then echo ''; fi; primaryDBMD5=$(${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "md5sum ${PIHOLE_DIR}/${GRAVITY_FI}") error_validate From 62794740c33b10cae2f72ee603ce95fe18c98235 Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 14:20:50 -0500 Subject: [PATCH 26/35] apt-get --- CHANGELOG.md | 7 ++++--- gravity-sync.sh | 1 + 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5d52c99..c31bb38 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,9 +12,10 @@ - No new Star Trek references. #### 1.7.6 -- Detects `dbclient` install as alternative to OpenSSH Client. (DietPi) -- Attempts to install OpenSSH Client if not found, and Dropbear is not alternative. (DietPi) -- Fix bug with `dropbearkey` not finding .ssh folder +- Detects `dbclient` install as alternative to OpenSSH Client. +- Attempts to install OpenSSH Client if not found, and Dropbear is not alternative. +- Fix bug with `dropbearkey` not finding .ssh folder. +- Numerous fixes to accomidate DietPi in general. #### 1.7.5 - No code changes! diff --git a/gravity-sync.sh b/gravity-sync.sh index 3247f74..99e4d5a 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -640,6 +640,7 @@ function detect_sshkeygen { function distro_check { if hash apt-get 2>/dev/null then + PKG_MANAGER="apt-get" PKG_INSTALL="sudo apt-get --yes --no-install-recommends --quiet install" elif hash rpm 2>/dev/null then From 5f3df0b663a00e5c9e377ddec7303a5f377e0411 Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 14:23:31 -0500 Subject: [PATCH 27/35] >/dev/null 2>&1 --- gravity-sync.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/gravity-sync.sh b/gravity-sync.sh index 99e4d5a..87ce262 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -733,7 +733,7 @@ function md5_compare { MESSAGE="Analyzing ${REMOTE_HOST} ${GRAVITY_FI}" echo_stat # if [ "$SSH_CMD" = "dbclient" ]; then echo ''; fi; - primaryDBMD5=$(${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "md5sum ${PIHOLE_DIR}/${GRAVITY_FI}") + primaryDBMD5=$(${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "md5sum ${PIHOLE_DIR}/${GRAVITY_FI}") >/dev/null 2>&1 error_validate MESSAGE="Analyzing $HOSTNAME ${GRAVITY_FI}" @@ -765,7 +765,7 @@ function md5_compare { MESSAGE="Analyzing ${REMOTE_HOST} ${CUSTOM_DNS}" echo_stat - primaryCLMD5=$(${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "md5sum ${PIHOLE_DIR}/${CUSTOM_DNS}") + primaryCLMD5=$(${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "md5sum ${PIHOLE_DIR}/${CUSTOM_DNS}") >/dev/null 2>&1 error_validate MESSAGE="Analyzing $HOSTNAME ${CUSTOM_DNS}" From 7a1d5c46d477bf72189ab05d1749ee840ab27c82 Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 14:24:12 -0500 Subject: [PATCH 28/35] >/dev/null 2>&1 inside --- gravity-sync.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/gravity-sync.sh b/gravity-sync.sh index 87ce262..8bf88f8 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -733,7 +733,7 @@ function md5_compare { MESSAGE="Analyzing ${REMOTE_HOST} ${GRAVITY_FI}" echo_stat # if [ "$SSH_CMD" = "dbclient" ]; then echo ''; fi; - primaryDBMD5=$(${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "md5sum ${PIHOLE_DIR}/${GRAVITY_FI}") >/dev/null 2>&1 + primaryDBMD5=$(${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "md5sum ${PIHOLE_DIR}/${GRAVITY_FI}" >/dev/null 2>&1) error_validate MESSAGE="Analyzing $HOSTNAME ${GRAVITY_FI}" @@ -765,7 +765,7 @@ function md5_compare { MESSAGE="Analyzing ${REMOTE_HOST} ${CUSTOM_DNS}" echo_stat - primaryCLMD5=$(${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "md5sum ${PIHOLE_DIR}/${CUSTOM_DNS}") >/dev/null 2>&1 + primaryCLMD5=$(${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "md5sum ${PIHOLE_DIR}/${CUSTOM_DNS}" >/dev/null 2>&1) error_validate MESSAGE="Analyzing $HOSTNAME ${CUSTOM_DNS}" From 8836be495d0f80bdbab3d1611027c2048f59a3e3 Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 15:17:01 -0500 Subject: [PATCH 29/35] sed 's/\s.*$//' --- CHANGELOG.md | 1 + gravity-sync.sh | 12 ++++-------- 2 files changed, 5 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c31bb38..6dfe55f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,7 @@ - Attempts to install OpenSSH Client if not found, and Dropbear is not alternative. - Fix bug with `dropbearkey` not finding .ssh folder. - Numerous fixes to accomidate DietPi in general. +- Fixes issue where `compare` function would show changes where actually none existed. #### 1.7.5 - No code changes! diff --git a/gravity-sync.sh b/gravity-sync.sh index 8bf88f8..5395e7c 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -732,13 +732,12 @@ function md5_compare { MESSAGE="Analyzing ${REMOTE_HOST} ${GRAVITY_FI}" echo_stat - # if [ "$SSH_CMD" = "dbclient" ]; then echo ''; fi; - primaryDBMD5=$(${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "md5sum ${PIHOLE_DIR}/${GRAVITY_FI}" >/dev/null 2>&1) + primaryDBMD5=$(${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "md5sum ${PIHOLE_DIR}/${GRAVITY_FI}" | sed 's/\s.*$//') error_validate MESSAGE="Analyzing $HOSTNAME ${GRAVITY_FI}" echo_stat - secondDBMD5=$(md5sum ${PIHOLE_DIR}/${GRAVITY_FI}) + secondDBMD5=$(md5sum ${PIHOLE_DIR}/${GRAVITY_FI} | sed 's/\s.*$//') error_validate if [ "$primaryDBMD5" == "$secondDBMD5" ] @@ -756,21 +755,18 @@ function md5_compare { then if [ -f ${PIHOLE_DIR}/${CUSTOM_DNS} ] then - # MESSAGE="Comparing ${CUSTOM_DNS} Changes" - # echo_info - if ${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} test -e ${PIHOLE_DIR}/${CUSTOM_DNS} then REMOTE_CUSTOM_DNS="1" MESSAGE="Analyzing ${REMOTE_HOST} ${CUSTOM_DNS}" echo_stat - primaryCLMD5=$(${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "md5sum ${PIHOLE_DIR}/${CUSTOM_DNS}" >/dev/null 2>&1) + primaryCLMD5=$(${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} "md5sum ${PIHOLE_DIR}/${CUSTOM_DNS} | sed 's/\s.*$//'") error_validate MESSAGE="Analyzing $HOSTNAME ${CUSTOM_DNS}" echo_stat - secondCLMD5=$(md5sum ${PIHOLE_DIR}/${CUSTOM_DNS}) + secondCLMD5=$(md5sum ${PIHOLE_DIR}/${CUSTOM_DNS} | sed 's/\s.*$//') error_validate if [ "$primaryCLMD5" == "$secondCLMD5" ] From 19d4c9c744d28a6eefbfc4497fcfdb7d6bdbc1d7 Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 15:27:36 -0500 Subject: [PATCH 30/35] WARN headers are purple --- CHANGELOG.md | 1 + gravity-sync.sh | 15 ++++++--------- 2 files changed, 7 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6dfe55f..a256b81 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,6 +17,7 @@ - Fix bug with `dropbearkey` not finding .ssh folder. - Numerous fixes to accomidate DietPi in general. - Fixes issue where `compare` function would show changes where actually none existed. +- [WARN] header is now purple all the way across, consistent with [INFO] as of 1.7.4. #### 1.7.5 - No code changes! diff --git a/gravity-sync.sh b/gravity-sync.sh index 5395e7c..f78032d 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -725,9 +725,6 @@ function error_validate { ## Validate Sync Required function md5_compare { - # MESSAGE="Comparing ${GRAVITY_FI} Changes" - # echo_info - HASHMARK='0' MESSAGE="Analyzing ${REMOTE_HOST} ${GRAVITY_FI}" @@ -747,7 +744,7 @@ function md5_compare { HASHMARK=$((HASHMARK+0)) else MESSAGE="${GRAVITY_FI} Differenced" - echo_info + echo_warn HASHMARK=$((HASHMARK+1)) fi @@ -776,7 +773,7 @@ function md5_compare { HASHMARK=$((HASHMARK+0)) else MESSAGE="${CUSTOM_DNS} Differenced" - echo_info + echo_warn HASHMARK=$((HASHMARK+1)) fi else @@ -791,11 +788,11 @@ function md5_compare { if [ "$HASHMARK" != "0" ] then - MESSAGE="Replication Necessary" - echo_info + MESSAGE="Replication Required" + echo_warn HASHMARK=$((HASHMARK+0)) else - MESSAGE="Replication Unncessary" + MESSAGE="No Changes to Replicate" echo_info exit_nochange fi @@ -1126,7 +1123,7 @@ function echo_info { ## Warning function echo_warn { - echo -e "${WARN} ${MESSAGE}" + echo -e "${WARN} ${PURPLE}${MESSAGE}${NC}" } ## Executing From 44a25555c5f84f3df4a8ae1157feb1679beb2a22 Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 15:30:17 -0500 Subject: [PATCH 31/35] 1.7.6 --- ADVANCED.md | 4 ++-- README.md | 2 +- VERSION | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/ADVANCED.md b/ADVANCED.md index 963cd7f..3f3c977 100644 --- a/ADVANCED.md +++ b/ADVANCED.md @@ -11,8 +11,8 @@ Download the latest release from [GitHub](https://github.com/vmstan/gravity-sync ```bash cd ~ -wget https://github.com/vmstan/gravity-sync/archive/v1.7.5.zip -unzip v1.7.5.zip -d gravity-sync +wget https://github.com/vmstan/gravity-sync/archive/v1.7.6.zip +unzip v1.7.6.zip -d gravity-sync cd gravity-sync ``` diff --git a/README.md b/README.md index 4670bb8..fd0e798 100644 --- a/README.md +++ b/README.md @@ -16,7 +16,7 @@ Gravity Sync **requires** Pi-hole 5.0 or higher be already installed on your ser - Gravity Sync is regularly tested during development with on any of the Linux distrobutions that Pi-hole is [certified to run on](https://docs.pi-hole.net/main/prerequesites/#supported-operating-systems). As Gravity Sync is just an (admittedly) long bash script, it will likely work on other Linux distributions that have the the necessary components. - Gravity Sync uses the `SUDO` command to elevate permissions for itself, on both Pi-hole systems, during execution. You will need to make sure that you have passwordless SUDO enabled for the accounts on both the primary and secondary Pi-hole that will be performing the work. Most of the pre-built images available for the Raspberry Pi already have this configured, but if you have your Pi-hole running in a virtual machine built from a generic ISO, you may need to [adjust this manually](https://linuxize.com/post/how-to-run-sudo-command-without-password/). - Gravity Sync has not been tested with Docker deployments of Pi-hole, and is not expected to work there without major modifications. You will need Pi-hole setup with a "traditional" install directly in the base operating system. There are likely other methods of sharing the `gravity.db` file between multiple Docker instances that are better suited to a container environment. -- Gravity Sync leverages SSH and RSYNC to do the leavy lifting between your Pi-hole nodes. If you're using a ultra-lightweight Pi distrbution (such as DietPi) make sure you have OpenSSH setup as your SSH client/server and not an alternative (such as Dropbear) which may act differently than OpenSSH. +- Gravity Sync leverages SSH and RSYNC to do the leavy lifting between your Pi-hole nodes. OpenSSH is reccomended but if you're using a ultra-lightweight Pi distrbution (such as DietPi) that uses Dropbear by default, it should work as well. Other SSH client/server combonations are not supported at this time. ### Pi-hole Architecture You will need to designate one Pi-Hole as primary and at least one as secondary. The primary Pi-hole is where you'll make all your configuration changes through the Web UI, doing things such as; manual whitelisting, adding blocklists, device/group management, configuring custom/local network DNS, and other changing other list settings. The secondary Pi-hole(s) are where you will install and configure Gravity Sync. diff --git a/VERSION b/VERSION index 5849151..d263485 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.7.5 \ No newline at end of file +1.7.6 \ No newline at end of file From b7e31b61c3459975eb2ed7f9f625dbbe082a92c0 Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 15:38:07 -0500 Subject: [PATCH 32/35] Fix to pull custom.list --- gravity-sync.sh | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/gravity-sync.sh b/gravity-sync.sh index f78032d..1e60164 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -781,6 +781,12 @@ function md5_compare { echo_info fi else + if ${SSHPASSWORD} ${SSH_CMD} -p ${SSH_PORT} -i "$HOME/${SSH_PKIF}" ${REMOTE_USER}@${REMOTE_HOST} test -e ${PIHOLE_DIR}/${CUSTOM_DNS} + then + REMOTE_CUSTOM_DNS="1" + MESSAGE="${REMOTE_HOST} has ${CUSTOM_DNS}" + echo_info + fi MESSAGE="No ${CUSTOM_DNS} Detected on $HOSTNAME" echo_info fi From 3f5558bbf322fd303d3db8f523eca4a3cc930968 Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 15:39:15 -0500 Subject: [PATCH 33/35] README --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index a256b81..ef2bd63 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,7 @@ - Numerous fixes to accomidate DietPi in general. - Fixes issue where `compare` function would show changes where actually none existed. - [WARN] header is now purple all the way across, consistent with [INFO] as of 1.7.4. +- Fixes issue where `custom.list` would only pull if the file already existed on the secondary Pi-hole. #### 1.7.5 - No code changes! From 0c604685c7ac8588c66e664c73cc4ff7b263b104 Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 15:45:44 -0500 Subject: [PATCH 34/35] Fix backup custom.list if not already created --- gravity-sync.sh | 3 +++ 1 file changed, 3 insertions(+) diff --git a/gravity-sync.sh b/gravity-sync.sh index 1e60164..14224d3 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -192,10 +192,13 @@ function pull_gs { then if [ "$REMOTE_CUSTOM_DNS" == "1" ] then + if [ -f ${PIHOLE_DIR}/${CUSTOM_DNS} ] + then MESSAGE="Backing Up ${CUSTOM_DNS} on $HOSTNAME" echo_stat cp ${PIHOLE_DIR}/${CUSTOM_DNS} $HOME/${LOCAL_FOLDR}/${BACKUP_FOLD}/${CUSTOM_DNS}.backup >/dev/null 2>&1 error_validate + fi MESSAGE="Pulling ${CUSTOM_DNS} from ${REMOTE_HOST}" echo_stat From ef82f4777378e84acb6ccbb96cbf01e250910d30 Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Mon, 1 Jun 2020 15:52:35 -0500 Subject: [PATCH 35/35] 1.7.6(1) --- CHANGELOG.md | 1 + gravity-sync.sh | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ef2bd63..88dcd17 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,7 @@ - Fixes issue where `compare` function would show changes where actually none existed. - [WARN] header is now purple all the way across, consistent with [INFO] as of 1.7.4. - Fixes issue where `custom.list` would only pull if the file already existed on the secondary Pi-hole. +- One new Star Trek reference. #### 1.7.5 - No code changes! diff --git a/gravity-sync.sh b/gravity-sync.sh index 14224d3..e783d4f 100755 --- a/gravity-sync.sh +++ b/gravity-sync.sh @@ -250,7 +250,7 @@ function pull_gs { fi fi - MESSAGE="Inverting Tachyon Pulse" + MESSAGE="Isolating Regeneration Pathways" echo_info sleep 1 @@ -320,7 +320,7 @@ function push_gs { fi fi - MESSAGE="Contacting Borg Collective" + MESSAGE="Inverting Tachyon Pulse" echo_info sleep 1