From 6a1a132ce69fe040c7aba80b4beb8eb2ca223941 Mon Sep 17 00:00:00 2001 From: Michael Stanclift Date: Thu, 28 May 2020 12:21:24 -0500 Subject: [PATCH] Pre 1.5.0 updates --- README.md | 80 ++++++++++++++++++++++++++++++++++----------------- gs-install.sh | 5 ++++ 2 files changed, 58 insertions(+), 27 deletions(-) create mode 100644 gs-install.sh diff --git a/README.md b/README.md index a6d3249..5920a40 100644 --- a/README.md +++ b/README.md @@ -21,12 +21,18 @@ Additionally, some things to consider: - Gravity Sync has not been tested with Docker container deployments of Pi-hole, and is not expected to work there without major modifications. You will need Pi-hole setup with a "traditional" install directly in the base operating system. ## Installation -The main purpose of this script is my own personal use, but if you find it helpful then I encourage you to use it and if you'd like provide feedback or contribute. As such, I'll lay out two ways to consume it. The first is more bleeding edge in that you'll download and run whatever the latest version of the script is on GitHub. +### The Really Easy Way -If this is too aggressive for you, maybe because you want to make changes to the script that are specific to your environment, or you're worried it'll blow something up, then please proceed to option 2. +Login to your *secondary* Pi-hole, and run: -### Option 1 -Login to your *secondary* PH, and while in your users home directory, use `git` to clone the script to your server and keep the latest copy of the script on your server. +``` +curl -sR https://raw.githubusercontent.com/vmstan/gravity-sync/master/gs-install.sh | bash +``` + +Proceed to the Configuration section. + +### The Easy Way +Login to your *secondary* Pi-hole, and while in your users home directory, use `git` to clone the script to your server and keep the latest copy of the script on your server. (Note, this is exactly what **The Really Easy Way** does above.) ``` cd ~ @@ -36,8 +42,8 @@ cd gravity-sync Proceed to the Configuration section. -### Option 2 -So a life on the wildside of file sync isn't for you? That's fine. +### The Less Easy Way +So a life on the wildside of file sync isn't for you? That's fine. Keep in mind that installing via this method means you won't be able to use Gravity Sync's built-in update mechanism. Download the latest release from [GitHub](https://github.com/vmstan/gravity-sync/releases) and extract the files to your *secondary* PH server. @@ -52,9 +58,10 @@ cd gravity-sync Please note the script **must** be run from a folder in your user home directory (ex: /home/USER/gravity-sync) -- I wouldn't suggest deviating from the gravity-sync folder name. If you do you'll need to also change the configuration settings defined in the `gravity-sync.sh` script, which can be a little tedious to do everytime you upgrade the script. ## Configuration -### The Easy Way After you install Gravity Sync to your server (reguardless of the option you selected above) you will need to create a configuration file called `gravity-sync.conf` in the same folder as the script. +### The Easy Way + ``` ./gravity-sync config ``` @@ -66,17 +73,17 @@ This will guide you through the process of: - Configuring your key-pair and applying it to your primary Pi-hole - Testing your authentication method -After you've completed your configuration, proceed to the Execution phase. Unless you feel like making this (slightly) harder on yourself. +After you've completed your configuration, proceed to the Execution phase. ### The Less Easy Way There will be a file called `gravity-sync.conf.example` that you can use as the basis for your own `gravity-sync.conf` file. Make a copy of the example file and modify it with your site specific settings. ``` cp gravity-sync.conf.example gravity-sync.conf -vim gravity-sync.conf +vi gravity-sync.conf ``` -*Note: If you don't have VIM on your system use VI, if you don't like VI use NANO, or if you don't like any of those subsitute for your text editor of choice. I'm not here to start a war.* +*Note: If you don't like VI or don't have VIM on your system, use NANO, or if you don't like any of those subsitute for your text editor of choice. I'm not here to start a war.* Make sure you've set the REMOTE_HOST and REMOTE_USER variables with the IP (or DNS name) and user account to authenticate to the primary Pi. This account will need to have sudo permissions on the remote system. @@ -85,7 +92,7 @@ REMOTE_HOST='192.168.1.10' REMOTE_USER='pi' ``` -Do not set the `REMOTE_PASS` variable until you've read the next section on SSH. +*Do not set the `REMOTE_PASS` variable until you've read the next section on SSH.* ### SSH Configuration Gravity Sync uses SSH to run commands on the primary Pi-hole, and sync the two systems by performing file copies. There are two methods available for authenticating with SSH. @@ -107,7 +114,7 @@ Subsitute REMOTE_USER for the account on the primary PH with sudo permissions, a Make sure to leave the `REMOTE_PASS` variable set to nothing in `gravity-sync.conf` if you want to use key-pair authentication. #### Password Authentication -This is the non-preferred option, as it depends on an non-standard utility called `sshpass` which must be installed on your secondary PH. Install it using your package manage or choice. The example below is for Raspbian or Ubuntu. +This is the non-preferred option, as it depends on an non-standard utility called `sshpass` which must be installed on your secondary PH. Install it using your package manager of choice. The example below is for Raspberry Pi OS (previously Raspbian) or Ubuntu. ``` sudo apt install sshpass @@ -130,21 +137,23 @@ Now test the script. You can run a comparison between the two which will be non- ./gravity-sync.sh compare ``` -Assuming Gravity Sync runs successfully, it'll indicate if there are changes pending between the two databases. If not, I suggest making a subtle change to a whitelist/blacklist on your primary PH, such as a description field, and then running it again to validate your installation is working correctly. +Assuming Gravity Sync runs successfully, it'll indicate if there are changes pending between the two databases. If not, I suggest making a subtle change to a whitelist/blacklist on your primary PH, such as changing a description field or disabling a whitelist item, and then running `./gravity-sync.sh compare` again to validate your installation is working correctly. -Gravity Sync, when functioning in `pull` mode, will not prompt for user input after execution. It will perform some checks to help insure success and then stop before making changes if it detects an issue. If there are no changes pending, it will exit without making an attempt to copy data. +### The Pull Function + +The Gravity Sync Pull, is the standard method of sync operation, and will not prompt for user input after execution. It will perform some checks to help insure success and then stop before making changes if it detects an issue. It will also perform the same `compare` function outlined above, and if there are no changes pending, it will exit without making an attempt to copy data. ``` ./gravity-sync.sh pull ``` -If the execution completes, you will now have overwritten your running gravity.db on the secondary PH after creating a copy of the running database (`gravity.db.backup`) in the `backup` subfolder located with your script. The script will also keep a copy of the last sync'd gravity.db from the master, in the `backup` folder identified as `gravity.db.pull` should you need it for some reason. +If the execution completes, you will now have overwritten your running gravity.db on the secondary PH after creating a copy of the running database (`gravity.db.backup`) in the `backup` subfolder located with your script. Gravity Sync will also keep a copy of the last sync'd gravity.db from the master, in the `backup` folder identified as `gravity.db.pull` for future use. -Finally, a file called `gravity-sync.log` will be created in the `gravity-sync` folder along side the script, with the date the script was last executed appended to the bottom. +Finally, a file called `gravity-sync.log` will be created in the `gravity-sync` folder along side the script with the date the script was last executed appended to the bottom. You can check for successful pull attempts by running: `./gravity-sync.sh logs` -## Failover +### The Push Function Gravity Sync includes the ability to `push` from the secondary PH back to the primary. This would be useful in a situation where your primary PH is down for an extended period of time, and you have made list changes on the secondary PH that you want to force back to the primary, when it comes online. ``` @@ -156,35 +165,52 @@ Before executing, this will make a copy of the remote database under `backup/gra This function purposefuly asks for user interaction to avoid being accidentally automated. ## Updates -If you installed via Option 1, you can run the built-in updater to get the latest version of all the files. +### The Easy Way +If you installed **The Really Easy Way** or **The Easy Way**, you can run the built-in updater to get the latest version of all the files. ``` ./gravity-sync.sh update ``` -Your copy of the `gravity-sync.conf` file, logs and backups should not be be impacted by this update, as they are specifically ignored by git. +Your copy of the `gravity-sync.conf` file, logs and backups should not be be impacted by this update, as they are specifically ignored. -If you installed via Option 2, download and overwrite the `gravity-sync.sh` file with a newer version. With either version, you should review the contents of the script bundle, specifically the example configuration file, to make sure there are no new required settings. +### The Less Easy Way -The goal of Gravity Sync is to be simple, so any additional requirements should also be called out when it's executed. After updating, be sure to manually run a `./gravity-sync.sh compare` or `./gravity-sync.sh pull` to validate things are still working as expected. You can run a `./gravity-sync.sh config` at any time to generate a new configuration file. +You will essentially download and overwrite the `gravity-sync.sh` file with a newer version. You should review the contents of the script bundle (specifically the example configuration file) to make sure there are no new additional files or required settings. + +### Either Way +The main goal of Gravity Sync is to be simple to execute and maintain, so any additional requirements should also be called out when it's executed. After updating, be sure to manually run a `./gravity-sync.sh compare` or `./gravity-sync.sh pull` to validate things are still working as expected. + +You can run a `./gravity-sync.sh config` at any time to generate a new configuration file if you're concerned that you're missing something. ## Automation -I've automated my synchronization using Crontab. If you'd like to keep this a manual process then ignore this section. By default my script will run at the top and bottom of every hour (1:00 PM, 1:30 PM, 2:00 PM, etc) but you are free to dial this back if you feel this is too aggressive by adjusting your cron timer. +Automation of sync is accomplished by adding an execution of the script to the user's crontab file. As Gravity Sync won't make any changes if it doesn't detect a difference to sync, then the impact should be minor to your systems. -As Gravity Sync won't make any changes if it doesn't detect a difference to sync, then it's impact should be minor to your systems. +### The Easy Way +Just run the built in `automate` function: + +``` +./gravity-sync.sh automate +``` + +Select the frequency per hour that you'd like to sync (once, twice, quadrice, etc) and that's it. + +### The Less Easy Way +If you prefer to still use cron but modify your settings by hand, using the entry below will cause the entry to run at the top and bottom of every hour (1:00 PM, 1:30 PM, 2:00 PM, etc) but you are free to dial this back or be more agressive if you feel the need. ``` crontab -e */30 * * * * /bin/bash /home/USER/gravity-sync/gravity-sync.sh pull > /home/USER/gravity-sync/gravity-sync.cron ``` -Now, make another small adjustment to your primary settings. Now just wait until the annointed hour, and see if your changes have been synchronized. If so, profit! +### Either Way +You can verify your cron entry by running `crontab -l` and see it listed at the bottom of the file. If you used the built in automation function and decide to change your frequency, you'll need to run `crontab -e` and adjust this by hand, or delete the entire line in the crontab file and then re-run the `./gravity-sync automate` function. -If not, start from the beginning. +Now, make another small adjustment to your primary settings and wait until annointed time to see if your changes have been synchronized. If so, profit! If not, start from the beginning. -From this point forward any blocklist changes you make to the primary will reflect on the secondary within 30 minutes. +From this point forward any blocklist changes you make to the primary will reflect on the secondary within the frequency you select. -If you'd like to see the log of what was run the last crontab, you can view that output. +If you'd like to see the log of what was run the last crontab, you can view that output by running: ``` ./gravity-sync.sh cron diff --git a/gs-install.sh b/gs-install.sh new file mode 100644 index 0000000..39fca56 --- /dev/null +++ b/gs-install.sh @@ -0,0 +1,5 @@ +#!/bin/bash + +cd $HOME +git clone https://github.com/vmstan/gravity-sync.git +cd gravity-sync \ No newline at end of file