[feat] Use glob patterns to auto-find local paths, translate to remote paths (#3)

- Advanced options: Use glob patterns to match local files, translate to remote paths
- Add optional file path to Unraid template

README.md

@@ -94,10 +94,11 @@ docker run -d \
 
 #### Paths and Environment Variables
 
-| Path      | Description                                                           |
+| Path                     | Description                                                                                   |
-|-----------|-----------------------------------------------------------------------|
+|--------------------------|-----------------------------------------------------------------------------------------------|
-| `/config` | Path to config directory (`config.yaml` should be in this directory)  |
+| `/config`                | Path to config directory (`config.yaml` should be in this directory)                          |
-| `/logs`   | Path to log directory (`Plex Prerolls.log` will be in this directory) |
+| `/logs`                  | Path to log directory (`Plex Prerolls.log` will be in this directory)                         |
+| `/path/to/preroll/files` | Path to the root directory of all preroll files (for [Path Globbing](#path-globbing) feature) |
 
 | Environment Variable | Description                                                       |
 |----------------------|-------------------------------------------------------------------|
@@ -131,7 +132,8 @@ You can define as many schedules as you want, in the following categories (order
 All schedule entries accept an optional `weight` value that can be used to adjust the emphasis of this entry over
 others by adding the listed paths multiple times. Since Plex selects a random preroll from the list of paths, having the
 same path listed multiple times increases its chances of being selected over paths that only appear once. This allows
-you to combine, e.g. a `date_range` entry with an `always` entry, but place more weight/emphasis on the `date_range` entry.
+you to combine, e.g. a `date_range` entry with an `always` entry, but place more weight/emphasis on the `date_range`
+entry.
 
 ```yaml
 date_range:
@@ -148,8 +150,10 @@ date_range:
 #### Disable Always
 
 Any schedule entry (except for the `always` section) can disable the inclusion of the `always` section by setting the
-`disable_always` value to `true`. This can be useful if you want to make one specific, i.e. `date_range` entry for a holiday, 
+`disable_always` value to `true`. This can be useful if you want to make one specific, i.e. `date_range` entry for a
-and you don't want to include the `always` section for this specific holiday, but you still want to include the `always` section
+holiday,
+and you don't want to include the `always` section for this specific holiday, but you still want to include the `always`
+section
 for other holidays.
 
 ```yaml
@@ -205,6 +209,81 @@ You should [adjust your cron schedule](#scheduling-script) to run the script mor
 
 ---
 
+## Advanced Configuration
+
+### Path Globbing
+
+**NOTE**: This feature will only work if you are running the script/Docker container on the same machine as your Plex
+server.
+
+Instead of listing out each individual preroll file, you can use glob (wildcard) patterns to match multiple files in a
+specific directory.
+The application will search for all files on your local filesystem that match the pattern(s) and automatically translate
+them to Plex-compatible remote paths.
+
+#### Setup
+
+Enable the feature under the advanced section of the config file, and specify the path to the root directory of your
+preroll files, as well as the path to the same directory as seen by Plex.
+
+```yaml
+advanced:
+  path_globbing:
+    enabled: true
+    root_path: /path/to/preroll/directory/in/relation/to/application
+    plex_path: /path/to/same/directory/as/seen/by/plex
+```
+
+For example, if your prerolls on your file system are located at `/mnt/user/media/prerolls` and Plex sees them
+at `/media/prerolls`, you would set the `root_path` to `/mnt/user/media/prerolls` and the `plex_path`
+to `/media/prerolls`.
+
+If you are using the Docker container, you can mount the preroll directory to the container at any location you would
+prefer (recommended: `/files`) and set the `root_path` accordingly.
+
+If you are using the Unraid version of this container, the "Files Path" path is mapped to `/files` by default; you
+should set `root_path` to `/files` and `plex_path` to the same directory as seen by Plex.
+
+#### Usage
+
+In any schedule section, you can use the `path_globs` key to specify a list of glob patterns to match files.
+
+```yaml
+always:
+  enabled: true
+  paths:
+    - /remote/path/1.mp4
+    - /remote/path/2.mp4
+    - /remote/path/3.mp4
+  path_globs:
+    - "*.mp4"
+```
+
+The above example will match all `.mp4` files in the `root_path` directory and append them to the list of prerolls.
+
+If you have organized your prerolls into subdirectories, you can specify specific subdirectories to match, or use `**`
+to match all subdirectories.
+
+```yaml
+always:
+  enabled: true
+  paths:
+    - /remote/path/1.mp4
+    - /remote/path/2.mp4
+    - /remote/path/3.mp4
+  path_globs:
+    - "subdir1/*.mp4"
+    - "subdir2/*.mp4"
+    - "subdir3/**/*.mp4"
+```
+
+You can use both `paths` and `path_globs` in the same section, allowing you to mix and match specific files with glob
+patterns.
+Please note that `paths` entries must be fully-qualified **remote** paths (as seen by Plex), while `path_globs` entries
+are relative to the **local** `root_path` directory.
+
+---
+
 ## Scheduling Script
 
 **NOTE:** Scheduling is handled automatically in the Docker version of this script via the `CRON_SCHEDULE` environment

config.yaml.example

@@ -9,9 +9,11 @@ plex:
 always:
   enabled: true
   paths:
-    - "path/to/video1.mp4"
+    - "remote/path/to/video1.mp4"
-    - "path/to/video2.mp4"
+    - "remote/path/to/video2.mp4"
-    - "path/to/video3.mp4"
+    - "remote/path/to/video3.mp4"
+  path_globs:
+    - "local/path/to/prerolls/*.mp4" # Optional, use globbing to match local paths
   count: 10 # Optional, randomly select X many videos from the list rather than all of them
   weight: 1 # Optional, how much to emphasize these pre-rolls over others (higher = more likely to play)
 
@@ -23,30 +25,32 @@ date_range:
       start_date: 2020-01-01 # Jan 1st, 2020
       end_date: 2020-01-02 # Jan 2nd, 2020
       paths:
-        - "path/to/video1.mp4"
+        - "remote/path/to/video1.mp4"
-        - "path/to/video2.mp4"
+        - "remote/path/to/video2.mp4"
-        - "path/to/video3.mp4"
+        - "remote/path/to/video3.mp4"
+      path_globs:
+        - "local/path/to/prerolls/*.mp4" # Optional, use globbing to match local paths
       weight: 2 # Optional, add these paths to the list twice (make up greater percentage of prerolls - more likely to be selected)
       disable_always: true # Optional, if present and true, disable the always prerolls when this schedule is active
     - start_date: xxxx-07-04 # Every year on July 4th
       end_date: xxxx-07-04 # Every year on July 4th
       paths:
-        - "path/to/video1.mp4"
+        - "remote/path/to/video1.mp4"
-        - "path/to/video2.mp4"
+        - "remote/path/to/video2.mp4"
-        - "path/to/video3.mp4"
+        - "remote/path/to/video3.mp4"
       disable_always: false
     - start_date: xxxx-xx-02 # Every year on the 2nd of every month
       end_date: xxxx-xx-03 # Every year on the 3rd of every month
       paths:
-        - "path/to/video1.mp4"
+        - "remote/path/to/video1.mp4"
-        - "path/to/video2.mp4"
+        - "remote/path/to/video2.mp4"
-        - "path/to/video3.mp4"
+        - "remote/path/to/video3.mp4"
     - start_date: xxxx-xx-xx 08:00:00 # Every day at 8am
       end_date: xxxx-xx-xx 09:30:00 # Every day at 9:30am
       paths:
-        - "path/to/video1.mp4"
+        - "remote/path/to/video1.mp4"
-        - "path/to/video2.mp4"
+        - "remote/path/to/video2.mp4"
-        - "path/to/video3.mp4"
+        - "remote/path/to/video3.mp4"
 
 # Schedule prerolls by week of the year
 weekly:
@@ -54,15 +58,17 @@ weekly:
   weeks:
     - number: 1 # First week of the year
       paths:
-        - "path/to/video1.mp4"
+        - "remote/path/to/video1.mp4"
-        - "path/to/video2.mp4"
+        - "remote/path/to/video2.mp4"
-        - "path/to/video3.mp4"
+        - "remote/path/to/video3.mp4"
+      path_globs:
+        - "local/path/to/prerolls/*.mp4" # Optional, use globbing to match local paths
       weight: 1 # Optional, how much to emphasize these pre-rolls over others (higher = more likely to play)
     - number: 2 # Second week of the year
       paths:
-        - "path/to/video1.mp4"
+        - "remote/path/to/video1.mp4"
-        - "path/to/video2.mp4"
+        - "remote/path/to/video2.mp4"
-        - "path/to/video3.mp4"
+        - "remote/path/to/video3.mp4"
       disable_always: true # If true, disable the always prerolls when this schedule is active
 
 # Schedule prerolls by month of the year
@@ -71,13 +77,21 @@ monthly:
   months:
     - number: 1 # January
       paths:
-        - "path/to/video1.mp4"
+        - "remote/path/to/video1.mp4"
-        - "path/to/video2.mp4"
+        - "remote/path/to/video2.mp4"
-        - "path/to/video3.mp4"
+        - "remote/path/to/video3.mp4"
       weight: 1 # Optional, how much to emphasize these pre-rolls over others (higher = more likely to play)
     - number: 2 # February
       paths:
-        - "path/to/video1.mp4"
+        - "remote/path/to/video1.mp4"
-        - "path/to/video2.mp4"
+        - "remote/path/to/video2.mp4"
-        - "path/to/video3.mp4"
+        - "remote/path/to/video3.mp4"
+      path_globs:
+        - "local/path/to/prerolls/*.mp4" # Optional, use globbing to match local paths
       disable_always: false # If true, disable the always prerolls when this schedule is active
+
+advanced:
+  path_globbing:
+    enabled: true # If true, use globbing to match paths
+    root_path: /files # The root folder to use for globbing
+    plex_path: /path/to/prerolls/in/plex # The path to use for the Plex server

modules/config_parser.py

@@ -4,6 +4,7 @@ from typing import List, Union
 import confuse
 import yaml
 
+import modules.files as files
 import modules.logs as logging
 
 
@@ -28,10 +29,34 @@ class Entry(YAMLElement):
         super().__init__(data)
         self.data = data
 
+    def all_paths(self, advanced_settings: 'AdvancedConfig' = None) -> List[str]:
+        paths = []
+        paths.extend(self.remote_paths)
+
+        if not advanced_settings or not advanced_settings.path_globbing.enabled:
+            return paths
+
+        local_files_root = advanced_settings.path_globbing.local_root_folder
+        remote_files_root = advanced_settings.path_globbing.remote_root_folder
+
+        for glob in self.local_path_globs:
+            local_files = files.get_all_files_matching_glob_pattern(directory=local_files_root, pattern=glob)
+            for local_file in local_files:
+                remote_file = files.translate_local_path_to_remote_path(local_path=local_file,
+                                                                        local_root_folder=local_files_root,
+                                                                        remote_root_folder=remote_files_root)
+                paths.append(remote_file)
+
+        return paths
+
     @property
-    def paths(self) -> List[str]:
+    def remote_paths(self) -> List[str]:
         return self._get_value(key="paths", default=[])
 
+    @property
+    def local_path_globs(self) -> List[str]:
+        return self._get_value(key="path_globs", default=[])
+
     @property
     def weight(self) -> int:
         return self._get_value(key="weight", default=1)
@@ -67,7 +92,8 @@ class DateRangeEntry(Entry):
         return self._get_value(key="end_date", default=None)
 
     def __repr__(self):
-        return f"DateRangeEntry(start_date={self.start_date}, end_date={self.end_date}, paths={self.paths}, weight={self.weight})"
+        return (f"DateRangeEntry(start_date={self.start_date}, end_date={self.end_date}, "
+                f"remote_paths={self.remote_paths}, local_path_globs={self.local_path_globs}, weight={self.weight})")
 
 
 class WeekEntry(NumericalEntry):
@@ -75,7 +101,8 @@ class WeekEntry(NumericalEntry):
         super().__init__(data=data)
 
     def __repr__(self):
-        return f"WeekEntry(number={self.number}, paths={self.paths}, weight={self.weight})"
+        return (f"WeekEntry(number={self.number}, remote_paths={self.remote_paths}, "
+                f"local_path_globs={self.local_path_globs}, weight={self.weight})")
 
 
 class MonthEntry(NumericalEntry):
@@ -83,7 +110,8 @@ class MonthEntry(NumericalEntry):
         super().__init__(data=data)
 
     def __repr__(self):
-        return f"MonthEntry(number={self.number}, paths={self.paths}, weight={self.weight})"
+        return (f"MonthEntry(number={self.number}, remote_paths={self.remote_paths}, "
+                f"local_path_globs={self.local_path_globs}, weight={self.weight})")
 
 
 class ConfigSection(YAMLElement):
@@ -134,6 +162,32 @@ class PlexServerConfig(ConfigSection):
         return port
 
 
+class PathGlobbingConfig(ConfigSection):
+    def __init__(self, data):
+        super().__init__(section_key="path_globbing", data=data)
+
+    @property
+    def enabled(self) -> bool:
+        return self._get_value(key="enabled", default=False)
+
+    @property
+    def local_root_folder(self) -> str:
+        return self._get_value(key="root_path", default="/")
+
+    @property
+    def remote_root_folder(self) -> str:
+        return self._get_value(key="plex_path", default="/")
+
+
+class AdvancedConfig(ConfigSection):
+    def __init__(self, data):
+        super().__init__(section_key="advanced", data=data)
+
+    @property
+    def path_globbing(self) -> PathGlobbingConfig:
+        return PathGlobbingConfig(data=self.data)
+
+
 class ScheduleSection(ConfigSection):
     def __init__(self, section_key: str, data):
         super().__init__(section_key=section_key, data=data)
@@ -148,20 +202,44 @@ class AlwaysSection(ScheduleSection):
         super(ScheduleSection, self).__init__(section_key="always", data=data)
 
     # Double inheritance doesn't work well with conflicting "data" properties, just re-implement these two functions.
+    def all_paths(self, advanced_settings: 'AdvancedConfig' = None) -> List[str]:
+        paths = []
+        paths.extend(self.remote_paths)
+
+        if not advanced_settings or not advanced_settings.path_globbing.enabled:
+            return paths
+
+        local_files_root = advanced_settings.path_globbing.local_root_folder
+        remote_files_root = advanced_settings.path_globbing.remote_root_folder
+
+        for glob in self.local_path_globs:
+            local_files = files.get_all_files_matching_glob_pattern(directory=local_files_root, pattern=glob)
+            for local_file in local_files:
+                remote_file = files.translate_local_path_to_remote_path(local_path=local_file,
+                                                                        local_root_folder=local_files_root,
+                                                                        remote_root_folder=remote_files_root)
+                paths.append(remote_file)
+
+        return paths
+
     @property
-    def paths(self) -> List[str]:
+    def remote_paths(self) -> List[str]:
         return self._get_value(key="paths", default=[])
 
+    @property
+    def local_path_globs(self) -> List[str]:
+        return self._get_value(key="path_globs", default=[])
+
     @property
     def weight(self) -> int:
         return self._get_value(key="weight", default=1)
 
-    @property
+    def random_count(self, advanced_settings: 'AdvancedConfig' = None) -> int:
-    def random_count(self) -> int:
+        return self._get_value(key="count", default=len(self.all_paths(advanced_settings=advanced_settings)))
-        return self._get_value(key="count", default=len(self.paths))
 
     def __repr__(self):
-        return f"AlwaysSection(paths={self.paths}, weight={self.weight}, random_count={self.random_count})"
+        return (f"AlwaysSection(remote_paths={self.remote_paths}, local_path_globs={self.local_path_globs}, "
+                f"weight={self.weight}")
 
 
 class DateRangeSection(ScheduleSection):
@@ -222,6 +300,7 @@ class Config:
         self.date_ranges = DateRangeSection(data=self.config)
         self.monthly = MonthlySection(data=self.config)
         self.weekly = WeeklySection(data=self.config)
+        self.advanced = AdvancedConfig(data=self.config)
 
         logging.debug(f"Using configuration:\n{self.log()}")
 
@@ -236,8 +315,8 @@ class Config:
             "Plex - URL": self.plex.url,
             "Plex - Token": "Exists" if self.plex.token else "Not Set",
             "Always - Enabled": self.always.enabled,
-            "Always - Paths": self.always.paths,
+            "Always - Paths": self.always.all_paths(advanced_settings=self.advanced),
-            "Always - Count": self.always.random_count,
+            "Always - Count": self.always.random_count(advanced_settings=self.advanced),
             "Always - Weight": self.always.weight,
             "Date Range - Enabled": self.date_ranges.enabled,
             "Date Range - Ranges": self.date_ranges.ranges,
@@ -245,6 +324,9 @@ class Config:
             "Monthly - Months": self.monthly.months,
             "Weekly - Enabled": self.weekly.enabled,
             "Weekly - Weeks": self.weekly.weeks,
+            "Advanced - Path Globbing - Enabled": self.advanced.path_globbing.enabled,
+            "Advanced - Path Globbing - Local Root Folder": self.advanced.path_globbing.local_root_folder,
+            "Advanced - Path Globbing - Remote Root Folder": self.advanced.path_globbing.remote_root_folder
         }
 
     def log(self) -> str:

modules/files.py

@@ -0,0 +1,32 @@
+import glob
+import os
+from typing import List
+
+
+def get_all_files_matching_glob_pattern(directory: str, pattern: str) -> List[str]:
+    """
+    Get all files matching a glob pattern in a directory.
+
+    Args:
+        directory (str): The directory to search in.
+        pattern (str): The glob pattern to search for.
+
+    Returns:
+        List[str]: A list of file paths that match the glob pattern.
+    """
+    return [file for file in glob.glob(os.path.join(directory, pattern)) if os.path.isfile(file)]
+
+
+def translate_local_path_to_remote_path(local_path: str, local_root_folder: str, remote_root_folder: str) -> str:
+    """
+    Translate a local path to a remote path.
+
+    Args:
+        local_path (str): The local path to translate.
+        local_root_folder (str): The root folder of the local path.
+        remote_root_folder (str): The root folder of the remote path.
+
+    Returns:
+        str: The translated remote path.
+    """
+    return local_path.replace(local_root_folder, remote_root_folder, 1)

modules/schedule_manager.py

@@ -21,32 +21,41 @@ class ScheduleManager:
         logging.info("Parsing schedules...")
         if self._config.weekly.enabled:
             for week in self._config.weekly.weeks:
-                self.weekly_schedules.append(models.schedule_entry_from_week_number(week_number=week.number,
+                self.weekly_schedules.append(models.schedule_entry_from_week_number(
-                                                                                    paths=week.paths,
+                    week_number=week.number,
-                                                                                    weight=week.weight,
+                    paths=week.all_paths(
-                                                                                    disable_always=week.disable_always))
+                        advanced_settings=self._config.advanced),
+                    weight=week.weight,
+                    disable_always=week.disable_always))
 
         if self._config.monthly.enabled:
             for month in self._config.monthly.months:
-                self.monthly_schedules.append(models.schedule_entry_from_month_number(month_number=month.number,
+                self.monthly_schedules.append(models.schedule_entry_from_month_number(
-                                                                                      paths=month.paths,
+                    month_number=month.number,
-                                                                                      weight=month.weight,
+                    paths=month.all_paths(
-                                                                                      disable_always=month.disable_always))
+                        advanced_settings=self._config.advanced),
+                    weight=month.weight,
+                    disable_always=month.disable_always))
         if self._config.date_ranges.enabled:
             for date_range in self._config.date_ranges.ranges:
-                entry = models.schedule_entry_from_date_range(start_date_string=date_range.start_date,
+                entry = models.schedule_entry_from_date_range(
-                                                              end_date_string=date_range.end_date,
+                    start_date_string=date_range.start_date,
-                                                              paths=date_range.paths,
+                    end_date_string=date_range.end_date,
-                                                              weight=date_range.weight,
+                    paths=date_range.all_paths(
-                                                              name=date_range.name,
+                        advanced_settings=self._config.advanced),
-                                                              disable_always=date_range.disable_always)
+                    weight=date_range.weight,
+                    name=date_range.name,
+                    disable_always=date_range.disable_always)
                 if entry:
                     self.date_range_schedules.append(entry)
 
         if self._config.always.enabled:
-            self.always_schedules.append(models.schedule_entry_from_always(paths=self._config.always.paths,
+            self.always_schedules.append(models.schedule_entry_from_always(
-                                                                           count=self._config.always.random_count,
+                paths=self._config.always.all_paths(
-                                                                           weight=self._config.always.weight))
+                    advanced_settings=self._config.advanced),
+                count=self._config.always.random_count(
+                    advanced_settings=self._config.advanced),
+                weight=self._config.always.weight))
 
     @property
     def valid_weekly_schedules(self) -> List[ScheduleEntry]:

templates/plex_prerolls.xml

@@ -21,6 +21,7 @@
     <Config Name="Cron Schedule" Target="CRON_SCHEDULE" Default="0 0 * * *" Description="How often to update schedules. See https://crontab.guru for help." Type="Variable" Display="always" Required="true" Mask="false" />
     <Config Name="Dry Run" Target="DRY_RUN" Default="" Description="Enable dry-run (testing) mode. No changes will be made to Plex." Type="Variable" Display="always" Required="false" Mask="false" />
     <Config Name="Timezone" Target="TZ" Default="UTC" Description="Timezone to use when calculating schedules" Type="Variable" Display="always" Required="false" Mask="false">UTC</Config>
-    <Config Name="Config Path" Target="/config" Default="/mnt/user/appdata/plex_prerolls/config" Mode="rw" Description="Where config file will be stored" Type="Path" Display="advanced" Required="true" Mask="false">/mnt/user/appdata/plex_prerolls/config</Config>
+    <Config Name="Config Path" Target="/config" Default="/mnt/user/appdata/plex_prerolls/config" Mode="ro" Description="Where config file will be stored" Type="Path" Display="always" Required="true" Mask="false">/mnt/user/appdata/plex_prerolls/config</Config>
     <Config Name="Log Path" Target="/logs" Default="/mnt/user/appdata/plex_prerolls/logs" Mode="rw" Description="Where debug logs will be stored" Type="Path" Display="advanced" Required="true" Mask="false">/mnt/user/appdata/plex_prerolls/logs</Config>
+    <Config Name="Files Path" Target="/files" Default="/mnt/user/appdata/plex_prerolls/files" Mode="ro" Description="(Optional) Where preroll files are stored, for local path glob feature" Type="Path" Display="advanced" Required="false" Mask="false">/mnt/user/appdata/plex_prerolls/files</Config>
 </Container>