v1.5 Merge

Boerderij · Dec 30, 2018 · fa19954 · fa19954
2 parents 3fb63da + c480de5
commit fa19954
Show file tree

Hide file tree

Showing 12 changed files with 525 additions and 415 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,6 +1,25 @@
 # Change Log
 
-## [v1.4](https://github.com/Boerderij/Varken/tree/v1.4) (2018-12-18)
+## [v1.5](https://github.com/Boerderij/Varken/tree/v1.5) (2018-12-30)
+[Full Changelog](https://github.com/Boerderij/Varken/compare/v1.4...v1.5)
+
+**Implemented enhancements:**
+
+- \[Feature Request\] Add issues from Ombi [\#70](https://github.com/Boerderij/Varken/issues/70)
+- \[Feature Request\] Allow DNS Hostnames [\#66](https://github.com/Boerderij/Varken/issues/66)
+- Replace static grafana configs with a Public Example [\#32](https://github.com/Boerderij/Varken/issues/32)
+
+**Fixed bugs:**
+
+- \[BUG\] unexpected keyword argument 'channel\_icon' [\#73](https://github.com/Boerderij/Varken/issues/73)
+- \[BUG\] Unexpected keyword argument 'addOptions'  [\#68](https://github.com/Boerderij/Varken/issues/68)
+
+**Merged pull requests:**
+
+- v1.5 Merge [\#75](https://github.com/Boerderij/Varken/pull/75) ([DirtyCajunRice](https://github.com/DirtyCajunRice))
+- Add Ombi Issues [\#74](https://github.com/Boerderij/Varken/pull/74) ([anderssonoscar0](https://github.com/anderssonoscar0))
+
+## [v1.4](https://github.com/Boerderij/Varken/tree/v1.4) (2018-12-19)
 [Full Changelog](https://github.com/Boerderij/Varken/compare/v1.3-nightly...v1.4)
 
 **Implemented enhancements:**
@@ -92,10 +111,6 @@
 ## [v0.2-nightly](https://github.com/Boerderij/Varken/tree/v0.2-nightly) (2018-12-06)
 [Full Changelog](https://github.com/Boerderij/Varken/compare/v0.1...v0.2-nightly)
 
-**Implemented enhancements:**
-
-- Tautulli - multiple server support? [\#25](https://github.com/Boerderij/Varken/issues/25)
-
 **Closed issues:**
 
 - Create the DB if it does not exist. [\#38](https://github.com/Boerderij/Varken/issues/38)
@@ -105,7 +120,12 @@
 - use a config.ini instead of command-line flags [\#33](https://github.com/Boerderij/Varken/issues/33)
 - Migrate crontab to python schedule package [\#31](https://github.com/Boerderij/Varken/issues/31)
 - Consolidate missing and missing\_days in sonarr.py [\#30](https://github.com/Boerderij/Varken/issues/30)
+- Database Withou any scripts [\#29](https://github.com/Boerderij/Varken/issues/29)
+- Grafana dashboard json doesn't match format of readme screenshot? [\#28](https://github.com/Boerderij/Varken/issues/28)
 - Ombi something new \[Request\] [\#26](https://github.com/Boerderij/Varken/issues/26)
+- Users Online not populating [\#24](https://github.com/Boerderij/Varken/issues/24)
+- Missing dashboard [\#23](https://github.com/Boerderij/Varken/issues/23)
+- Is there a Docker Image available for these scripts? [\#22](https://github.com/Boerderij/Varken/issues/22)
 - Support for Linux without ASA [\#21](https://github.com/Boerderij/Varken/issues/21)
 
 **Merged pull requests:**

diff --git a/README.md b/README.md
@@ -10,14 +10,14 @@ from the Plex ecosystem into InfluxDB. Examples use Grafana for a
 frontend
 
 Requirements:
-* Python3.6+
+* Python3.6.7+
 * Python3-pip
 * [InfluxDB](https://www.influxdata.com/)
 
 <p align="center">
 Example Dashboard
 
-<img width="800" src="https://nickflix.io/sharex/firefox_NxdrqisVLF.png">
+<img width="800" src="https://i.imgur.com/G5bnpjs.png">
 </p>
 
 Supported Modules:
@@ -103,9 +103,16 @@ do not include database creation, please ensure you create an influx database
 named `varken`
 
 ### Grafana
-[Grafana Installation Documentation](http://docs.grafana.org/installation/)
+[Grafana Installation Documentation](http://docs.grafana.org/installation/)  
+[Official Example Dashboards](https://grafana.com/dashboards?search=Varken%20%5BOfficial%5D)
 
 Grafana is used in our examples but not required, nor packaged as part of
-Varken. Panel example pictures are pinned in the grafana-panels channel of
-discord. Future releases may contain a json-generator, but it does not exist
-as varken stands today.
+Varken. Panel examples now exist in both nightly and tagged releases hosted 
+on grafana.com (link above). 
+
+1. Use the link above, then click on your desired dashboard version
+2. Click `Copy ID to Clipboard`
+3. In grafana, click your dashboards menu dropdown, and then click `Import dashboard`
+4. Paste the ID into the `Grafana.com Dashboard` field and then click into empty space on the screen. (This should change the screen to show `Importing Dashboard from Grafana.com`
+5. Select your varken datasource name in the dropdown labeled `Varken`
+6. Click Import!
diff --git a/Varken.py b/Varken.py
@@ -121,6 +121,8 @@ def threaded(job):
                 schedule.every(server.request_type_run_seconds).seconds.do(threaded, OMBI.get_request_counts)
             if server.request_total_counts:
                 schedule.every(server.request_total_run_seconds).seconds.do(threaded, OMBI.get_all_requests)
+            if server.issue_status_counts:
+                schedule.every(server.issue_status_run_seconds).seconds.do(threaded, OMBI.get_issue_counts)
 
     if CONFIG.sickchill_enabled:
         for server in CONFIG.sickchill_servers:

diff --git a/data/varken.example.ini b/data/varken.example.ini
@@ -2,26 +2,28 @@
 #    - Sonarr + Radarr scripts support multiple servers. You can remove the second
 #      server by putting a # in front of the lines and section name, and removing
 #      that number from your server_ids list
-#    - fallback_ip, This is used when there is no IP listed in tautulli.
-#      This can happen when you are streaming locally. This is usually your public IP.
+#    - fallback_ip, This is used when there is no IP listed in Tautulli.
+#      This can happen when you are streaming locally. Set this to your public IP.
+#      You do not need to change this value if your IP changes. This is only for
+#      location lookups when there is a failure.
 
 [global]
 sonarr_server_ids = 1,2
 radarr_server_ids = 1,2
 tautulli_server_ids = 1
 ombi_server_ids = 1
-ciscoasa_firewall_ids = false
+ciscoasa_server_ids = false
 sickchill_server_ids = false
 
 [influxdb]
 url = influxdb.domain.tld
 port = 8086
-username =
-password =
+username = root
+password = root
 
 [tautulli-1]
 url = tautulli.domain.tld:8181
-fallback_ip = 0.0.0.0
+fallback_ip = 1.1.1.1
 apikey = xxxxxxxxxxxxxxxx
 ssl = false
 verify_ssl = false
@@ -83,6 +85,8 @@ get_request_type_counts = true
 request_type_run_seconds = 300
 get_request_total_counts = true
 request_total_run_seconds = 300
+get_issue_status_counts = true
+issue_status_run_seconds = 300
 
 [sickchill-1]
 url = sickchill.domain.tld:8081
@@ -92,8 +96,6 @@ verify_ssl = false
 get_missing = true
 get_missing_run_seconds = 300
 
-
-
 [ciscoasa-1]
 url = firewall.domain.tld
 username = cisco

diff --git a/varken/__init__.py b/varken/__init__.py
@@ -1,2 +1,2 @@
-VERSION = 1.4
+VERSION = 1.5
 BRANCH = 'master'
diff --git a/varken/helpers.py b/varken/helpers.py
@@ -27,17 +27,20 @@ def __init__(self, data_folder):
 
     def lookup(self, ipaddress):
         ip = ipaddress
-        self.logger.debug('Getting lat/long for Tautulli stream')
+        self.logger.debug('Getting lat/long for Tautulli stream using ip with last octet ending in %s',
+                          ip.split('.')[-1:])
         return self.reader.city(ip)
 
     def update(self):
         today = date.today()
-        dbdate = None
+
         try:
             dbdate = date.fromtimestamp(stat(self.dbfile).st_ctime)
         except FileNotFoundError:
             self.logger.error("Could not find GeoLite2 DB as: %s", self.dbfile)
             self.download()
+            dbdate = date.fromtimestamp(stat(self.dbfile).st_ctime)
+
         first_wednesday_day = [week[2:3][0] for week in monthcalendar(today.year, today.month) if week[2:3][0] != 0][0]
         first_wednesday_date = date(today.year, today.month, first_wednesday_day)
 
@@ -52,7 +55,6 @@ def update(self):
             else:
                 self.logger.debug('Geolite2 DB will update in %s days', abs(td.days))
 
-
     def download(self):
         tar_dbfile = abspath(join(self.data_folder, 'GeoLite2-City.tar.gz'))
         url = 'http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.tar.gz'

diff --git a/varken/iniparser.py b/varken/iniparser.py
@@ -59,15 +59,17 @@ def read_file(self):
             self.logger.error('Config file missing (varken.ini) in %s', self.data_folder)
             exit(1)
 
-    def url_check(self, url=None, include_port=True):
+    def url_check(self, url=None, include_port=True, section=None):
         url_check = url
+        module = section
         inc_port = include_port
 
-        search = (r'(?:(?:[A-Z0-9](?:[A-Z0-9-]{0,61}[A-Z0-9])?\.)+(?:[A-Z]{2,6}\.?|[A-Z0-9-]{2,}\.?)|'  # domain...
+        search = (r'(?:([a-z0-9]+(-[a-z0-9]+)*\.)+[a-z]{2,}|'  # domain...
                   r'localhost|'  # localhost...
+                  r'^[a-zA-Z0-9_-]*|'  # hostname only. My soul dies a little every time this is used...
                   r'\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})'  # ...or ip
                   )
-
+        # Include search for port if it is needed.
         if inc_port:
             search = (search + r'(?::\d+)?' + r'(?:/?|[/?]\S+)$')
         else:
@@ -78,28 +80,28 @@ def url_check(self, url=None, include_port=True):
         valid = match(regex, url_check) is not None
         if not valid:
             if inc_port:
-                self.logger.error('%s is invalid! URL must host/IP and port if not 80 or 443. ie. localhost:8080',
-                                  url_check)
+                self.logger.error('%s is invalid in module [%s]! URL must host/IP and '
+                                  'port if not 80 or 443. ie. localhost:8080',
+                                  url_check, module)
                 exit(1)
             else:
-                self.logger.error('%s is invalid! URL must host/IP. ie. localhost', url_check)
+                self.logger.error('%s is invalid in module [%s]! URL must host/IP. ie. localhost', url_check, module)
                 exit(1)
         else:
-            self.logger.debug('%s is a valid URL in the config.', url_check)
+            self.logger.debug('%s is a valid URL in module [%s].', url_check, module)
             return url_check
 
     def parse_opts(self):
         self.read_file()
         # Parse InfluxDB options
-        url = self.url_check(self.config.get('influxdb', 'url'), include_port=False)
+        url = self.url_check(self.config.get('influxdb', 'url'), include_port=False, section='influxdb')
 
         port = self.config.getint('influxdb', 'port')
 
         username = self.config.get('influxdb', 'username')
 
         password = self.config.get('influxdb', 'password')
-
-        self.influx_server = InfluxServer(url, port, username, password)
+        self.influx_server = InfluxServer(url=url, port=port, username=username, password=password)
 
         # Check for all enabled services
         for service in self.services:
@@ -111,7 +113,7 @@ def parse_opts(self):
                     server = None
                     section = f"{service}-{server_id}"
                     try:
-                        url = self.url_check(self.config.get(section, 'url'))
+                        url = self.url_check(self.config.get(section, 'url'), section=section)
 
                         apikey = None
                         if service != 'ciscoasa':
@@ -137,9 +139,11 @@ def parse_opts(self):
 
                             queue_run_seconds = self.config.getint(section, 'queue_run_seconds')
 
-                            server = SonarrServer(server_id, scheme + url, apikey, verify_ssl, missing_days,
-                                                  missing_days_run_seconds, future_days, future_days_run_seconds,
-                                                  queue, queue_run_seconds)
+                            server = SonarrServer(id=server_id, url=scheme + url, api_key=apikey, verify_ssl=verify_ssl,
+                                                  missing_days=missing_days, future_days=future_days,
+                                                  missing_days_run_seconds=missing_days_run_seconds,
+                                                  future_days_run_seconds=future_days_run_seconds,
+                                                  queue=queue, queue_run_seconds=queue_run_seconds)
 
                         if service == 'radarr':
                             queue = self.config.getboolean(section, 'queue')
@@ -150,8 +154,9 @@ def parse_opts(self):
 
                             get_missing_run_seconds = self.config.getint(section, 'get_missing_run_seconds')
 
-                            server = RadarrServer(server_id, scheme + url, apikey, verify_ssl, queue, queue_run_seconds,
-                                                  get_missing, get_missing_run_seconds)
+                            server = RadarrServer(id=server_id, url=scheme + url, api_key=apikey, verify_ssl=verify_ssl,
+                                                  queue_run_seconds=queue_run_seconds, get_missing=get_missing,
+                                                  queue=queue, get_missing_run_seconds=get_missing_run_seconds)
 
                         if service == 'tautulli':
                             fallback_ip = self.config.get(section, 'fallback_ip')
@@ -164,9 +169,11 @@ def parse_opts(self):
 
                             get_stats_run_seconds = self.config.getint(section, 'get_stats_run_seconds')
 
-                            server = TautulliServer(server_id, scheme + url, fallback_ip, apikey, verify_ssl,
-                                                    get_activity, get_activity_run_seconds, get_stats,
-                                                    get_stats_run_seconds)
+                            server = TautulliServer(id=server_id, url=scheme + url, api_key=apikey,
+                                                    verify_ssl=verify_ssl, get_activity=get_activity,
+                                                    fallback_ip=fallback_ip, get_stats=get_stats,
+                                                    get_activity_run_seconds=get_activity_run_seconds,
+                                                    get_stats_run_seconds=get_stats_run_seconds)
 
                         if service == 'ombi':
                             request_type_counts = self.config.getboolean(section, 'get_request_type_counts')
@@ -177,17 +184,26 @@ def parse_opts(self):
 
                             request_total_run_seconds = self.config.getint(section, 'request_total_run_seconds')
 
-                            server = OmbiServer(server_id, scheme + url, apikey, verify_ssl, request_type_counts,
-                                                request_type_run_seconds, request_total_counts,
-                                                request_total_run_seconds)
+                            issue_status_counts = self.config.getboolean(section, 'get_issue_status_counts')
+
+                            issue_status_run_seconds = self.config.getint(section, 'issue_status_run_seconds')
+
+                            server = OmbiServer(id=server_id, url=scheme + url, api_key=apikey, verify_ssl=verify_ssl,
+                                                request_type_counts=request_type_counts,
+                                                request_type_run_seconds=request_type_run_seconds,
+                                                request_total_counts=request_total_counts,
+                                                request_total_run_seconds=request_total_run_seconds,
+                                                issue_status_counts=issue_status_counts,
+                                                issue_status_run_seconds=issue_status_run_seconds)
 
                         if service == 'sickchill':
                             get_missing = self.config.getboolean(section, 'get_missing')
 
                             get_missing_run_seconds = self.config.getint(section, 'get_missing_run_seconds')
 
-                            server = SickChillServer(server_id, scheme + url, apikey, verify_ssl,
-                                                     get_missing, get_missing_run_seconds)
+                            server = SickChillServer(id=server_id, url=scheme + url, api_key=apikey,
+                                                     verify_ssl=verify_ssl, get_missing=get_missing,
+                                                     get_missing_run_seconds=get_missing_run_seconds)
 
                         if service == 'ciscoasa':
                             username = self.config.get(section, 'username')
@@ -198,8 +214,10 @@ def parse_opts(self):
 
                             get_bandwidth_run_seconds = self.config.getint(section, 'get_bandwidth_run_seconds')
 
-                            server = CiscoASAFirewall(server_id, scheme + url, username, password, outside_interface,
-                                                      verify_ssl, get_bandwidth_run_seconds)
+                            server = CiscoASAFirewall(id=server_id, url=scheme + url, verify_ssl=verify_ssl,
+                                                      username=username, password=password,
+                                                      outside_interface=outside_interface,
+                                                      get_bandwidth_run_seconds=get_bandwidth_run_seconds)
 
                         getattr(self, f'{service}_servers').append(server)