UsersGuide

• Introduction
• Features
  • New features and enhancements vs. the Sage EPG service
  • Incomplete or unstable Sage EPG features
  • Missing Sage EPG features: Not available
  • Known Issues
• System Requirements
• Plugin PRE-Installation
• Plugin Installation
  • Remove Video Sources
  • Configure Plugin
    • Configure Your Schedules Direct JSON Service Settings
  • Restart SageTV
  • Reconfigure Video Sources
  • Forge EPG update
• Upgrading the Plugin
• Uninstalling Plugin
  • Before uninstalling
  • Remove video sources
  • Stop SageTV
  • Remove EPG plugin property
  • Restart SageTV
  • Reconfigure video sources with Sage/alternate EPG service
• Advanced Features
  • Program & Airing Filters
    • Filter Script Locations
    • Object Insertion
    • Object Modification
    • Execution Order
    • Script Classpath
    • Execution Time Considerations
    • Example: Fix for Fox HD Flag
  • Channel Generators
    • Script Location
    • Runtime Environment
    • Station ID for Generated Channels
    • Example: Add Missing NFL Sunday Ticket HD Channels
  • Lineup Editors
    • Script Location
    • Runtime Envrionment
    • Required Return Value
    • Example: Add Missing NFL Sunday Ticket HD Channels
  • Program Generators
    • Script Location
    • Runtime Environment
    • Warning: External IDs for Generated Shows
    • Example: Basic Show Collection Generator
  • Airing Generators
    • Script Location
    • Runtime Environment
    • Example: Fix NFL Sunday Ticket Listings
  • Force EPG Update via STV

Introduction

The Schedules Direct EPG (sdepg) plugin for SageTV 7 provides an alternative to the Sage EPG data service. There are some advantages and disadvantages to this plugin, all of which will be discussed throughout this user guide.

Eventually, the Sage EPG data service will no longer be available and when that happens this is one of your alternatives for getting listings data into your SageTV EPG.

Though this is a native SageTV plugin, there are some special installation and uninstallation instructions that must be followed so please read those sections carefully.

Features

New features and enhancements vs. the Sage EPG service

• Continued access to EPG data after SageTV discontinues its EPG data service
• Support for multiple lineups
• Advanced filtering and preprocessing of EPG data prior to it being added to the EPG through the use of Groovy scripting
• Ability to pull in additional EPG data from other sources to supplement or correct/update the available EPG data through the use of Groovy scripting
• Ability to trigger on demand updates of the EPG data through the SageTV STV

Incomplete or unstable Sage EPG features

• Team roles are applied to shows in some instances where it may not necessarily make sense Fixed as of v0.0.6.0

Missing Sage EPG features: Not available

• An actor's character details are not available (i.e. Michael C. Hall as Dexter Morgan is not available, but, of course, Michael C. Hall is listed as an actor in the show). These details will remain available for those series that are already in your wiz.bin prior to switching EPG sources; new shows starting after you switch data sources will not have this data available.
• Channel logos not available. (Note: There are other plugins available that provide channel logos) Channel logos available for import as of v0.0.5.2

Known Issues

• The Schedules Direct show ID received may have zeros prefixed to the ID number, which will confuse Sage into thinking it hasn't recorded some episodes even though it has. For example, using the Sage EPG service, the show ID for "Family Guy" is EP296001nnnn, but using Schedules Direct, the show ID is EP00296001nnnn. Notice that Schedules Direct is prefixing the leading zeros after the EP? I have to investigate if it's safe to simply remove the prefixed zeros always or if I should just leave it. For now, I'm just leaving it. This will require users to manually remark airings as watched to prevent duplicate recordings of such episodes. Update (26 Apr 2014): It has been determined that SageTV incorrectly modifies the unique show ids it receives from Tribune. Therefore, it is the policy of this plugin to not munge the ids received from Tribune and so the issue discussed here will never be addressed.

System Requirements

To use this plugin, you need:

• SageTV v7.1.9 server (nothing older nor nothing newer will work)
• Windows and Linux servers supported
• Java 7 (Java 8 untested, probably will work, but no promises)

All of these prerequisites are enforced by the SageTV plugin manager.

Plugin PRE-Installation

STOP!! Before beginning installation you must check the following:

• If you have the mc2xml EPG plugin installed, you MUST uninstall it NOW! That plugin cannot be installed simultaneously with this plugin.
• Check your current version of HttpClient plugin. If the version installed is v4.0.x then you must manually uninstall and upgrade the HttpClient and HttpCore plugins to v4.2.1. This manual removal is required only due to a packaging error in the older versions of the plugin. I apologize in advance, but there's nothing I can do about this. This is a one time only situation; the newest version of these plugins are packaged properly and will be able to be properly upgraded in the future. Follow these directions to remove and upgrade the two plugins.

Plugin Installation

Go to the list of available SageTV plugins in your Sage STV and choose to install the Schedules Direct EPG plugin.

IMPORTANT: If Sage says you must restart SageTV DO NOT perform the restart at this time. Click "Restart Later" and continue on with the steps below. Restarting now is not catastrophic, but it just means you'll have to do a second restart later on.

Remove Video Sources

Now it's time to cut ties with the SageTV EPG service. You must reconfigure all your video sources within SageTV, there is no choice.

From the standard STV, go to Setup > Setup Video Sources, which will bring you to the following screen:

Click on each video source you have configured and on the next screen for each, click "Remove Source" on the left hand side. Once you have removed all video sources then continue on to the next step.

Configure Plugin

It's now time to configure the EPG plugin. Using the standard STV, select Setup > SageTV Plugins > Installed Plugins.

Scroll down and click on the Schedules Direct EPG Plugin item then click on Configure Plugin. NOTE: If the Configure Plugin option is not available then you must restart SageTV at this point then come back and load the plugin configuration screen. In this case, you won't be able to avoid the multiple restarts.

{TODO: Describe each config option}

{TODO: Describe that SD id/pwd MUST be set at this stage, before reconfig of video sources}

Configure Your Schedules Direct JSON Service Settings

Before you restart SageTV, you need to register all the lineups you wish to access within Sage. The lineups you have configured at the Schedules Direct web site are NOT used by the JSON service. You must register lineups separately for the JSON service.

This must be done using the JSON service API. If you have the web UI plugin installed then the sdepg plugin includes modules that add web based configuration tools to the web ui. Simply visit the home page (or any page) in your web ui after installing the sdepg plugin. You will see a new menu item called Schedules Direct. Be sure that you've already configured the plugin with your Schedules Direct credentials before loading any of these web pages. Click on Configure Lineups. Use the search box to locate your lineups and add them to your account. Be sure the lineups you've added show up at the top of the page under Configured Headends. Once you see them there then they are registered and you may continue. Use the Status web page to view the current status of your account and the Schedules Direct service.

NOTE: If you do not use the web ui plugin then you must configure your Schedules Direct account using the bundled command line tool. See this page for those details.

Restart SageTV

You must now restart SageTV, even if you haven't been told it's necessary. It is necessary to restart SageTV at this time in order for the EPG plugin hook to be properly configured.

Reconfigure Video Sources

Reconfigure each of your video sources. Now when you select your EPG lineups, you should be presented with the lineups configured in your Schedules Direct account.

NOTE: Be patient when configuring lineups as a complete EPG download and update must be performed on each configured lineup before your channels are presented. Depending on the number of channels, your internet speed, and your hardware, this can be very slow - up to 20 minutes. Be patient and if after 20 minutes the screen is stuck here:

Then check the sagetv_0.txt log file for errors or the \SageTV\plugins\sdepg\logs\*.log files for errors.

Once the EPG update is complete, you can select your channels, etc. From this point forward, you complete your lineup configurations just like using the SageTV EPG service.

Forge EPG update

After an initial setup of this plugin, you may notice that your EPG is empty. Do the following after you've configured all the channels on all your lineups:
+ Go to the plugin settings screen
+ Click on Force Refresh
You will need to do these same steps anytime you add new channels to a lineup. If you don't, then those new channels will have no data until the next pull of new data from the Schedules Direct server.

Upgrading the Plugin

Upgrading the plugin is rather straightforward, simply apply the plugin updates when you're alerted about them.

Please apply updates when they're made available, especially during the v0.x betas. Unlike most plugins, if you're using this one then it's providing a rather critical service to your SageTV system and running the latest version available will be critical to proper EPG data being inserted into your wiz.bin.

Uninstalling Plugin

Before uninstalling

If you decide to stop using the sdepg plugin please be aware that when you switch back to the Sage EPG service that you might end up recording shows that you've already recorded. This is because some shows use a zero padded show ID with sdepg and those leading zeros are stripped by the Sage EPG service. Since the show IDs aren't identical, Sage will not consider those airings recorded when you switch EPG services. This is more inconvenient than anything, but it is something to be aware of.

Follow the steps below to uninstall this plugin.

Remove video sources

First you must remove all your video sources and reconfigure them after completely removing the EPG plugin.

Go to Setup > Video Sources

Remove each video source.

Go to Setup > SageTV Plugins > Installed Plugins

Select and uninstall the Schedules Direct EPG plugin.

NOTE: If Sage asks you to restart SageTV then say NO and do it later. You need to modify the Sage.properties file before restarting so don't bother doing a restart right now.

Stop SageTV

At this point, stop the SageTV server. Do NOT restart it at this time.

Remove EPG plugin property

Edit your Sage.properties file and look for a line that starts with epg/epg_import_plugin

The line should actually read:

epg/epg_import_plugin=sagex.epg.schedulesdirect.EPGImportPluginSchedulesDirect

Change it so it reads:

epg/epg_import_plugin=

Making this edit is what actually removes the EPG plugin from your system.

Restart SageTV

Now you can restart your SageTV server.

Reconfigure video sources with Sage/alternate EPG service

When you connect to the server for the first time you should be taken back to the video source configuration wizard, which will allow you to reconfigure your video sources. At this stage, you just reconfigure your video sources as normal.

Advanced Features

Program & Airing Filters

By using filters, you can massage the details of programs and airings before they're inserted into the Sage wiz.bin database. This provides some very interesting use cases and can help correct EPG errors from the source.

Filter Script Locations

All filters are Groovy scripts and must be inserted into \plugins\sdepg\airing_filters\ or \plugins\sdepg\program_filters\.

Object Insertion

The filters are provided two variables: _src_ and it is an sdjson Airing object if it's an airing filter or a SageProgram object if it's a program filter.

The other object is called _log_ and it is a preconfigured instance of a log4j Logger object that allows you to log output from your script to a private file. The file sits along side the location of the script. So if your script is called program_filters\filter1.groovy then anything you log via this object will be written to program_filters\filter1.log. NOTE: All log files are cleared at the start of each EPG update. This means the log files created always represent the output of the latest run only. If you want to log differently then you can create and configure your own Logger instance as log4j is on the classpath of all Groovy scripts executed.

Object Modification

Your filter script is free to modify the _src_ object's properties, but you cannot reassign the object itself. If you do reassign the object then your changes to the object will not be processed. In other words, think of the _src_ variable as a final variable, even though that can't be enforced.

// _src_ is an Airing
_src_.hdtv = true // This is fine and the hdtv flag will be set to true in wiz.bin for this airing
// _src_ is an Airing
_src_ = null // Ooops, you've lost the reference to the object, this filter script is now useless, the changes to _src_ are lost, and you're just wasting time
// _src_ is an Airing
_src_ = new Airing() // Ooops, you can't change the reference that _src_ is pointing to; again now anything you do to _src_ will be ignored
// _src_ is an Airing
_src_.hdtv = false
_src_ = new Airing() // Ooops, the hdtv change will be processed, but anything you do to _src_ after this line is not processed

So just remember not to modify the _src_ reference itself, just modify the object's properties as desired.

Execution Order

Every program in the json input is passed to every filter in the program_filters directory before it is inserted into the wiz.bin. Note, however, that in the case of programs, a program is only processed once per EPG update even though a Program appears multiple times in the json input. Filters are executed in alphabetical order and the _src_ object received in a filter contains the modifications of the filters executed before it. In other words, the filters are chained together to produce the final filtered result of an object.

The airing filters work exactly the same way except that since every airing in the json input is unique, the filters are run on every single airing in the json input.

Script Classpath

The filter scripts use the same classpath as SageTV so anything in the Sage classpath is in your script's classpath. In addition, the directory that the script is running from is also in the classpath. If you needed to add jars or .class files for a filter script, you can use the filter directory as a classpath root.

Execution Time Considerations

Remember that these filter scripts are all executed during an EPG update. Filters should be relatively fast and well tested. A filter that throws an exception will not halt your EPG update, but will usually slow down the EPG update significantly.

You can do complicated/relatively slow things in your filters, but just be aware that doing so means your EPG updates will take that much longer.

Example: Fix for Fox HD Flag

So for me, the HD flag of some Fox shows is not properly set. When it's not set then Sage considers the airing to be SD and in this case Sage tends to choose other SD channels to record the shows from instead of preferring to record the HD airing. Before this, I would simply restrict the favourite to the Fox HD channels, which is fine, but can produce conflicts unnecessarily. In other words, I want to record the HD airing, but if I can't because of conflicts then I want it to be able to consider the SD airings of the show as well.

The fix is simple: Filter the Fox HD airings and ensure the HD flag is set to true for all airings on the channels. With an airing filter, this is real easy:

import sagex.api.*

def cal = Calendar.getInstance()
cal.setTime(_src_.gmtStart)
if(cal.get(Calendar.HOUR_OF_DAY) > 18 && cal.get(Calendar.HOUR_OF_DAY) < 2) return
def chan = ChannelAPI.GetChannelForStationID(_src_.station.id.toInteger())
if(ChannelAPI.GetChannelNumber(chan) ==~ /287|291/) // Fox HD channels (for my lineup)
    _src_.hdtv = true

This filter ensures that every airing on my Fox channels between 6pm and 3am daily has the HD flag set to true. I do this because one HD channel (287) is from the east and the other is from the west (291) and the prime time schedule on 291 is 11pm-2am for me.

So the filter simply examines the start time of the received Airing object. If it falls in the given time frame then I inspect the channel of the airing and if it's a Fox channel then I set the hdtv property to true. That's it. Just like that, I've corrected the EPG errors I've been receiving for years. Now the Sage scheduler can properly consider HD vs. SD airings of Fox shows. Just drop the above script into the airing_filers directory and the filter will be applied on each and every EPG update. Now the HD flag is properly set for my Fox HD channels.

Channel Generators

If your lineup(s) are missing channels, you can use a channel generator to add additional channels to your lineup. Be aware that channels you add will not have any EPG data unless you create the EPG data with program and airing generators.

Script Location

All channel generator scripts must be placed in the SageTV\plugins\sdepg\channel_generators\ directory.

Runtime Environment

All scripts must be Groovy scripts and must return a value of type Collection<Channel>.

All channel generators receive a log4j Logger object named _log_ and another variable named _chans_. The _chans_ variable is a List<Channel> that contains all of the channels created for the lineup currently being processed.

When a script returns a collection, all of those Channel objects will be added to the wiz.bin as valid channels.

Station ID for Generated Channels

When generating new channels, you must ensure that the station id you generate for each channel does not conflict with any id that may be included with the channels from your downloaded lineups. All ids must be positive integers. I suggest starting at a number close to 2³² since these numbers are less likely to be used by Schedules Direct channels.

Example: Add Missing NFL Sunday Ticket HD Channels

So for my lineup, sometimes my provider adds additional HD channels for NFL on Sundays and these channels don't get added to my linup upstream in time. Not a problem now, since I can just ensure they're always in my lineup.

import sagex.api.*
import sagex.epg.schedulesdirect.data.*

def chanId = 2000000000
def sageChans = ChannelAPI.GetAllChannels()
def chans = []
for(def name : ['NFLHD1', 'NFLHD2', 'NFLHD3', 'NFLHD4', 'NFLHD5']) {
    def exists = false
    for(def c : sageChans) {
        if(ChannelAPI.GetChannelName(c) == name) {
            exists = true
            break
        }
    }
    if(!exists)
        chans.add(new Channel(callsign: name, longName: "NFL Sunday Ticket HD ${name[name.length() - 1]}", network: 'Premium Sports', stationId: chanId))
    ++chanId
}
return chans

So this generator only generates the channel if it doesn't already exist. Also notice that my channel IDs start at 2,000,000,000.

Lineup Editors

Lineup editors allow you to add additional channels to a Schedules Direct lineup. Usually you'd do this when you created new channels with a channel generator and want to add those channels to one or more of your lineups.

Script Location

All lineup editor scripts must be placed in the plugins\sdepg\lineup_editors\<lineup_id>\ directory of the lineup you wish to add it to and it must be named channels.groovy. Each lineup can have only one lineup editor script. The lineup_id value is the Schedules Direct unique lineup id for the lineup. You can find this id via your online Schedules Direct account.

Runtime Envrionment

The lineup editor is injected with a variable _lineup_, which is a Map<Integer, String[]> and is the mapping of channels to channel numbers of the lineup currently being processed. You may or may not want to use this information in your editor script.

Required Return Value

The channels.groovy script must return a value of type Map<Integer, String[]>. The key is the station id of the channel to be added to the lineup and the value is a string array of channel numbers that channel is located at on this lineup.

Example: Add Missing NFL Sunday Ticket HD Channels

So this example completes the channel generator example above. In that example, I added missing Sunday Ticket HD channels. Now I'm going to map those channels to my lineup.

import sagex.api.*

def map = [(1000): ['436'], (1001): ['437'], (1002): ['438'], (1003): ['439'], (1004): ['440']]
def itr = map.keySet().iterator()
while(itr.hasNext()) {
    def id = itr.next()
    if(ChannelAPI.GetChannelForStationID(id) == null)
        itr.remove()
}
return map

So in this example, I'm mapping station id 2000000000 to channel 436, id 2,000,000,001 to channel 437, etc.

Then I remove channels from the map that don't exist. This would happen in the case where the NFLHDx channel was already in my providers lineup. In this case, I don't want to use a self generated channel, so I wouldn't and need to remove the mapping here.

Program Generators

You can use show generators to modify or add shows to your EPG. All program generators are Groovy scripts.

To be inserted into the EPG, a script must return an object of type Collection<GeneratedProgram>. When a script returns a collection, each entry of that collection is added to the EPG database. Once programs are added, you must use airing generators to create airings of the shows in the EPG. Without airings, the shows you create with these generators will never show up in your EPG listings.

Generators are independent of each other and their execution order is not guaranteed. Last one to insert wins, the others are lost and since the order of generator execution is unknown, you need to make sure that your generators aren't overwriting each other unexpectedly.

Script Location

All show generators must be located in SageTV\plugins\sdepg\program_generators\. Every .groovy file found in this directory will be executed.

Runtime Environment

Generators use the same runtime environment as filters, so read up on that section for more details. There is one difference: Generators do not have any variables injected into the script (except for the logger: _log_. Remember, generators are to generate programs so you need to build the collection so there's no need to pass any variables to generator scripts.

Warning: External IDs for Generated Shows

Be careful when generating your own program objects. If you generate a program with an external ID that is the same as one received from Schedules Direct then you will overwrite that program object in the wiz.bin. It is assumed that users who execute program generators know what they're doing and so no sanity checks, etc. are performed prior to inserting the results of the generators into wiz.bin. If you start to see weird results in your EPG then you may want to disable your program generators.

Example: Basic Show Collection Generator

I'm personally not using program generators in my environment yet, so I'll just provide a simple Groovy script that generates a new program object. If you added this generator to your installation then this show object would be added into your wiz.bin on every EPG update.

import sagex.epg.schedulesdirect.data.GeneratedProgram

def show = new GeneratedProgram(id: 'MV0000000000', title: 'My Movie' description: 'This is a fake movie to be inserted into the Sage database.')
return [show]

Basic generator script... of note, again, be aware that the external id you generate should be unique from anything received from Schedules Direct. If not, then you're going to end up overwriting the show objects received from Schedules Direct.

Airing Generators

You can use airing generators to modify or add airings to your EPG. All airing generators are Groovy scripts.

To be inserted into the EPG, a script must return an object of type Collection<SageAiring>. When a script does, the airings are then inserted into the Sage EPG. Generator scripts are independent of each other and the order in which they are run is not guaranteed. Furthermore, if two or more generators create an airing for the same channel at the same or overlapping times then the results are undefined. In other words, the Sage EPG does not allow more than one airing on a channel at the same time and enforces this by removing older airings. Last one to insert wins, the others are lost and since the order of generator execution is unknown, you need to make sure that your generators aren't overwriting each other.

Script Location

All airing generator scripts must be located in SageTV\plugins\sdepg\airing_generators\. Every .groovy file found in this directory will be executed.

Runtime Environment

Generators use the same runtime environment as filters, so read up on that section for more details. There is one difference: Generators do not have any variables injected into the script, except for the logger, _log_. Remember, generators are to generate airings so you need to build the collection so there's no need to pass any variables to generator scripts.

Example: Fix NFL Sunday Ticket Listings

For my lineup, the NFL Sunday Ticket SD channels are properly filled in with game data, but the HD channels aren't - they all say "No Data". Because of this, if my Dolphins game is only available on NFLST on a given week then Sage will only find it in SD. Until now, I've simply created manual recordings to grab the games off of the HD channels. This is ok, but it requires I remember to set the manual recording then I also have to go and remember to create an override for SRE to monitor the game as well. Instead, I simply modify the EPG to fill in the game data for the HD channels. A slightly modified version of the script I'm using is provided below. It's modified to aid in explaining a few things.

import sagex.api.*
import java.text.*
import sagex.epg.schedulesdirect.data.Airing

def input = """
<epgmods>
    <record start="201110161300" channel="NFLHD1">
        <title>NFL Football</title>
        <episode>Carolina Panthers at Atlanta Falcons</episode>
    </record>
</epgmods>
"""

int[] findStationId(String callsign) {
    def ids = []
    Database.FilterByMethod(Database.FilterByBoolMethod(ChannelAPI.GetAllChannels(), 'IsChannelViewable', true), 'GetChannelName', callsign, true).each {
        ids.add(ChannelAPI.GetStationID(it))
    }
    return ids
}

def dateFmt = new SimpleDateFormat('yyyyMMddHHmm')
def xml = new XmlSlurper().parseText(input)
def entries = []

recLoop:
for(def record : xml.record) {
    def start = dateFmt.parse(record.@start.text()).getTime()
    def airings = Database.FilterByMethod(Database.GetAiringsOnViewableChannelsAtTime(start, start + 1, true), 'GetShowTitle', record.title.text(), true)
    for(def airing : airings) {
        if(ShowAPI.GetShowEpisode(airing) == record.episode.text()) {
            def extId = ShowAPI.GetShowExternalID(airing)
            findStationId(record.@channel.text()).each {
                entries.add(new Airing(externalId: extId, start: start, end: start + (3L * 3600000L), hdtv: true, live: true, stationId: it))
            }
            break recLoop
        }
    }
}
return entries

So this generator script parses the XML at the top of the file and for each record in the XML, it creates an airing on the given channel. This particular script will only create the airing if it finds the SD airing of the game. This ensures that Sage doesn't try to record the game more than once. So in this case, the script is adding the HD airing of the Panthers/Falcons game to channel "NFLHD1" on Oct 16 at 13:00. I'd obviously have a lot of additional records in the XML for all the HD channels, but I stripped it down to one to make the example as small as possible. So for this particular generator, I have to modify the XML each week to update the HD channel listings for football. Not perfect, but not bad either. A better version of this script might download the HD listings data from their provider's web site, parse it out, and then fill in the HD channel listings. That would be much more automated an approach, but also makes the script much more complicated for a generator example for this user guide. :)

Force EPG Update via STV

You can force an EPG update at any time from the SageTV STV by simply loading the plugin configuration screen for this plugin and selecting the Force EPG button on that screen. This button will delete all locally cached EPG data and force a complete download of all data from the Schedules Direct servers. The force EPG link on the web UI will not do this. That link will only force a reload of the cached data, unless the cache is stale then, and only then, would a download from Schedules Direct be forced.