-
Notifications
You must be signed in to change notification settings - Fork 14
FAQ
No. 1 Troubleshooting Question: If you've been using Shepherd a while but some of the data looks wrong, you want this question.
Shepherd is software for compiling an Electronic Program Guide (EPG), by managing a flock of guide data grabbers and postprocessors.
It works by calling a series of scripts that grab data from a large variety of sources, and then analysing the resulting XML data sets and determining which of the many is the most reliable. Postprocessors are used to augment the data sets with additional information (e.g. movie information from http://www.imdb.com, HDTV programming from http://www.dba.org.au etc.).
When switching between data sources, Shepherd's reconciler also tries to ensure that programme names are consistent. e.g. if you're used to recording a programme called "House" yet a different data source names it as "House, M.D.", Shepherd is smart enough to remember the original name and substitute it. No configuration is necessary to enable this; it happens automatically.
Shepherd is designed to be future proof, never requiring manual intervention once initially installed and configured. Shepherd will automatically update itself with fixes, enhancements and additional plugin components as and when they become available.
By default, Shepherd performs one full run per day, gathering guide data for the next 8 days. It also autorefreshes the current day's data once every four hours or so, to catch for last-minute schedule changes.
The shepherd_logic wiki page contains a more complete technical description of the various stages of Shepherd and how it works.
By default, Shepherd prints the EPG to STDOUT and to the file ~/.shepherd/output.xmltv.
Feel free to join our mailing list at http://groups.google.com/group/shepherd-list.
Absolutely. Shepherd is a community project and is the result of countless contributors. If you wish to enhance some functionality within Shepherd (e.g. write a new postprocessor), implement some new fancy reconciling logic, implement a new grabber or just help out in answering questions or contributing to Wiki documentation, feel free to help!
In theory, Shepherd (and its underlying components) will run on any operating system that supports Perl, as all scripts are currently written in Perl.
In practice, the developers all use Linux, so that's what is tested. There are Shepherd users running on Windows and OS X but these require more fiddling.
Shepherd will work with any PVR-type software capable of reading XMLTV (which is most of them). MythTV users get a relatively painless installation, since Shepherd can set itself up automatically, but MythTV is not a pre-requisite. Shepherd is often used on alternatives, such as Freevo.
If you are having trouble getting Shepherd to work outside the usual environment of Linux + MythTV, don't hesitate to write in to our mailing list for help.
Some of the grabbers used by Shepherd read web sites that say they don't want their data used in PVRs, but that doesn't mean it's illegal. Shepherd doesn't host or distribute data, but rather allows individuals at home to access it via their PVRs. Like a browser, it sends HTTP requests to public websites and formats the result for display in a manner appropriate to the user.
See the Installation page.
Some of Shepherd's grabbers require additional Perl modules to be installed, without which they won't function. They are listed as "optional" because Shepherd does not rely on any individual grabber to do its job; instead it draws on as many or as few of its available grabbers as necessary to acquire guide data for the time period and channels you want.
Sometimes Shepherd can do this with a single grabber. More commonly, it employees multiple grabbers and combines their results.
Generally speaking, Shepherd can perform very well even if some of its grabbers are disabled or unsupported (i.e. missing modules). However, it will probably perform more efficiently, reliably, and possibly more accurately if you can enable all of its grabbers.
A summary of Shepherd's performance can be viewed by:
tv_grab_au --status
In particular, note the last line, which tells you the percentage of wanted data it acquired. If it's less than 100.00%, Shepherd wasn't able to completely data for all your channels over the next 7 days. If you haven't enabled all of Shepherd's grabbers, you will probably benefit from doing this.
If Shepherd is grabbing 100% of wanted data, then enabling additional grabbers may be unnecessary. However, doing so will still improve Shepherd's ability to tolerate a grabber failure, may allow it to run faster and use less bandwidth, and may improve its data quality.
Yes, in the log/ subdirectory of your Shepherd installation (usually ~/.shepherd/log/).
Shepherd spaces out its downloads to avoid overloading its data source web sites. Most of the time Shepherd is running, it's not doing anything at all: it's simply pausing between downloads. Many times in the past, excessive traffic has prompted online guides to take measures to block grabbers, and its essential for the benefit of all users that Shepherd play nice.
Since Shepherd runs in the background, how long it takes makes no difference. If you'd like it to complete faster, though, you may use the "--mode" option.
By default Shepherd operates in "Quality" mode, whereby it selects grabbers based on maximizing data quantity and quality. This mode can take a long time: around 15-30 minutes under normal operating conditions, and several hours if it's being run for the first time, with no cache and needing to fill a full week.
For fastest operation, you can run Shepherd in "Speed" mode, which will make it select grabbers that complete fastest, even if their data quality is not the highest. "Efficiency" mode is a balance between the two.
In all modes, Shepherd will employ as many grabbers as necessary to completely fill the next 8 days with data.
To run Shepherd once in a particular mode:
tv_grab_au --mode speed
or to permanently set Shepherd in a particular mode:
tv_grab_au --component-set shepherd:mode=speed
If you want Shepherd to always be called as if it was sent a particular command-line option, you can use:
tv_grab_au --component-set shepherd:<option>
For example, this would make Shepherd always run as if called with the option "--grabwith=abc_website":
tv_grab_au --component-set shepherd:grabwith=abc_website
If you want to add multiple options they all need to be set with one command otherwise the final command will override any previously set commands. For example to add both "--notquiet" and "--grabwith=abc_website" :
tv_grab_au --component-set shepherd:notquiet:grabwith=abc_website
You can also set default options for any component, e.g.:
tv_grab_au --component-set abc_website:do-extra-days
To clear all default options, call --component-set with no argument. E.g.:
tv_grab_au --component-set shepherd
(Note: The Shepherd default is now eight days.)
You can ask Shepherd to try to grab however many days you like. Some channels in some regions offer up to 28 days of data; others as few as three or four. Generally, you can get at least 7 days for all but the community channels, and 14+ days for ABC and SBS.
The main reason you may not want more days is consistency. If you receive different numbers of days of guide data for different channels--which tends to happen once you push above 7 or 8 days--it's harder to spot new shows. The default of 8 days tends to keep your channels in sync, while also letting you see if the show you just recorded is also on next week.
To specify the number of days for one particular Shepherd run:
~/.shepherd/shepherd --days <n>
To set this as the default:
~/.shepherd/shepherd --component-set shepherd:days=<n>
Note: This will disable autorefresh, which is where Shepherd refreshes the current day's data once every 4 hours, unless you also specify --allowautorefresh.
Yes, but not easily. When you install/configure Shepherd, it will ask you for a region. It needs this because TV stations often use different schedules in different locations: what a channel is broadcasting at 9pm in Melbourne is no guarantee it's broadcasting the same thing in Broken Hill.
Most of the time, users only require guide data for channels within their own region. But if you have a satellite dish or can otherwise receive channels from multiple regions, you might want Shepherd to collect data on these.
The only 100% sure way to manage this is to run multiple independent versions of Shepherd, and feed their output files one at a time to your PVR (which can hopefully combine them). This is a little tricky, because Shepherd forcibly runs from ~/.shepherd/: you cannot change this, and the only workaround is to install and run it as a different users, or alter the HOME variable. See the below FAQ for details.
This is necessary because Shepherd and its grabbers use channel names to identify channels, and it will not distinguish between "Prime" in one region and "Prime" in another. Invoking Shepherd a second time with a different region code will cause it to see that it has Prime data cached from the first run and return that, rather than fetching the potentially unique data from the second region.
Even if you only want channels that have unique names, and do not exist in both regions, you need to setup multiple instances of Shepherd. If you don't, Shepherd will encounter a range of minor problems such as refusing to run for the second region because it thinks it has successfully completely too recently.
There are a number of users who would benefit from Shepherd handling multiple regions better, but the single-region assumption is quite ingrained in Shepherd and difficult to fix. Sorry.
Users grabbing data from regions in different timezones should also ensure that each Shepherd instance has the correct TZ variable set.
No. Shepherd always expects to use the same configuration file (usually ~/.shepherd/shepherd.conf).
You can use tricks like changing the environment variable HOME, to run multiple installs of shepherd. But it's not very efficient, in that it will lead to a fair bit of redundant downloading, but otherwise it is a good solution for someone wanting data from multiple regions.
HOME=/first/directory /first/directory/.shepherd/shepherd
HOME=/second/directory /second/directory/.shepherd/shepherd
Another similar way is to 'mv' the .shepherd directory.
mv ~/.shepherd ~/.shepherd.1
mv ~/.shepherd.2 ~/.shepherd
~/.shepherd/shepherd
mv ~/.shepherd ~/.shepherd.2
mv ~/.shepherd.1 ~/.shepherd
~/.shepherd/shepherd
Multiple versions of Shepherd will produce multiple output.xmltv files, which you will need to pass to your PVR. In MythTV, for example, you should not use the default cron job (which invokes mythfilldatabase rather than Shepherd), but rather create your own cron jobs to run each instance of Shepherd, then feed their output to MythTV as described here.
Because Shepherd employs many different grabbers, the first step is to figure out where the dodgy data came from. If you're interested in a particular time, you can use the "--ancestry" option to see how Shepherd put together guide data for a particular time. For example, to look at the ancestry of data for next Tuesday from 10:30pm - 11pm:
tv_grab_au --ancestry "tuesday 10:30pm+30"
This will print out relevant guide data obtained during Shepherd's last successful run from each component. What you want to do is find the earliest point at which the timestamps are wrong.
- If the data looks wrong in a grabber, it's either a problem with the grabber itself or the grabber's data source.
- If the data looks fine in the grabber, but is bad in the output of a reconciler, postprocessor, or Shepherd itself, it's a Shepherd problem.
Either way, armed with this information you should be able to get further help from the mailing list.
The --ancestry option requires the Perl module File::Find::Rule. You can install it with the command:
sudo cpan File::Find::Rule
or, for Debian-based distributions:
sudo apt-get install libfile-find-rule-perl
Some shows have titles ALL IN CAPS, and often including unwanted info like "LIVE:" or "SEASON FINALE."
You have EIT enabled in MythTV, which is overriding the guide data supplied by Shepherd. EIT is an over-the-air program guide broadcast by the TV stations themselves. In an ideal world, this would mean you don't need Shepherd at all. Unfortunately, so far EIT data seems to be quite unreliable: show names change, times are inaccurate, descriptions are missing, etc. It is therefore recommended that you disable EIT in MythTV.
Shepherd asks you to allow it to auto-disable EIT in MythTV during the --configure process. You can check or re-do this with:
tv_grab_au --configure-mythtv
Note that EIT can be enabled on a per-card and per-channel basis, so it's important to be turned off everywhere.
This is often due to users having EIT enabled in MythTV, which overwrites the Shepherd-supplied names. Please see the answer above, particularly if their name changes to something like "LIVE: (your show)" or is ALL IN CAPS.
In rare cases a show name can change simply because the datasource renames it and the new name is so different that Shepherd thinks it's a genuinely different show. (Subtle name changes, by contrast, are overridden by Shepherd to avoid breaking your recording rules.)
Shepherd's reconciler attempts to keep show names consistent, to allow you to reliably set recording rules based on show names. To do so, it may rename shows like "House, M.D." to "House" (if that's what it saw in the past), "Home and Away Catch-Up" to "Home and Away," and so on. This is necessary as show names frequently contain variations like these even within the same datasource; they also tend to vary from datasource to datasource.
Unfortunately sometimes Shepherd settles on a title that's not the one you want -- naming every "Home and Away" episode "Home and Away Catch-Up," for example, rather than the other way around. You can see whether this is the case by inspecting ~/.shepherd/reconcilers/reconciler_mk2/reconciler_mk2.alt_title.log.
There is no simple way to specify a preferred title. (Although it is a much-requested feature.) If you cannot tolerate the titles, the easiest option is to delete the reconciler's config:
rm ~/.shepherd/reconcilers/reconciler_mk2/reconciler_mk2.storable.config
This will make Shepherd forget its old standards for show names and start anew. Be aware that this may break many of your recording rules (if, for example, you have a rule to record "House" and it now begins to show up as "House, M.D."). It would also be a good idea to make sure that the first time you run Shepherd after deleting the reconciler's config, you have high-quality grabbers enabled, as Shepherd will use these to build a new list of show name standards.
Alternatively, there are Perl libraries for editing Storable files, including http://www.lothosoft.ch/thomas/perl-storableedit/ - using such a program, you can edit the hash under title_xlate_table for the title you no longer want to use set the 'translation' field to the new title you want.
Generally it's best to let Shepherd decide which grabbers to use, and in which order. One of its main benefits is its fault-tolerance: guide data sources tend to be fragile and can stop working unexpectedly, but Shepherd will work regardless. Relying on a particular grabber always being there is, unfortunately, not safe.
Additionally, Shepherd makes reasonably intelligent decisions about which grabbers it would be best to use, and in which order.
If you have a general preference for speed over quality, or vice versa, you can control this via the "--mode" option.
To specify an exact order for grabbers, use the "--grabwith <grabber/s>" option. Shepherd will run the specified grabber(s) first, then others as needed to fill remaining holes in the data. For example:
tv_grab_au --grabwith oztivo,sbsnews_website
When it's been more than 4 hours but less than 22 hours since the last run, Shepherd will switch into autorefresh mode, gathering data for the current day only. It will output the result via STDOUT, as usual, and create the file ~/.shepherd/refresh.xmltv. This allows Shepherd to catch last-minute schedule changes. It tends to complete very quickly (for Shepherd), since even the slower grabbers are usually able to quickly verify if there have been any time or title changes since they last ran.
If you don't wish Shepherd to enter autorefresh mode, send the --noautorefresh option.
Shepherd will not enter autorefresh mode if you specify a number of --days, since this would require overriding your preference. But if you are happy for Shepherd to override it during autorefreshes only, send --allowautorefresh.
Autorefresh does not touch the default output.xmltv file, so functions such as --refill-mythtv will use your last full successful run, not any subsequent autorefresh. You may wish to override this if you want to be able to rely on output.xmltv always containing the latest data, regardless of whether it's a single day's data or a week's. If so, use --output. For example, to always output into output.xmltv, even autorefreshes:
tv_grab_au --output ~/.shepherd/output.xmltv
In that case, you would probably want to set it as a default option, i.e.:
tv_grab_au --component-set shepherd:output=output.xmltv
Or to refill MythTV will the latest autorefresh data, rather than the latest full run:
tv_grab_au --refill-mythtv --output ~/.shepherd/refresh.xmltv
Yes, via Information Center -> System Status -> Listings Status.
If you've just finished installing, Shepherd may be running right now. It can take up to three or four hours, especially the first time.
During configuration, you should have answered "yes" to the question, "Would you like Shepherd to auto-configure MythTV?" This registers Shepherd as MythTV's default grabber, so that mythfilldatabase will invoke Shepherd as required. It also installs /usr/bin/tv_grab_au as a symlink to Shepherd, and creates a cron job to run Shepherd (via mythfilldatabase) once per hour.
If you didn't answer "yes," or you're not sure it worked, re-do it. Most problems relating to integration between Shepherd and MythTV can be solved with this:
tv_grab_au --configure-mythtv
Shepherd will run within a few minutes of you doing this. Please wait to see if it works by itself.
To see if Shepherd is currently running, look for "*** IN PROGRESS***" at the bottom of output from this command:
tv_grab_au --history
To be absolutely sure, you may wish to scan your system processes with:
ps -eopid,user,cmd | egrep "(mythfill|tv_grab_au|shepherd)"
If Shepherd is running, just wait. It may require several hours to complete its first full run.
If Shepherd doesn't seem to be running, see whether mythfilldatabase is running by inspecting the log file /var/log/mythtv/mythfilldatabase.log. If it's not, either --configure-mythtv failed (in which case it should have produced an informative error message), or you're impatient (i.e. you haven't waited a few minutes).
If mythfilldatabase is running but Shepherd is not: Firstly, be aware that mythfilldatabase may appear to be hung (or frozen), because while Shepherd runs it doesn't output anything. (It also suppresses Shepherd's output.) Some users see mythfilldatabase not printing anything to the screen for a while and assume it has hung, when in fact it is just quietly waiting for Shepherd to complete. So do follow the above advice to see whether Shepherd is in progress, rather than assuming it is freezing.
If mythfilldatabase is definitely running but Shepherd is definitely not, the answer probably lies in the output of --configure-mythtv. Inspect it carefully for an error message.
Shepherd updates its log file as it runs (with some buffering), so you can generally follow its progress with:
tail -f ~/.shepherd/log/shepherd.log
A complete history of Shepherd's runs are stored in ~/.shepherd/log/. If this directory is empty, Shepherd has never been run (by that user). If there are log files, browsing the latest few may tell you what, if anything, went wrong.
Other useful commands to check Shepherd's basic health:
tv_grab_au --status
tv_grab_au --history
tv_grab_au --check
tv_grab_au --show-channels
If Shepherd seems to be running OK, and reports that it has successfully run, the issue may be that data is not getting into MythTV. See below.
First, see above. Because mythfilldatabase suppresses Shepherd's output, it can appear to be hung when it's not. When you run Shepherd directly, you see lots of nice output that reassure you it's doing something. So follow the advice above to verify whether Shepherd is in fact working.
Most integration problems can be solved with:
tv_grab_au --configure-mythtv
Then reload Shepherd's data into MythTV:
tv_grab_au --reoutput-mythtv
If you need to debug further, run mythfilldatabase manually and monitor Shepherd's output via its log. You will need to open two terminals, one for each command:
mythfilldatabase
tail -f ~/.shepherd/shepherd.log
Note that Shepherd's normal behavior is to prevent itself from running too frequently, in order to avoid overtaxing guide data resources. This is not an error. There is no need to override this behavior, as it means you already have a nice, fresh ~/.shepherd/output.xmltv file. Unless there is something wrong with that output (e.g. zero size), forcing Shepherd to rebuild it just wastes your time and bandwidth.
On some systems without a mail transport agent (like sendmail), a bug in cron can cause silent failure of jobs. This can be solved by piping the output of your cron job to /dev/null with ">/dev/null 2>&1" (or maybe using --quiet option :-)
If after all this you cannot figure out what's going on, and Shepherd is producing guide data but you can't get it into MythTV, you can modify the default cron job (crontab -e) to this:
51 * * * * tv_grab_au && mythfilldatabase --update --file --sourceid 1 --xmlfile /home/<user>/.shepherd/output.xmltv
This will run Shepherd once per hour, then manually feed the output file to mythfilldatabase. It is a somewhat suboptimal solution, however, because: a. On MythTV systems that shut down, mythwelcome will not stall shutdown while Shepherd is running, meaning it may often terminate halfway through a. MythTV may or may not continue to invoke mythfilldatabase according to its own schedule, producing misleading errors; a. It causes MythTV to process Shepherd's output file every hour, even though it doesn't change that often, which is a reasonably significant waste of system resources on older hardware a. It does not ensure that tv_grab_au is symlinked to where you think it is, leading to situations where, for example, Shepherd periodically runs in ~mythtv/.shepherd/ while the confused user wonders why there's no output in ~user/.shepherd/.
To ensure Shepherd is installed as the user you think it is:
ls -l `which tv_grab_au`
Yes. You might want to manually reload Shepherd data into MythTV if you think Shepherd ran successfully but for some reason the data isn't showing up in MythTV--perhaps your channels weren't configured correctly or something. So after changing things, you can reload the data with:
tv_grab_au --reoutput-mythtv
Or if that doesn't work, try:
tv_grab_au --refill-mythtv
... which accomplishes the same thing, only using mythfilldatabase's --file option, rather than sending data directly.
You can also use these commands to modify Shepherd's output file (~/.shepherd/output.xmltv) and see the changes in MythTV, if you ever have the urge to tweak (or test) your guide data.
That's fine, so long as it also says you have guide data for at least the next few days. Shepherd runs hourly by default, but only updates MythTV once a day. So most of the time, your most recent run won't have inserted new data.
If, however, you are seeing a "FAILED" message, or you are running low on guide data, something may be wrong.
A default installation will create an hourly cron job, running Shepherd once per hour. Unless it's been a reasonable period of time since the last successful run, though, Shepherd will simply exit. Shepherd will resist running more frequently in order to avoid putting undue strain on TV guide datasources.
If you prefer, you can edit your crontab to run Shepherd whenever you like. However, there's no downside to running Shepherd frequently (since it will just exit unless it's due for more data), whereas running Shepherd infrequently may lead to dwindling guide data if your system misses its run a few days in a row (e.g. due to powering off).
If it's been 22 hours since its last successful full run, Shepherd will attempt to gather guide data for the next 8 days. If it's been less than 22 hours but more than 4 hours, Shepherd will perform an "autorefresh", gathering data for just the current day.
Make sure mythfilldatabase (not Shepherd) is invoked with the "--update" option, so it will not add any missing channels to your video sources. (This can be an issue if you have video sources that receive different sets of channels, for example Free-to-Air TV and Pay TV.)
If you have setup Shepherd the default way, you can add the "--update" option to mythfilldatabase with:
crontab -e
Look for "mythfilldatabase" and insert "--update" immediately after it, so the line looks something like this:
44 * * * * mythfilldatabase --update
When mythfilldatabase runs (SVN release of mythtv, mythtv 0.21 and later), I can't see what Shepherd is doing
Starting with MythTV 0.21 and SVN release around February 2007, mythfilldatabase (by default) adds '--quiet' onto the command-line when calling the tv_grab_au script. To negate this, you can use Shepherd's "--notquiet" option. You can make this a default by:
tv_grab_au --component-set shepherd:notquiet
To disable this, use:
tv_grab_au --component-set shepherd
This will try to compare your Shepherd channels to your MythTV channels:
tv_grab_au --show-channels
If it doesn't look right, re-configure:
tv_grab_au --configure
If Shepherd is not able to access your MythTV (e.g. it's a remote backend), you will need to manually check that the XMLTV IDs you assigned to channels in Shepherd match those you assigned in MythTV. It doesn't matter what each XMLTV ID is, just that they match. For example, if in the mythtv-setup Channel Editor you have "ABC Melbourne" set to an XMLTV ID of "abc.free.au", you should also specify this as the XMLTV ID for the ABC channel in Shepherd. You can do this via --configure, as shown above.
MythTV versions 0.27 and later
MythTV dropped its timezone setting in 0.27, which is no longer accessible in mythtv-setup. So this is a moot question.
The change may create a problem for users upgrading from an earlier version of MythTV, however, since whereas MythTV 0.26 assumes guide data with no timezones to be in local time, v0.27 assumes it to be in UTC time. It is essential that the Shepherd component augment_timezone be able to add timestamps to all show data. This will happen automatically if your MythTV backend lives on the same machine as Shepherd and Shepherd is up to date, so there should be no need to do anything, but to verify, run tv_grab_au --update and make sure everything is at the latest version.
If you do find that your guide data is out by 8-11 hours after upgrading from v0.26, look at ~/.shepherd/log/shepherd.log (or whichever log file belongs to the most recent full run) under "augment_timezone".
MythTV versions 0.26 and earlier
Usually it doesn't matter. Shepherd will look up MythTV's timezone setting, compare this to your system clock, and ensure everything lines up.
Note that it can only do this if the 'augment_timezone' postprocessor can run which in turn requires access to your MythTV database. Check that it is running with:
tv_grab_au --status
and shows up as 'Enabled' and 'Ready' and has a status indicating it successfully processed data.
The ideal timezone, though (the one that requires no extra adjustments from Shepherd) is "None." If Shepherd is not on the same box as your MythTV, you must set your timezone to "None."
Shepherd assumes that your machine's timezone has been set correctly for the region you've selected. To check this is the case, make sure these commands both display the correct time and timezone for the region you're using Shepherd for:
perl -e 'use POSIX; print POSIX::strftime("%z %x %X %Z\n", localtime(time));'
date "+%z %x %X %Z"
Setting your machine's timezone
- Debian-based distributions can use the 'tzconfig' command (run as root).
- Gentoo users should copy the correct timezone file to /etc/localtime as per the [handbook instructions]
- The TZ environment variable could also be set to correct any differences. Add in ~/.profile (to make it user specific), /etc/profile (to make it machine wide), or just before you execute shepherd. eg:
TZ="Australia/Brisbane"
export TZ
You will need to log off and back in to pick up the change.
Check your system timezone info is up to date
Some grabbers (eg. Yahoo7widget) correct for badly stored time zone data on the remote server by using your systems general time zone information. In the case of yahoo7widget, this uses the "Australia/Melbourne" file. If your system time zone files are out of date and the cut over dates for Daylight Savings time has changed since you last updated, your guide data may appear to be out by an hour.
To check if your system files are up to date, confirm the following command gives the correct time for Melbourne.
perl -e 'use POSIX; $ENV{TZ}="Australia/Melbourne"; print POSIX::strftime("%z %x %X %Z\n", localtime(time));'
To update your system files:
- Debian based distributions should use 'apt-get install tzdata'
- Gentoo distributions should use 'emerge timezone-data'
- Windows users should refer to: http://support.microsoft.com/kb/914387
Timezone problems with Mythweb
A common timezone problem (unrelated to Shepherd) is when times are out in MythWeb, but correct in the rest of MythTV. This is because PHP5, unlike PHP4, requires you to explicitly set a timezone in php.ini:
In a terminal, type:
sudo gedit /etc/php5/apache2/php.ini
Within gedit, Search > Go to line > 603. Change the line from:
;date.timezone =
to
date.timezone = Australia/Adelaide
... or whatever is appropriate for your location. Make sure to remove the semi-colon (comment)!
Save, exit gedit, and restart apache:
sudo /etc/init.d/apache2 restart
Your MythWeb times should now match what is displayed within MythTV.
This is a timezone problem, and may have many causes: the data may be wrong at its source, or be being misinterpreted by Shepherd or MythTV.
There are generally two categories of timezone problems, identifiable by the symptoms: data that is out by around 10 hours means your system is interpreting local times as UTC times, or vice versa; while data that is out by 1 or 2 hours is usually the result of Daylight Savings Time changes.
For MythTV 0.26 and earlier, this is usually caused by configuring MythTV to have a fixed timezone such as +1000. As discussed above, it's best to set MythTV's timezone to "None," which makes it interpret all times as local. If you don't do this, Shepherd needs to figure out what timezone MythTV expects show data to be in and adjust accordingly. Shepherd can do this, using its augment_timezone component, but if this component fails or is unable to access the MythTV database for some reason, your times may be out.
For MythTV 0.27 and later, this probably means that the augment_timezone component isn't adding timezones to your data, which is essential in 0.27+. You can confirm this by looking at ~/.shepherd/output.xmltv and seeing if show times look like "20130426100000" (bad) instead of "20130426100000 +1000" (good). The reason you're not getting timezones is probably that you don't have an up-to-date Shepherd installation: in particular, you may be missing the perl module Sort::Versions. Run tv_grab_au --update to confirm, and (if you're on Debian/Ubuntu) apt-get install libsort-versions-perl to fix.
If that's not it, there is probably another reason why augment_timezone can't figure out which version of MythTV you have. One reason might be that you run Shepherd on a remote system. In that case, in ~/.shepherd/log/shepherd.log (or whichever log file corresponds to Shepherd's most recent full run), you'll see something like this:
: ERROR: Could not find info to establish database connection to MythTV.
: ERROR: Missing essential DB connection info.
: No valid response from MythTV.
: Assuming MythTV's timezone is "None".
: *** If this is wrong, guide data may be in wrong timezone! ***
:
: - Target timezone is "None". No need to do anything.
And to fix, you will need to manually instruct augment_timezone which zone to stamp its times with. You can do this with the command to set default options, e.g.:
tv_grab_au --component-set augment_timezone:timeoffset=Auto
The above tells augment_timezone to try to figure out your timezone based on its local system. If this isn't the same as your MythTV box, it will probably get things wrong. You can specify a timezone by replacing "Auto" in the above command with something like "+1000"; however, bear in mind this may become inaccurate during Daylight Savings Time changes.
Can be the same cause as above, but also occurs following changes in Daylight Savings Time. It is more common for users in Queensland and Western Australia, mainly because datasources in NSW and Victoria tend to forget you exist. Several times in the past, datasources have supplied incorrect data for Queensland and Western Australia for a few days following Daylight Savings Time changeover in NSW, until somebody finally notices and fixes it. You can inspect this for yourself by visiting online guides such as Yahoo! and YourTV.
Aside from waiting, all you can really do is email the mailing list to alert others to the problem. If it persists, Shepherd developers will sometimes hack in a temporary fix, but this can cause new problems when the problem is fixed at its source.
(Various tricks have been proposed to deal with this, such as checking to see a show such as the News is always on at a set time. But special events can cause any show to unexpectedly move, causing more problems.)
Either way, to figure out where the problem lies, use the --ancestry option to inspect a sample of your incorrect data. See:
tv_grab_au --ancestry help
For example, you might run tv_grab_au --ancestry "today 6pm+10" to see if the 6PM News bulletins are listed at the correct times. Always look up guide data for the future, which can be cross-checked.
If the data looks fine here but wrong in MythTV, you most likely need to set MythTV's timezone to "None" in mythtv-settings. If the --ancestry data itself looks wrong, the problem is with Shepherd, one of its grabbers, or in the datasource: please contact the mailing list including your --ancestry output.