mcupdate.exe crashing when running standard tasks after import

[Carl Fosler]

Steve,

 System had been working fine for months when on 10/18/20 the load started to fail.  When I manual ran it, after it loaded the mxf file into the database, WMC would generated a popup saying that mcupdate had failed.  The output from EPGC would show NO errors.  I had been running fixpack 24, but I upgraded to 27 and that didn't fix the problem.  What I found was that guide was being updated, but the scheduling was messed up.  Mcupdate.exe would run for hours until I killed it, and WMC would only schedule 14 of 96 shows.

 After spending a huge amount of time tracking down the error and possible solutions, I discovered that the mxf file was fine.  The problem was when EPGC ran the WMC tasks after importing the xmf file.  Once I turn off the "run standard tasks after import" feature, EPGC would load the file, and exit without error, I could then start up the two database task without error (I do PvrSchedule first, then DoReindexSearchRoot).  Mcupdate would then correctly do the update (finish in 5 minutes or so) and schedule all of the shows.

 From error run

LoadMXF message: Loading... 98%
LoadMXF message: Loading... 100%
Windows Media Centre import utility LoadMXF has completed: exit code 0
Running Windows Media Centre standard task 'Private job reindex'
Windows Media Centre standard task has started
Running Windows Media Centre standard task 'WMC update PVR schedule'
Windows Media Centre standard task has started
<c> Finished - output 6064 EPG entries
<c> Exiting with code = 0</c></c>

Carl

[Steve Bickell]

Hi Carl,

What happens if you run the 2 database tasks in the same order as EPGC ie reindex then PVR schedule. Does that fail?

Steve

From: Carl Fosler [mailto:crf@users.sourceforge.net]
Sent: Tuesday, 20 October 2020 7:18 p.m.
To: stevebickell@xtra.co.nz
Subject: [epgcollector:discussion] mcupdate.exe crashing when running standard tasks after import

Steve,

System had been working fine for months when on 10/18/20 the load started to fail. When I manual ran it, after it loaded the mxf file into the database, WMC would generated a popup saying that mcupdate had failed. The output from EPGC would show NO errors. I had been running fixpack 24, but I upgraded to 27 and that didn't fix the problem. What I found was that guide was being updated, but the scheduling was messed up. Mcupdate.exe would run for hours until I killed it, and WMC would only schedule 14 of 96 shows.

After spending a huge amount of time tracking down the error and possible solutions, I discovered that the mxf file was fine. The problem was when EPGC ran the WMC tasks after importing the xmf file. Once I turn off the "run standard tasks after import" feature, EPGC would load the file, and exit without error, I could then start up the two database task without error (I do PvrSchedule first, then DoReindexSearchRoot). Mcupdate would then correctly do the update (finish in 5 minutes or so) and schedule all of the shows.

From error run

LoadMXF message: Loading... 98%
LoadMXF message: Loading... 100%
Windows Media Centre import utility LoadMXF has completed: exit code 0
Running Windows Media Centre standard task 'Private job reindex'
Windows Media Centre standard task has started
Running Windows Media Centre standard task 'WMC update PVR schedule'
Windows Media Centre standard task has started
<c> Finished - output 6064 EPG entries
<c> Exiting with code = 0</c></c>

Carl

---

mcupdate.exe crashing when running standard tasks after import https://sourceforge.net/p/epgcollector/discussion/1125946/thread/6f429cfa3c/?limit=25#98a0

---

Sent from sourceforge.net because stevebickell@xtra.co.nz is subscribed to https://sourceforge.net/p/epgcollector/discussion/1125946/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/epgcollector/admin/discussion/forums. Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Guy M]

Hey Steve,

Having the same issue here. WMC also immediatly crashes if I try to record, add scheduled recording, or cancel a scheduled recording.

I did a complete ehome folder wipe and WMC reinstall, and the issue remains.

Oh, and it started immediatly after an EPGC run. My guess is there is some malformed data in the zap2xml download...

Anyway, I just wanted to provide some additional data from my side.

[Steve Bickell]

Try doing what Carl did and don't get EPGC to run the WMC task. Run them
manually after EPGC finishes.

From: Guy M [mailto:fluxtx@users.sourceforge.net]
Sent: Wednesday, 21 October 2020 11:44 a.m.
To: stevebickell@xtra.co.nz
Subject: [epgcollector:discussion] mcupdate.exe crashing when running
standard tasks after import

Hey Steve,

Having the same issue here. WMC also immediatly crashes if I try to record,
add scheduled recording, or cancel a scheduled recording.

I did a complete ehome folder wipe and WMC reinstall, and the issue remains.

Oh, and it started immediatly after an EPGC run. My guess is there is some
malformed data in the zap2xml download...

Anyway, I just wanted to provide some additional data from my side.

---

mcupdate.exe crashing when running standard tasks after import
https://sourceforge.net/p/epgcollector/discussion/1125946/thread/6f429cfa3c /?limit=25#dcec

---

Sent from sourceforge.net because stevebickell@xtra.co.nz is subscribed to
https://sourceforge.net/p/epgcollector/discussion/1125946/

To unsubscribe from further messages, a project admin can change settings at
https://sourceforge.net/p/epgcollector/admin/discussion/forums. Or, if this
is a mailing list, you can unsubscribe from the mailing list.

[Carl Fosler]

Steve,

Yes, when I switch the order, it fails.

Guy,

 Here the commands I'm using.  I'm pretty sure that EPGC doesn't do the "/WAIT", but it not clear if it makes a difference.  On my system, the Schedule only takes a minute or two, but the Indexing can take 5 to 10 minutes.

 START /WAIT %SystemRoot%\ehome\mcupdate.exe -PvrSchedule
 START /WAIT %SystemRoot%\ehome\ehPrivJob.exe /DoReindexSearchRoot

 BTW Steve, on failure, mcupdate.exe will schedule a few programs, but then seems to go into a loop, taking up 50% of my cpu cycles.  After killing mcupdate.exe, redoing the pvr and indexing (in the reverse order) does not fix the problem.  The only way that I found to fix the problem was to rebuild the WMC database (via epg123 utility).

Carl

[Guy M]

Carl - I also noticed a 'loop' of sorts, the task scheduler's 'PvrRecoveryTask' appeared to run over and over again continuously... I adjusted the task to stop it from doing that - but that appears to be the reason for my 'recording' difficulties (and maybe even the mcupdate crashes?). After setting it back to original config, it 'looped' for a while, but then eventually stopped on its own, and I was able to set recordings again.

BTW, this is what sent me down the rabbit hole in the first place (along w/ seeing the mcupdate failure popups) - I went to 'Scheduled Recordings' and wasn't able to navigate the list (kept hearing the WMC scrollover sound and focus kept being switched to the menu) - this stopped when I stopped 'PvrRecoveryTask'.

Steve - Carl's steps appear to work after undoing my change to the 'PvrRecoveryTask'. Note that I am on my second iteration of a full WMC reset (not including EPG123 instigated "resets" and my own separate hamfisted attempts at various 'fixes'). That is to say I manually added back the channels and scheduled recordings (wasn't sure if I could simply load just that one file back from the 'mcupdate -b' without also doing the lineup and subscriptions as well).

What I should have done initially is slow down and take a look here first, then patiently go from there...

[Steve Bickell]

Carl or Guy - can I have a look at the EPGC log when you try it with EPGC running the tasks.

Looking at the code EPGC first tries to run ehPrivJob.exe /DoReindexSearchRoot (without the wait). If that fails to run then it tries to run the same job again but uses Task Scheduler this time using schtasks.exe /run /tn etc etc.

It does the same for mcupdate pvrschedule.

So I'm wondering if, for some reason, the first attempt indicates it failed even though it may not have and the job is initiated again using Task Scheduler.

Anyway the log should show what is happening.

In the meantime just don't use EPGC to run these tasks.

[Carl Fosler]

Steve,

 In my initial posting, I include the last lines of the log of a run that failed, in it EPGC is doing the two tasks.  I can't see any errors being return.  EPGC only takes two minutes to run my configuration, the DoReindexSearchRoot takes 4 or 5 when I start it, so EPGC can't be doing a wait on that job, so it return code wouldn't reflect any error condition.

 OK, the order of the two tasks might not be the main problem.  When I first discovered the problem, I was pulling 14 days of shows, but to simpify my debugging, I reduced it to 7 days.  It was with only 7 days that I found switching the task order worked.  Once I when back to 14 days, I discovered that even with the change order, the problem came back.  After much work, I was able to figure out that it data from November 1 that causing the problem.  I was able to load data to Nov 1 1:00am without a problem, but when I try up to 2:00 am, then the mcupate.exe problem shows up.  I loaded the 2:00am load, but didn't start the two tasks.  I then pulled up the guide.  What I found was that 14 channels showed data, 4 channels showed "NO data available", and 9 channels show only the blue background.  To me, it looks like there something wrong with the guide entries, and once the schedule starts running, it finds the corrupted guide and can only match a few programs.

 At first, I thought that the Novermber 1 date was the problem, but I found that it would load the schedule when there were 10 midnight shows.

 I have attached the xmltv file with the error.  I have looked at the entries for November 1, but I can't see anything wrong with them, maybe a binary character?

 Carl

[Guy M]

Nov 1st you say? Then likely a glitch having to do with the handling of "daylight saving time." Conflicting timestamps or the like? This lines up with Friday's DL working here then Mon's DL causing the issue, which included Nov 1st. Bet I wouldn't have seen any issue at all if my DL was Tue or Wed instead of Monday - in fact, my currently working 14-Day DL is from Wed the 21st.

Wish I still had Monday's xml or mxf to take a look at, alas they have been overwritten. Maybe Carl still has his?

I am back to getting 14 days (w/ manual end tasks), and WMC is running great. Keeping my hands off it for the time being...

[Guy M]

Additional note, if I go into the guide in WMC and advance to the Nov1 1AM area, the guide gets glitchy- sporadically showing no data for some channels through the entire schedule, but resolves when I close and reopen the guide within WMC. And just like the zap2it web site, it has 12:30AM-1AM-1:30AM-1AM-1:30AM-2AM (reflecting the 1hr gain from DST).

[Jim Koch]

Mine quit updating when the scheduled update ran on Monday as well. WMC had the same message that the update failed. I ran the EPG Collector manually, the listings were populated, but it stopped at the line "LoadMXFmessage: Loading...99%". It never completed. If I close the program manually, the listings dissapeared and the WMC message is displayed again. I ran the TPG Collector again with the same results. So this time I just minimized the program so the listings would be saved. It also kills scheduled recordings except manually created ones when the listings are not there. Steve's program was working flawlessly since March. I will close the window Sunday night and see what happens when the scheduled update happens on Monday. The update worked fine on another computer of mine.

[Carl Fosler]

Steve,

Looks like Guy is right, is something to do with DST.

Here are two "Paid Programming" entrie from the bad xml file. Note that the stop date/time is before the start. The problem is that EPGC generates a negative duration in the mxf file (see ScheduleEntry down at the bottom). In looking at the bad mxf file, I find 8 -1800 durations, all that start at 05:30 or 05:32. I see other with 0 durations, but I'm guessing they don't cause a problem. I think all the times from zapit are in GMT, I'm in Eastern, so 4 hours off GMT.

Carl

<programme start="20201101013200 -0400" stop="20201101010200 -0500" channel="I4.1.19578.zap2it.com">
    <title lang="en">Paid Programming</title>
    <desc lang="en">Paid programming.</desc>
    <category lang="en">Series</category>
    <length units="minutes">30</length>
    <url>https://tvlistings.zap2it.com//overview.html?programSeriesId=SH00000001&amp;tmsId=SH000000010000</url>
    <episode-num system="dd_progid">SH00000001.0000</episode-num>
    <previously-shown />
    <rating>
        <value>TV-G</value>
    </rating>
</programme>

<programme start="20201101013000 -0400" stop="20201101010000 -0500" channel="I5.1.20367.zap2it.com">
    <title lang="en">Paid Programming</title>
    <desc lang="en">Paid programming.</desc>
    <category lang="en">Series</category>
    <length units="minutes">30</length>
    <url>https://tvlistings.zap2it.com//overview.html?programSeriesId=SH00000001&amp;tmsId=SH000000010000</url>
    <episode-num system="dd_progid">SH00000001.0000</episode-num>
    <previously-shown />
    <rating>
        <value>TV-G</value>
    </rating>
</programme>

  <Program id="prg9685" uid="!Program!SH00000001:0000" title="Paid Programming" description="Paid programming." keywords="k3,k301" originalAirdate="1970-01-01T00:00:00" isSeries="1" series="si46" isGeneric="0" isSpecial="0" isMovie="0" isSports="0" isNews="0" isKids="0" />

  <ScheduleEntry program="prg9685" startTime="2020-11-01T05:32:00" duration="-1800" />

[Steve Bickell]

I wonder if it would be OK to just add 1 hour to the duration if it is not > 0?

[Steve Bickell]

Actually thinking about it there will be a problem when you go back the other way in 6 months time. The durations over the change will all be 1 hour too long.

[Steve Bickell]

The program tag might be the answer as it has the time offset from GMT. Adjusting the duration by the start time offset - the end time offset would work.

In your example above -4 - -5 will add an hour to the duration and in 6 months time -5 - -4 will reduce the duration by an hour.

I'll take a look at changing the code.

[Guy M]

Not sure how WMC does/would handle it, but maybe if its <0 (i.e. negative) just make the duration 0?? Then only the offending entries are affected? Or maybe only do the offset fix if the offsets differ? Bah, if the offsets are the same, the formula will equal zero anyway - so not the best input from me :)

[Carl Fosler]

I have been using EPGC before DST in the spring and it didn't blow up then, so I'm guessing that EPGC added an extra hour in the spring and WMC was able to handle it. It only shown a problem now since we lose an hour and for 30 minute shows, the duration goes negative. I'm guessing that that why the GMT offset is included in the program tag, it only needed twice a year. There are some area of the US where DST doesn't exist, so I'm guessing that zap2it handles that via the GMT offset.

Carl

[Steve Bickell]

Try the attached version.

It creates the correct duration for programmes over the DST change.

[Carl Fosler]

Steve,

No, that didn't fix it.  Sourceforge will not allow me to post xml and mxf file, so I will try just the xml.

Carl

[Carl Fosler]

Steve,

 I when and hand edit the mxf file, changed the -1800 to 1800.  Then loaded it with loadmxf, but I got the mcupdate error.  I when and pull up the guide, and I now see the guide display 12:30, 1:00, 1:30, 1:00, 1:30, 2:00am along the top for Nov 1.  Half of the programs are loaded in the first 1:00am, the second half are loaded in the second 1:00 slot.  The missing data is filled with "No data available", and the rest of the guide is filled out.  The schedule is messed up, so that didn't fix all of the problems.

 I guess you don't have DST down there, so this is something that doesn't happen to you.

Carl

[Steve Bickell]

Are you saying the MXF file still had negative durations in the schedule entries with the version I posted above.

It didn't when I tried the new version with the xml file you first posted.

We do use DST - changed to summertime about 3 weeks ago.

[Carl Fosler]

Steve,

OK, I was right and wrong.  I made another attempt by hand editing the mxf file, this time I delete all the ScheduleEntry with the -1800 values, and I also found a number of entries with duration of "0", which I deleted as well.  ALL of ScheduleEntry for the shows in the DST zone are now gone and then reloaded it.  This time it loaded without error, and I didn't get an mcupdate error.  When I look at the guide, all the shows in the DST zone are "no data available", and the guide for the next couple of days loaded as well.  I can see programs scheduled for Nov 1 and after, so in my mind, it was successful, and an indacation that the problem is with DST entries.

Carl

[Carl Fosler]

I have been having a problem with sourceforge drop the connect to me. I will try to load the mxf file that generated with the version that you just posted.

[Carl Fosler]

Steve,
Here my log file from my run.

Carl

[Steve Bickell]

Yes it looks like my change doesn't work in the real DST situation even though it didn't generate any -ve or zero durations when I tested it with your zap2 file here. They are still there in the mxf file you just posted.

I'll take another look.