Doesn't work for most websites because of "empty content"
Monitors a number of web pages for changes.
Brought to you by:
mortenmacfly
Menu
▾
▴
I tried 4 of the urls that didn't work on your system and they all seem to work for me.
If your aren't on the newest version download it from here (64/32bit version)
Could you give more information about the steps that lead to this problem?
Have you tried to only set the "item's adress" for testing?
I am using the latest build (20.01).
All I did was install it, "Add a new item", fill "Item's address" and "description", press Ok. List 11 entries, check them all to see if it works.
I can't do simpler.
I also checked my firewall but like 2/11 are working, it was not going to be that. I also ran it as admin without success.
I did as you said, created another entry for the reddit one with only the address. "Empty content" again.
Not sure if it depends on your standard browser but you can try to set a different one. I'm using chrome.
Have you tried to use "check now" on one of the once that didn't work?
Maybe it is because of too many requests in a short time.
If you have the possibility try it on a different pc.
And where do I set my browser? I didn't find any option. I set IE 10.0.9200.17457 as the default browser for my OS if that was what you meant but nothing changed. I set back to FF Developer 73.0b11.
Why does your tool need my browser in the first place to check websites? I want to use it because I do not want to have to use a browser's addon...
(And, yes, I "checked now".)
Last edit: bim 2020-01-31
It's not my tool, I just want to help you^^
I made a program in java that manipulated the contend of a html file and it also got the http header from the webbrouser if I remember correctly so I thought lets give it a try and change the default browser. (How to change the default browser)
Last edit: Sebastian Hausler 2020-01-31
WebChangeMonitor (WCM) works as following:
It reads the content of a web page then it filters the content according to what has been setup in the settings.
If the content is empty then there could be the following reasons:
1.) You are behind a proxy and did not set it up
2.) You have a filter setup (Tools -> Configuration -> Ignore pattern) that makes the content empty
3.) I am using the cURL library that is strict on SSL certificates - if the server is misconfigured you'll retrieve no content.
I'll check with the samples provided and get back to you. Please be patient, I am pretty busy at the moment, unfortunately.
For the browser: WCM calls the system-wide standard browser, so the one you set up in Windows as default app. There is an option planned in the furture to allow to set an individual brwoser in WCM itself.
One more note: For the next time, please export your list to a CSV file and attach this file and the INI file here. This makes life easier for testing since I see ALL settings. Being provided just with the log I don't see any filters or proxies you setup, for example.
Unfortunately all of the items work just fine for me. But I saw this in the log provided for example:
0445: [2020.01.27 21:24:46]: Downloaded 73060 bytes (raw).
0446: [2020.01.27 21:24:46]: Downloaded 0 bytes (converted).
This operation basically converts the plain byte stream downloaded by cCURL into a wxString such that it can be parsed. This seems to fail on your machine for several (nota all) items. On my PC it works always. We have to find out why it fails for you.
What locale are you using in Windows, what is the language / charset of your operating system? Anything like Chinese, Japanese, Russian or alike?
Last edit: Morten MacFly 2020-02-03
Yes, my system uses Japanese but I am not behind a proxy and have no "ignore pattern".
Edit: I attached the csv but can't find any ini where I extracted WCM. It must be hidden god knows where inside the windows folder. If you want it, you will have to tell me where it is.
Last edit: bim 2020-02-02
Thank you for the help. If you don't use ignores/proxy, the INI file is not relevant.
I did some investigatiosn and might have found the error. I've uploaded a special test release for you:
https://sourceforge.net/projects/webchangemon/files/Windows/WebChangeMonitor_20_02-64bit-TEST.zip/download
Would you mind to try this release and report back if it is now working for you now? Please make sure to attach the log once again (just the log - that is enough this time).
Thank you!
When checking "jellyfish", I get this error. If I say yes, WCM crashes. If I cancel errors, it looks like it works but I can't say for sure.
Also, I suggest adding a way to cancel states for multiple entries. We can't now and it's really tedious when running first time and you don't want to go to the websites yet to cancel the states for your entire list. It's the same for removing entries and I guess for other similar actions as well.
I installed winmerge and I would like a quick explanation as how to integrate it to WCM if it's not too much to ask? I have no idea what I am supposed to put at "command for comparison". I would like to check if WCM works as intended but you can't really do that without diff. Each time I check chillockscans, it is in an updated state, which is not normal I think, but I can't say what is new, if it's a comment or just a letter. The website is not popular enough for this level of updates. Btw, I just checked reddit I was sure was updated as I put this website for the sake of checking the software and I got the same error "jellyfish" got (in thread 10cc though).
I uploaded the log but the error is not reported...
Last edit: bim 2020-02-03
Alright, this is rather unfortunate as I cannot reproduce so I can only guess whats happening here. It seems that unlike on my machine on your machine you donwnload somehow encoded content and I don't know what encoding this is. So I am sorry for the inconvenience.
If you don't mind:
I've put another TEST file on SourceForge:
https://sourceforge.net/projects/webchangemon/files/Windows/WebChangeMonitor_20_02-64bit-TEST2.zip/download
This version will try UTF-8 encoding first and, if that fails, dump the content into a file named "WebChangeMonitor.dump". Please send that one to me along with the log.
For the differences: Webpages nowadays contain a lot of counters, dates, refresh times and so on. While the visibe content might not change, these counters do. That's what filters (ignore patterns) in WCM are for. They allow you to filter this stuff and compare only the real content.
You can setup the diff tool easily: Just install WinMerge for example and point to the executable in the settings. WCM will extend the command with "%1 %2" which will be replaced with old/new file once enough data is recorded by WCM. There is no special magic to do. Then, when you see what has changed you can setup filters as needed. Don't forget to enable using the filters in the items settings (per item).
Hope that helps...
Getchu was the only one to get that error ("in thread e90) but no dump file was generated.
If you say "Getchu was the only one" - does it mean it works now? Because it works just fine for me and looking into the log you've attached it should also have worked for you. In that case (if it is working), there should also be no dump file. So that would be OK.
Is the bug finally fixed?
"Getchu was the only one, when I tested it."
Afaik, this error (the pop-up window) has never showed up in the logs. If I say yes to shut down the program (and WCM crashes), the logs reports "Operating issued getchu" and "config saved", period, but if I cancel further warnings, WCM starts again where it stopped and downloads, checks changes, saves config, etc.
So, the program appears to work but it throws this error each time it check Getchu (and I have to cancel warnings). In fact, it appears to work but doesn't because I ran a test, waiting for the website to be updated (though it is filling both sides with random ads, meaning it is in a constant updating state from WCM's POW), but WCM didn't detect it...
(I didn't run tests on the other entries but some of them find updates and WinMerge can testify so I guess those entries work as intended.)
Last edit: bim 2020-02-11
Ok - I looked into it another time. There wasindeed one case where you could have gotten the error and no dump file was created. I've changed that and have uploaded another test-version here:
https://sourceforge.net/projects/webchangemon/files/Windows/WebChangeMonitor_20_02-64bit-TEST3.zip/download
Maybe you could try it on more time, please with this version? Again, if the dump file iscreated, please add both: The log and dump fiile here. Thank you!
I don't know what to say. We are back to most entries having empty content. And no dump file. Are you sure you gave me the right version? It generates a .config file btw but I assume it's the normal behavior.
(I have no problem doing whatever it takes to make your software work as I can't find another one which is neither a website or a browser extension. I thought you bailed out on me actually...)
Last edit: bim 2020-02-18
Don't worry, I won't forget you - I want to have this issue solved, really! Its just that I am rather busy these days and travel a lot such that I can work only rarely on private projects.
I made a new trial:
https://sourceforge.net/projects/webchangemon/files/Windows/WebChangeMonitor_20_02-64bit-TEST4.zip/download
This time, I changed the way how to detect if a conversion fails and also provide a concrete error code (in the logs+UI) if that happens. This should solve the error that came back for you. Furthermore, I create the dump file in the folder that is setup for temporary files (which must be writeable): Configuration > Files/Folders > Directory for temporary files.
My guess is, that the default folder for the dump file might not be writeable in your case and thats why it was not created (creation fails silently in that case).
Please, try again with this version - I am still hoping to get hands on the dump file, this would help me a lot as I am "blind" otherwise. In parallel I have a discussion on a CURL forum going on to see if I can enforce a specific locale when downloading content... but that is still ongoing...
No problem. Still no dump file on the horizon but a bunch of .new files instead (the folder where I put WCM was writable).
I see no error code in the logs or in the UI so I am not sure you are looking at the right place for that error. I hope cancelling further warnings doesn't stop WCM from registering whatever it is supposed to register.
Last edit: bim 2020-02-23
OK, this time I'm kind of excited. I got another trial for you:
https://sourceforge.net/projects/webchangemon/files/Windows/WebChangeMonitor_20_02-64bit-TEST5.zip/download
It will not yet work entirely but it will certainly provide important information to me through the log file to make it work once and for all (hopefully).
I've implemented an encoding detector (not yet in SVN) that will try to detect the encoding of the file downloaded on your machine. It should suppport Japanese just fine. You'll see something like this in the logs (for each item):
0021: [2020.02.27 07:48:09]: Conversion result: 32. Encoding detected as: 'UTF8' (=22), reliable=no.
This information is what I needfor the pages not working to establish a correct conversion and then it should work properly also for encodings other than "default" and UTF-8.
So please, try again and provide the log to me. Many thanks!
Well well well, we have the choice: ...ASCII-7-bit, SJS, and EUC-JP for our culprit Getchu.
PS: isn't this a problem if you are forced to manually adapt your program to each charset? It means that next time I stumble against a website with a new charset (Getchu is one of the biggest Japanese store for what it sells), it will be buggy again. What about a user from another country?
Last edit: bim 2020-02-29
But thanks for the input! Now can get a step further... it looks promising... just that I still may need you for testing. :-)
Another day, another try...:
https://sourceforge.net/projects/webchangemon/files/Windows/WebChangeMonitor_20_02-64bit-TEST6.zip/download
This is the first version that will make use of the encoding detector, translating the result to a most-likely correct encoding conversion and proceed. It should use a specific converter for the homepage that doesn't work for you, hopefully. Please, give it a try once more...
BTW: You'll see the changes I did in SVN meanwhile...