#849 Page truncation in Opera (platform-specific Privoxy bug)

version 3.0.19
closed-fixed
Fabian Keil
other (166)
5
2012-10-22
2011-12-23
Kevin
No

When working with Opera 11.60 and Privoxy, pages, images, etc will randomly get truncated. It just started with the Opera upgrade, so it might be a bug with Opera, but when I stopped using Privoxy, I couldn't get any pages to fail. Note: 'stopped using' means configuring Opera to not go through Privoxy, simply unchecking 'Enable' on the Privoxy menu doesn't do it.

I attempted to capture data with Wireshark, but the pages I caught were gzipped and I can't be certain that they were good. I did notice that they were http 1.1, if that makes a difference.

Only one other person has mentioned on the Opera news group that they are having an issue like this. They too are running Privoxy.

NYTimes.com seems especially bad to me, but it happens on all sites including SouceForge.
Privoxy 3.0.18.
Opera 11.60
Windows 7 and Windows XP (I have two machines, both have the trouble)
No other proxy software. Windows Firewall and MSE on both machines.

SF user: photo2110

Discussion

1 2 3 > >> (Page 1 of 3)
  • Fabian Keil
    Fabian Keil
    2011-12-23

    Thanks for the report.

    Please reproduce the problem after enabling logging as described at http://www.privoxy.org/user-manual/contact.html#CONTACT-BUGS and provide a log file.

    If you can make the pcap file created with Wireshark available, that may be useful, too.

     
  • Fabian Keil
    Fabian Keil
    2011-12-23

    • assigned_to: nobody --> fabiankeil
    • status: open --> pending
     
  • Kevin
    Kevin
    2011-12-23

    Log file for truncated page (the-lives-they-lived)

     
    Attachments
  • Kevin
    Kevin
    2011-12-23

    • status: pending --> open
     
  • Kevin
    Kevin
    2011-12-23

    I've attached a log file for a truncated page. http://www.nytimes.com/interactive/2011/12/22/magazine/the-lives-they-lived.html?hp#view=dennis_ritchie

    Opera's source view goes along until:
    "100000001233615":{count:93, commentsEnabled:false},
    "100000001233630":{count:57, commentsEnabled:false},
    "100000001233632":{count:318, commentsEnabled:false},
    "100000001233655":{count:441, commentsEnabled:fa

    and we see it just stops in the middle of the word false.

    I'm apparently not understanding the log file, it looks like it grabbed the page twice, with the second grab bad.

     
  • Fabian Keil
    Fabian Keil
    2011-12-23

    Please additionally add "debug 32768" so we can see the data read from the network.

     
  • Fabian Keil
    Fabian Keil
    2011-12-23

    • status: open --> pending
     
  • Kevin
    Kevin
    2011-12-23

    New log file added with debug 32768 turned on. Truncated page is http://www.nytimes.com/pages/travel/index.html which dies at:

    <h6 class="kicker">Practical Traveler</h6>
    <div class="thumbnail">
    <a href="http://travel.nytimes.com/2011/12/25/travel/apps-to-keep-children-happy.html?ref=travel"><img src="http://graphics8.nytimes.com/images/2011/12/25/travel/25PRAC/25PRAC-thumbStandard-v2.jpg" width="75" height="75" alt="" border="0" /></a>
    </div>
    <h3><a href="http://trav

     
  • Kevin
    Kevin
    2011-12-23

    • status: pending --> open
     
  • Fabian Keil
    Fabian Keil
    2011-12-23

    Thanks.

    I can't see any obvious problems in the log. Please additionally add "debug 16" so the data written to the network is included as well.

    Does it make a difference if you add the following to one of your action files?:

    {+downgrade-http-version}
    # Sticky Actions = +downgrade-http-version
    # URL = http://www.nytimes.com/pages/travel/index.html
    .nytimes.com/

     
  • Fabian Keil
    Fabian Keil
    2011-12-23

    • status: open --> pending
     
  • qqqqqw
    qqqqqw
    2011-12-29

    I have the same problem.
    Initially i think that this problem is related with socks4a/5 bug, but after read this i'm not so sure.
    I have page truncation with Privoxy 3.0.18, not with 3.0.17.
    Now i try with 3.0.19

     
  • Kevin
    Kevin
    2011-12-29

    Sorry I've not replied sooner. Xmas break took priority. I put in a section to downgrade the NYTimes, and couldn't get any more issues with that site. So, I put in a downgrade for all of .com, and haven't seen anymore short pages on a .com site.

    I'll have to try the new .19 version.

     
  • Fabian Keil
    Fabian Keil
    2011-12-31

    qqqqqw, are you using Opera as well?

    Is the problem easy to reproduce?

    Can you provide a logfile with the usual debug directives and additionally "debug 16" and "debug 32768"?

    It would be especially useful to additionally get a pcap file that shows the packages Privoxy was receiving and sending.

    I think it's unlikely that upgrading to 3.0.19 makes a difference. You could try the recently uploaded privoxy-3.0.19-ipv4only.zip as it uses different code paths, but it probably will not make a difference either.

     
  • qqqqqw
    qqqqqw
    2012-01-01

    Yes, i use Opera 11.60 and the problem is always on same site (i dont know why)

    After the upgrade to 3.0.19 and a quick surf seem that the problem is socks5 + Tor (so Privoxy 3.0.18 for bug in socks4a) and NOT socks4a + Tor (so Privoxy 3.0.19)
    But i need more test (surf to more internet pages and try to use socks5 on Privoxy 3.0.19)
    If i have other page truncation i post a log

     
  • qqqqqw
    qqqqqw
    2012-01-13

    Page truncation on mashable.com with Opera 11.60 + Privoxy 3.0.19 and socks4a + Tor

    Log file with this debug enabled:
    debug 1
    debug 2
    debug 4
    debug 8
    debug 64
    debug 128
    debug 256
    debug 512
    debug 1024
    debug 4096
    debug 8192
    debug 32768

    Log file is here:
    http://www.zshare.net/download/986014474c9073b3/

    How can i attach log file in this discussion?

     
  • Fabian Keil
    Fabian Keil
    2012-01-13

    Downloading the file from zshare.net doesn't work for me.
    The download server doesn't even bother to return HTTP headers.

    In theory you should be able to upload a file through the "Add a file" section on this page.

     
  • qqqqqw
    qqqqqw
    2012-01-14

    I cannot see "add a file" so i upload another log file (with same debugs enabled) here:
    http://www.sendspace.com/file/n4v8k2

    Another 2 thing:
    1. I have never page truncation on https
    2. If i have page truncation on a site and the i use a cgi proxy to go to the same page i have no page truncation

     
  • Lee
    Lee
    2012-01-15

    Try disabling turbo mode in Opera.

    I installed Opera and, with extremely limited testing, it worked fine until I enabled "turbo mode". At which point it broke.

    Changing the privoxy config to
    keep-alive-timeout 0
    helped some, but did NOT fix all of the problems.

    I had wireshark capturing traffic during my testing. I thought I'd figured out the problem but now I'm not so sure :( I need to look at the packet traces again.

    Lee

     
  • qqqqqw
    qqqqqw
    2012-01-17

    I never use turbo mode

     
  • Fabian Keil
    Fabian Keil
    2012-01-23

    I assume in the logfile from sendspace.com/ the truncated output is the response for mashable.com?

    I tried reproducing it by sending the same headers Opera used, but the response appears to be complete.

    Unfortunately Privoxy's log buffer is smaller than the receive buffer so full receive buffers aren't logged completely in which case the received data can't be reconstructed.

    Can you still reproduce the problem if you enable +prevent-compression? A log file without compressed content would probably be easier to analyze.

     
  • qqqqqw
    qqqqqw
    2012-02-05

    Yes, is from mashable.com
    when i have some time i post a new log

     
  • esmandil
    esmandil
    2012-02-25

    I have the same problem.

    One thing I have noticed is that when I install polipo proxy ( http://www.pps.jussieu.fr/~jch/software/polipo/ ) and put it between Opera and Privoxy (i.e. pointing Opera to polipo, and setting up polipo to forward to Privoxy), the problem seems to disappear.

    If I put polipo between Privoxy and Internet, however, web pages in Opera are still broken.

    I don't know if this helps to diagnose the problem, but it's a workaround at worst.

     

  • Anonymous
    2012-03-10

    opera:config -> Performance -> Enable Pipelining -> off

     
1 2 3 > >> (Page 1 of 3)