#128 better implementation of check-decoded-url

closed-accepted
Fabian Keil
5
2011-10-30
2011-10-28
Jamie Zawinski
No

Here is a better implementation of "fast-redirects{check-decoded-url}" that actually works properly with sites like Facebook.

The old implementation took the entire URL; decoded it; and then looked for the last http:. That doesn't work with URL redirectors done the way most modern sites do it, e.g.,

http://www.facebook.com/l.php?u=http%3A%2F%2F_real_url_&h=_other_crap_

which would assume the &h stuff was part of the URL.

This implementation first splits the URL at ? or &, then decodes each segment, and takes the last URL found in any of those.

Discussion

  • Jamie Zawinski
    Jamie Zawinski
    2011-10-28

     
    Attachments
  • Fabian Keil
    Fabian Keil
    2011-10-29

    • assigned_to: nobody --> fabiankeil
    • labels: 340251 -->
    • milestone: 1171315 -->
    • status: open --> pending
     
  • Fabian Keil
    Fabian Keil
    2011-10-29

    Thanks a lot for the patch.

    With the current implementation you can already solve the problem by additionally using a pcrs command with +redirect{}, but I agree that not having to do that would be an improvement worth having.

    Given that Privoxy is using threads on most platforms using strtok() without locks could cause problems, so I think we should use strtok_r() where available and fall back to strtok() with locks where it isn't.

    I'll try to sort this (and a couple of style issues) out in the next days.

     
  • Fabian Keil
    Fabian Keil
    2011-10-30

    Patch committed with follow-ups for style adjustments, locks and regression tests.

    For a single client the locks probably aren't that important, but running 50 Privoxy-Regression-Test forks with shuffles redirect tests lead to a couple of incorrect redirects.

    Assuming you don't intent to rebuild Privoxy from CVS, please note that the patch leaks dtoken which could amount to a few MB/day.

     
  • Fabian Keil
    Fabian Keil
    2011-10-30

    • labels: --> new feature
    • status: pending --> closed-accepted