Debugging CGI scripts skips breakpoints

Help
David
2013-05-02
2013-05-20
  • David
    David
    2013-05-02

    When I use a CGI debug configuration, it seems to set the breakpoints (in the debug console), but none of them break into the Eclipse editor when they are executed.  If I run the same file under a “Perl Local” debug configuration, the breakpoints work.

    What I’ve tried and checked is below,  at the suggestion of JPloski, who pointed out that the path to the source that perl –d is using must match what Eclipse uses, and suggested turning on the “debug console (experimental)” in preferences. ( https://sourceforge.net/projects/e-p-i-c/forums/forum/258688 )

    Here’s what’s below, briefly:

    • System versions

    • perl –d vs Eclipse paths for CGI and Local match except backslashes

    • include paths – epic version of perldb.pl found 1st in both

    • perl –d invocation command args

    • EPIC CGI webserver settings

    • p.s.: Perl –d interaction “debug console”

    • p.s.: CGI (server) process log

    Please suggest what else I should try.

    1. Settings
      OS: Windows 7
      Eclipse:  Juno SR1
      Epic: 0.6.47 (org.epic.debug 0.6.34) updated yesterday.
      Perl: ActiveState v5.16.2 for mswin32 build 1602

    2. In the CGI debug session, the perl –d debug console lists the path passed to epic_breakpoints::add_breakpoint as
      C:/Users/olsond/workspace/mySandbox/PerlApache/process_form.pl
      Perl  -d responds  to ‘.’ with:
      C:\Users\olsond\workspace\mySandbox\PerlApache\process_form.pl:8)

    So the paths match except for being forward-slash (/) from perl –d. (I tried making all the webserver settings forward-slash, but it still passes in the target file as back-slash. See invocation below.)

    1. in the Perl Local debug,
      add_breakpoint: C:/Users/olsond/workspace/mySandbox/PerlApache/process_form.pl
      perl –d: C:/Users/olsond/workspace/mySandbox/PerlApache/process_form.pl:8
      breakpoints work in this configuration. Is it just that the perl –d path is forward slashes to match the breakpoint?

    2. file resource properties
      Location: C:\Users\olsond\workspace\mySandbox\PerlApache\process_form.pl
      Path: /mySandbox/PerlApache/process_form.pl

    3. include paths:
      I deleted my PERLLIB env variable, and @INC comes through in the test file as:
      For CGI :
      $VAR1 = 'C:/Users/olsond/workspace/mySandbox/PerlApache';
      $VAR2 = 'C:/Users/olsond/workspace/.metadata/.plugins/org.epic.debug';
      $VAR3 = 'C:/Perl/site/lib';
      $VAR4 = 'C:/Perl/lib';
      $VAR5 = '.';

    For Perl Local:
    $VAR1 = 'C:/Users/olsond/workspace/.metadata/.plugins/org.epic.debug';
    $VAR2 = 'C:/Users/olsond/workspace/mySandbox/PerlApache';
    $VAR3 = 'C:/Perl/site/lib';
    $VAR4 = 'C:/Perl/lib';
    $VAR5 = '.';

    The only difference as you can see is that the target file directory is 1st when doing CGI, but 2nd under Local. There are NO other files in the PerlApache directory.

    I did a loop to check for perl5db.pl in each directory and both the CGI and Local runs first find it in … workspace/.metadata/.plugins/org.epic.debug' (and second  in Perl/lib), so they should be using the same debugger.

    1. invocation of perl –d
      CGI
      Requested URI: /process_form.pl
      --------------CGI Command Line---------------
      C:/Perl/bin/perl.exe
      -IC:/Users/olsond/workspace/mySandbox/PerlApache
      -IC:/Users/olsond/workspace/.metadata/.plugins/org.epic.debug
      -d
      C:\Users\olsond\workspace\mySandbox\PerlApache\process_form.pl

    Perl Local:
    Starting Perl debugger:
    Command line:
    C:\Perl\bin\perl.exe
    -IC:/Users/olsond/workspace/.metadata/.plugins/org.epic.debug
    -IC:/Users/olsond/workspace/mySandbox/PerlApache/cgi-bin
    -d
    -Mautoflush_epic
    C:/Users/olsond/workspace/mySandbox/PerlApache/cgi-bin/process_form.pl
    Working directory: C:\Users\olsond\workspace\mySandbox\PerlApache\cgi-bin

    1. CGI webserver settings:

    : HTML Root: C:/Users/olsond/workspace/mySandbox/PerlApache
    : HTML Startup: C:/Users/olsond/workspace/mySandbox/PerlApache/process_form.pl
    : CGI Root: C:/Users/olsond/workspace/mySandbox/PerlApache
    File extension: .pl

    I tried many things to make this work, but to no avail. I’ve read many of the older threads on skipping breakpoints but each seems to have a different error or root cause solution than mine, except one titled “Breakpoints are ignored while debugging cgis” but there was no solution posted at the end. (I’ll update that one too if we find an answer.)  (They did offer good suggestions – like checking the include paths)

    Any help or suggestions will be appreciated.  This could be a very useful tool for our existing code.

    Sincerely,
    DAvid

    ==Perl –d interaction “debug console”==
    oading DB routines from perl5db.pl version 1.37
    Editor support available.

    Enter h or 'h h' for help, or 'perldoc perldebug' for more help.

    main::(C:\Users\olsond\workspace\mySandbox\PerlApache\process_form.pl:8):
    8: print "Content-type: text/html\n\n";
      DB<1> printf $DB::OUT "%vd", $^V;
    5.16.2
      DB<2> print $DB::OUT eval { require PadWalker; PadWalker->VERSION(0.08) }
    1.96
      DB<3> ;{
    my $file = <<'EOT';
    C:/Users/olsond/workspace/mySandbox/PerlApache/process_form.pl
    EOT
    my $line = <<'EOT';
    10
    EOT
    my $cond = '';

    epic_breakpoints::add_breakpoint($file, $line, $cond);
    };

      DB<4> .
    main::(C:\Users\olsond\workspace\mySandbox\PerlApache\process_form.pl:8):
    8: print "Content-type: text/html\n\n";
      DB<5> T
      DB<6> .
    main::(C:\Users\olsond\workspace\mySandbox\PerlApache\process_form.pl:8):
    8: print "Content-type: text/html\n\n";
      DB<7> .
    main::(C:\Users\olsond\workspace\mySandbox\PerlApache\process_form.pl:8):
    8: print "Content-type: text/html\n\n";
      DB<8> .
    main::(C:\Users\olsond\workspace\mySandbox\PerlApache\process_form.pl:8):
    8: print "Content-type: text/html\n\n";
      DB<9>

    ==CGI process log==

    Found default config file
    Server started on 5010
    LOG: 5 5010-server: main.: starting handler: cgi
    LOG: 5 5010-server: main.: starting handler: file
    LOG: 4 5010-server: 0:0:0:0:0:0:0:1: new connection
    LOG: 3 5010-0:0:0:0:0:0:0:1-0: Request 24 GET /process_form.pl HTTP/1.1
    LOG: 5 5010-0:0:0:0:0:0:0:1-0: main.: invoking handler: cgi
    LOG: 5 5010-0:0:0:0:0:0:0:1-0: suffix=.pl root=C:/Users/olsond/workspace/mySandbox/PerlApache url: /process_form.pl
    LOG: 5 5010-0:0:0:0:0:0:0:1-0: Checking for suffix: .pl
    LOG: 5 5010-0:0:0:0:0:0:0:1-0: looking for: C:\Users\olsond\workspace\mySandbox\PerlApache\process_form.pl
    LOG: 5 5010-0:0:0:0:0:0:0:1-0: found: C:\Users\olsond\workspace\mySandbox\PerlApache\process_form.pl
    LOG: 5 5010-0:0:0:0:0:0:0:1-0: CGI output 590 bytes.
    LOG: 3 5010-0:0:0:0:0:0:0:1-0: request done
    LOG: 3 5010-0:0:0:0:0:0:0:1-0: Error: 408 Request Time-out: Read timed out
    LOG: 4 5010-0:0:0:0:0:0:0:1-0: socket close

     
  • Jan Ploski
    Jan Ploski
    2013-05-02

    Here's what you can do to debug this further: download (https://github.com/jploski/epic-ide/blob/testing/org.epic.debug/epic_breakpoints.pm) or make a copy of your local C:/Users/olsond/workspace/.metadata/.plugins/org.epic.debug/epic_breakpoints.pm and put the copy in your @INC path in front of EPIC's own version, e.g. in C:/Users/olsond/workspace/mySandbox/PerlApache. Then tweak sub _abs_path in the copy to normalize slashes/backslashes and test your hypothesis that the difference matters (it sounds plausible).

     
  • David
    David
    2013-05-03

    Dear Jan,

    I mixed up the / and \ explanations in my last post. The CGI debug passes forward-slashes / to perl -d, but perl -d always returns paths with back-slashes. a table helps:

    direction v CGI Local
    to perl -d /path/ /path/
    from perl -d \path\ /path/
    invoked as \path\ /path/

    the Local debug hits breakpoints. CGI debug doesn't. As you can see, perl -d seems to return the path style it was invoked with (the target file path it was passed determines style)  (The annoying part is setting the "HTML Startup File" with /path/ style doesn't change the invokation of perl -d: Eclipse swaps them to \path\ style.)  So since we can't change perl-d to return /paths/, can we change Eclipse to invoke perl -d with foward slashes, thus mimicing the invokation that works with Perl local debug configurations?  i.e. where do we find the Eclipse code when the CGI debug generates:

    C:/Perl/bin/perl.exe
    -IC:/Users/olsond/workspace/mySandbox/PerlApache/cgi-bin
    -IC:/Users/olsond/workspace/.metadata/.plugins/org.epic.debug
    -d
    C:\Users\olsond\workspace\mySandbox\PerlApache\cgi-bin\process_form.pl

    ?

    call this approach 2, but I have this nagging feeling that we shouldn't need to change Eclipse or EPIC debug code….

    (BTW the last line of the above invocation will have forward-slashes if you get it from a Perl Local debug session.)

    approach 1: So I tried changing sub _abs_path to always return paths with \ , hoping it would then do add_breakpoint with \ to match perl -d's \.  the debug console still shows add_breakpoint getting sent forward-slashes /  and doesn't hit the breakpoints.

    Am I correct in understanding that the debug console (experimental) shows a conversaion between Eclipse and perl -d, with everything in red being what Eclipse outputs to perl -d, and the black being perl -d's responses? If so, then all the following is generated by Eclipse:
    ;{
    my $file = <<'EOT';
    C:/Users/olsond/workspace/mySandbox/PerlApache/epic_breakpoints.pm
    EOT
    my $line = <<'EOT';
    94
    EOT
    my $cond = '';

    epic_breakpoints::add_breakpoint($file, $line, $cond);
    };

    Any idea where this comes from in the EPIC code? Didn't see in in epic_breakpoints.pm or otherorg.epic.debug folder pms.

    Am I correct in interpreting the above text sent to perl -d as kinda a command line Perl invokation where perl -d interprets the Here docs, setting each variable and then call add_breakpoint?  If so, I guess perl -d is the interpreter when add_breakpoint is called instead of EPIC?  but that can't be if it's relying on the lexical or global stack space where epic_breakpoints:: setup all the paths….hmm…

    and ideas or hints?

    DAvid

     
  • David
    David
    2013-05-03

    Take two on the plain text table:

    direction V    CGI           Local 
    to perl -d     /path/         /path/
    from perl -d       \path\         /path/
    invoked as     \path\         /pat
    

    h/

    (is there anyway to try out your BBCode in a sandbox with the sourceforge formatter? It doesn't support all the BBCodes at the link it suggests)

     
  • Jan Ploski
    Jan Ploski
    2013-05-03

    You understand correctly that the "Experimental debugger console" logs the command-line conversation between the Eclipse/EPIC and perl -d process. EPIC issues commands expected by "perl -d", which can be snippets of Perl code to evaluate, perl -d executes the commands and prints output, which is then parsed by EPIC. In short, EPIC emulates a human user who types in commands during an interactive session with perl -d and attempts to keep the common parts of state - such as breakpoints - on the Java and Perl side synchronized.

    (You can view EPIC's Java code specifically responsible for add_breakpoint at https://github.com/jploski/epic-ide/blob/testing/org.epic.debug/src/org/epic/debug/db/PerlThreadBreakpoints.java)

    Given what you describe, tweaking _abs_path is not going to help.

    Maybe you can figure out why the paths are reported back to EPIC differently in the Local and CGI environment? You posted the command line used to start the CGI script. Would it make a difference to how perl -d reports source paths if that command line contained C:/Users/olsond/workspace/mySandbox/PerlApache/cgi-bin/process_form.pl instead? You can try it yourself from a DOS box. Or maybe there are some environment variables that influence this? I don't know what makes perl -d output paths with \ or with / under Windows, but it seems to be key to solving the issue. I vaguely remember that it tends to refer to source files by whatever name they were loaded in the first place (which can be especially nasty with Unix symlinks), but I may be wrong.

    EPIC could possibly take care of converting backlashes to slashes (or both to something else) in the received paths (it already tries to normalize paths to some extent). But I'm a bit worried that it might break other configurations (which I can't really test), so it's important to understand the root cause in your envrionment first.

     
  • David
    David
    2013-05-04

    Yes, changing the command line to refer to
    C:/Users/olsond/workspace/mySandbox/PerlApache/process_form.pl
    does cause perl -d to report source paths with forward-slashes. I tried it in a DOS box.

    The question is how to get Eclipse to invoke it that way.  The HTML Startup file is already set to the above, but the debug CGI command converts it to back-slashes .  I even verified if I change HTML Startup file to some other file, the same line on the perl -d command line changes names - (and with converted back-slashes.) 

    So, what perl -d gets in the command line is the style it uses for formatting its source paths.  I don't think an environment variable is affecting perl -d. 

    I need to get Eclipse to invoke perl -d with a slashed version.  It does this for the Perl Local debug config fine.  What would influence the CGI debug run to swap the slashes for backslashes?
    Is there a site that documents or has source code for this part of Ecilpse? Would this be in Java or part of EPIC?  I looked up 'launchers' but they didn't seem to relate to it.

    Thanks for your thoughts. the root cause is become more focused…
    DAvid

     
  • Jan Ploski
    Jan Ploski
    2013-05-04

    I recorded a bug report for this: https://sourceforge.net/tracker/?func=detail&aid=3612632&group_id=75859&atid=545274

    Please try updating to the just released 0.6.48 and make sure that your "HTML Root Directory" setting of the CGI debug configuration contains forward slashes. This should trigger a code path that converts backslashes to slashes in the script path passed down to "perl -d".

    Re: "Would this be in Java or part of EPIC?" - EPIC is a set of Eclipse plugins, which must be written in Java (there are only small pieces of it written in Perl, those which handle interfacing with perl executable). The entire source code of EPIC is at https://github.com/jploski/epic-ide (where you can also see my commit that attempts to fix this issue).

     
  • David
    David
    2013-05-06

    Dear Jan,

    I updated EPIC to 0.6.48, and my HTML root is all forward slashes, but Eclipse is still invoking perl -d with back-slashes for the target file. (and perl -d responds with back-slash paths).  Is there perhaps something in the way I'm using it that doesn't trigger your new code path? what should I check?
    -David

    C:/Perl/bin/perl.exe
    -IC:/Users/olsond/workspace/mySandbox/PerlApache
    -IC:/Users/olsond/workspace/.metadata/.plugins/org.epic.debug
    -d
    C:\Users\olsond\workspace\mySandbox\PerlApache\process_form.pl

    debug configuration:
    HTML Root Directory: C:/Users/olsond/workspace/mySandbox/PerlApache
    HTML Startup File: C:/Users/olsond/workspace/mySandbox/PerlApache/process_form.pl

     
  • David
    David
    2013-05-06

    After installing 0.6.48, I compared the src.zip files to those from 0.6.47 and I'm not seeing any differences in the new org.epic.debug java files. Maybe the bugfix didn't make it in?

    diff on src.zip extraction from:
    C:\eclipse\plugins\org.epic.source_0.6.47\src\org.epic.debug_0.6.34
    C:\eclipse\plugins\org.epic.source_0.6.48\src\org.epic.debug_0.6.35
    David

     
  • Jan Ploski
    Jan Ploski
    2013-05-06

    I think you made a mistake somewhere in your diff. I just checked that src.zip aren't equal:

    /tmp> diff -qr 0.6.47 0.6.48
    Files 0.6.47/org/epic/debug/cgi/server/EpicCgiHandler.java and 0.6.48/org/epic/debug/cgi/server/EpicCgiHandler.java differ
    Files 0.6.47/org/epic/debug/db/PerlValue.java and 0.6.48/org/epic/debug/db/PerlValue.java differ

    One thing worth checking is the file workspace/.metadata/.plugins/org.epic.debug/brazil.cfg - specifically the line that begins with "root=". Does it contains slashes or backslashes? This file is written by EPIC when you launch a CGI debug configuration and used by the embedded "Brazil" web server that actually serves the CGI scripts to the browser.

     
  • David
    David
    2013-05-06

    You're right. I didn't notice the changed symbols in WinMerge.  I was expecting PerlThreadBreakpoints.java to change, but your fix must be elsewhere.
    Guess I was hoping your change didn't make it in because I see the same behavior of skipping CGI breakpoinst still.  HTTP root is all slashes.  Brazil.cfg root= has no backslashes (other than the escaped : on C\:/users/….),

    If you can take another look at it please, or if I can help in any way please let me know. 
    -David

     
  • Jan Ploski
    Jan Ploski
    2013-05-06

    The code I added to replace backslash->slash was ineffective. Try again with 0.6.49.

     
  • David
    David
    2013-05-06

    Great! it works now. I was even able to "chain" several Perl CGI scripts - serve one output, click a link, serve another output and set breakpoints in the new one that work too. 
    Thank you for your persistent efforts.

     
  • David
    David
    2013-05-06

    For others who may be trying to get Perl CGI breakpoints to work, here's a few tips:

    - use the latest EPIC: it starting work for Windows in 0.6.49 (try Help|Check for Updates)
    - As said above, you must set the debug configuration/Web Server/HTML Root Directory to a path with forward slashes.
    - for the HTML startup file to validate as in this directory, it too must use foward slashes.  Apparently, C:\XX\file is not considered in C:/XX
    - If you use the Browse button for either it will switch back to backslashes

    So, twiddle these two paths until you have only forward slashes in the Root directory and Startup file path, and no errors at the top of the setup dialog.
    go debug…