[perl-win32-gui-hackers] Re: Fixing Scintilla

[Robert May <rmay@po...>]

Jeremy White wrote:
> I'm trying to fix Scintilla, with the aim of creating a new release of 
> this module. The problem is that Scintilla defines and uses it's own 
> PERLWIN32GUI_USERDATA structure. As this structure has changed over time 
> in Win32-GUI it creates all kinds of problems with this module 
> (Scintilla subclasses Win32-GUI, and uses the PERLWIN32GUI_USERDATA 
> structure passed to it by Win32-GUI).  The most extreme behaviour is 
> with perl 5.8.x where the Scintilla does not process events - there is 
> also cases where Scintilla crashes.
> 
> I've managed to get a build of Scintilla working, but I've got several 
> issues. Instead of defining it's own version of PERLWIN32GUI_USERDATA it 
> now picks it up from GUI.h - which I think is the best solution (?).
> 
> 1) The original Scintilla:
> 
> #include "EXTERN.h"
> #include "perl.h"
> #include "XSUB.h"
> 
> #include <windows.h>
> #include "./include/Scintilla.h"
> 
> changed to:
> 
> #include "../Win32-GUI/GUI.h"
> #include "./include/Scintilla.h"
> 
> However, when compiling with VC I get two errors:
> 
> *** Using Preserved Perl context.
> 
> ../Win32-GUI/GUI.h(439) : error C2143: syntax error : missing ')' before 
> '='
> ../Win32-GUI/GUI.h(439) : error C2072: 'DoHook' : initialization of a 
> function
> ../Win32-GUI/GUI.h(439) : error C2059: syntax error : ')'
> ../Win32-GUI/GUI.h(809) : error C2059: syntax error : 'string'
> NMAKE : fatal error U1077: 'C:\WINDOWS\system32\cmd.exe' : return code 
> '0x2'
> Stop.
> 
> this is with the line:
> 
> void DoHook(NOTXSPROC LPPERLWIN32GUI_USERDATA perlud, UINT uMsg, WPARAM 
> wParam, LPARAM lParam, int* PerlResult, int notify = 0);
> 
> removing " = 0" fixes that issue.

At first glance these errors look like C++ vs. C syntax problems.  I 
haven't looked at the scintilla code, but it would be my guess that the 
XS files is being compiled as C (this is the standard Makefile.PL 
behaviour) whereas Win32::GUI (for reasons that escape me) is built 
using a C++ compiler (see the override in Win32::GUI's Makefile.PL (sub 
xs_c) that builds .cpp files from the XS rather than .c files).

There are 2 approaches: (1) add the same override to Scintilla to get it 
to make .cpp files from the .xs, forcing the compilers into c++ mode, 
(2) Fix up the Win32::GUI GUI.h file to work correctly with a standard C 
compiler.

I thin that (2) if preferable, and it is one of my longer-term goals to 
get Win32::GUI to use standard 'C' too.

Standard 'C' does not allow defaulting of function parameters in the 
prototype, so my solution here would be to remove the "= 0" from GUI.h, 
and modify the 2 calls to DoHook() in GUI_MessageLoops.cpp that don't 
pass the last parameter to pass 0 explicitly.

> The second error:
> 
> *** Using Preserved Perl context.
> 
> ../Win32-GUI/GUI.h(809) : error C2059: syntax error : 'string'
> NMAKE : fatal error U1077: 'C:\WINDOWS\system32\cmd.exe' : return code 
> '0x2'
> Stop.
> 
> this is with:
> 
> extern "C" BOOL WINAPI
> GetWindowInfo(
>    HWND hwnd,
>    PWINDOWINFO pwi
> );
> 
> Commenting out this code removes the error.

My fix would be to replace that code with
#ifdef __cplusplus
   extern "C"
#endif
BOOL WINAPI GetWindowInfo(
   HWND hwnd,
   PWINDOWINFO pwi
);

> 2) I'm confused about the Perl context and how it's used within 
> Win32-GUI - and how Scintilla should handle things.
> 
> I've got it working by doing:
> 
> dTHX;       /* fetch context */
> 
> in the Scintilla event handlers, but had to remove NOTXSCALL/NOTXSPROC 
> in some functions where the context isn't used/needed.
> 
> If it would help, I can check in what I've got, with the idea of fixing 
> things once I've got my head around these issues?

I'm not overly familiar with this, but the concept is that a call to a C 
  function that doesn't require perl context should look like a regular 
C call;  A call from an XS sub (or anywhere that has a perl context 
pointer available) to a function that requires a perl context should be 
defined with a first argument of NOTXACALL/NOTXSPROC macro (but with no 
following  comma).  These macros defined the correct type of argument to 
pass the perl context.  I can do more investigation and expand on this 
if necessary.

Regards,
Rob.

[perl-win32-gui-hackers] Fixing Scintilla

[Jeremy White <jez_white@ho...>]

Hi,

I'm trying to fix Scintilla, with the aim of creating a new release of this 
module. The problem is that Scintilla defines and uses it's own 
PERLWIN32GUI_USERDATA structure. As this structure has changed over time in 
Win32-GUI it creates all kinds of problems with this module (Scintilla 
subclasses Win32-GUI, and uses the PERLWIN32GUI_USERDATA structure passed to 
it by Win32-GUI).  The most extreme behaviour is with perl 5.8.x where the 
Scintilla does not process events - there is also cases where Scintilla 
crashes.

I've managed to get a build of Scintilla working, but I've got several 
issues. Instead of defining it's own version of PERLWIN32GUI_USERDATA it now 
picks it up from GUI.h - which I think is the best solution (?).

1) The original Scintilla:

#include "EXTERN.h"
#include "perl.h"
#include "XSUB.h"

#include <windows.h>
#include "./include/Scintilla.h"

changed to:

#include "../Win32-GUI/GUI.h"
#include "./include/Scintilla.h"

However, when compiling with VC I get two errors:

*** Using Preserved Perl context.

../Win32-GUI/GUI.h(439) : error C2143: syntax error : missing ')' before '='
../Win32-GUI/GUI.h(439) : error C2072: 'DoHook' : initialization of a 
function
../Win32-GUI/GUI.h(439) : error C2059: syntax error : ')'
../Win32-GUI/GUI.h(809) : error C2059: syntax error : 'string'
NMAKE : fatal error U1077: 'C:\WINDOWS\system32\cmd.exe' : return code '0x2'
Stop.

this is with the line:

void DoHook(NOTXSPROC LPPERLWIN32GUI_USERDATA perlud, UINT uMsg, WPARAM 
wParam, LPARAM lParam, int* PerlResult, int notify = 0);

removing " = 0" fixes that issue.

The second error:

*** Using Preserved Perl context.

../Win32-GUI/GUI.h(809) : error C2059: syntax error : 'string'
NMAKE : fatal error U1077: 'C:\WINDOWS\system32\cmd.exe' : return code '0x2'
Stop.

this is with:

extern "C" BOOL WINAPI
GetWindowInfo(
    HWND hwnd,
    PWINDOWINFO pwi
);

Commenting out this code removes the error.

In both cases I don't understand why they cause errors?

2) I'm confused about the Perl context and how it's used within Win32-GUI - 
and how Scintilla should handle things.

I've got it working by doing:

dTHX;       /* fetch context */

in the Scintilla event handlers, but had to remove NOTXSCALL/NOTXSPROC in 
some functions where the context isn't used/needed.

If it would help, I can check in what I've got, with the idea of fixing 
things once I've got my head around these issues?

Cheers,

jez.

[perl-win32-gui-hackers] Re: Fixing Scintilla

[Robert May <rmay@po...>]

Jeremy White wrote:
> I'm trying to fix Scintilla, with the aim of creating a new release of 
> this module. The problem is that Scintilla defines and uses it's own 
> PERLWIN32GUI_USERDATA structure. As this structure has changed over time 
> in Win32-GUI it creates all kinds of problems with this module 
> (Scintilla subclasses Win32-GUI, and uses the PERLWIN32GUI_USERDATA 
> structure passed to it by Win32-GUI).  The most extreme behaviour is 
> with perl 5.8.x where the Scintilla does not process events - there is 
> also cases where Scintilla crashes.
> 
> I've managed to get a build of Scintilla working, but I've got several 
> issues. Instead of defining it's own version of PERLWIN32GUI_USERDATA it 
> now picks it up from GUI.h - which I think is the best solution (?).
> 
> 1) The original Scintilla:
> 
> #include "EXTERN.h"
> #include "perl.h"
> #include "XSUB.h"
> 
> #include <windows.h>
> #include "./include/Scintilla.h"
> 
> changed to:
> 
> #include "../Win32-GUI/GUI.h"
> #include "./include/Scintilla.h"
> 
> However, when compiling with VC I get two errors:
> 
> *** Using Preserved Perl context.
> 
> ../Win32-GUI/GUI.h(439) : error C2143: syntax error : missing ')' before 
> '='
> ../Win32-GUI/GUI.h(439) : error C2072: 'DoHook' : initialization of a 
> function
> ../Win32-GUI/GUI.h(439) : error C2059: syntax error : ')'
> ../Win32-GUI/GUI.h(809) : error C2059: syntax error : 'string'
> NMAKE : fatal error U1077: 'C:\WINDOWS\system32\cmd.exe' : return code 
> '0x2'
> Stop.
> 
> this is with the line:
> 
> void DoHook(NOTXSPROC LPPERLWIN32GUI_USERDATA perlud, UINT uMsg, WPARAM 
> wParam, LPARAM lParam, int* PerlResult, int notify = 0);
> 
> removing " = 0" fixes that issue.

At first glance these errors look like C++ vs. C syntax problems.  I 
haven't looked at the scintilla code, but it would be my guess that the 
XS files is being compiled as C (this is the standard Makefile.PL 
behaviour) whereas Win32::GUI (for reasons that escape me) is built 
using a C++ compiler (see the override in Win32::GUI's Makefile.PL (sub 
xs_c) that builds .cpp files from the XS rather than .c files).

There are 2 approaches: (1) add the same override to Scintilla to get it 
to make .cpp files from the .xs, forcing the compilers into c++ mode, 
(2) Fix up the Win32::GUI GUI.h file to work correctly with a standard C 
compiler.

I thin that (2) if preferable, and it is one of my longer-term goals to 
get Win32::GUI to use standard 'C' too.

Standard 'C' does not allow defaulting of function parameters in the 
prototype, so my solution here would be to remove the "= 0" from GUI.h, 
and modify the 2 calls to DoHook() in GUI_MessageLoops.cpp that don't 
pass the last parameter to pass 0 explicitly.

> The second error:
> 
> *** Using Preserved Perl context.
> 
> ../Win32-GUI/GUI.h(809) : error C2059: syntax error : 'string'
> NMAKE : fatal error U1077: 'C:\WINDOWS\system32\cmd.exe' : return code 
> '0x2'
> Stop.
> 
> this is with:
> 
> extern "C" BOOL WINAPI
> GetWindowInfo(
>    HWND hwnd,
>    PWINDOWINFO pwi
> );
> 
> Commenting out this code removes the error.

My fix would be to replace that code with
#ifdef __cplusplus
   extern "C"
#endif
BOOL WINAPI GetWindowInfo(
   HWND hwnd,
   PWINDOWINFO pwi
);

> 2) I'm confused about the Perl context and how it's used within 
> Win32-GUI - and how Scintilla should handle things.
> 
> I've got it working by doing:
> 
> dTHX;       /* fetch context */
> 
> in the Scintilla event handlers, but had to remove NOTXSCALL/NOTXSPROC 
> in some functions where the context isn't used/needed.
> 
> If it would help, I can check in what I've got, with the idea of fixing 
> things once I've got my head around these issues?

I'm not overly familiar with this, but the concept is that a call to a C 
  function that doesn't require perl context should look like a regular 
C call;  A call from an XS sub (or anywhere that has a perl context 
pointer available) to a function that requires a perl context should be 
defined with a first argument of NOTXACALL/NOTXSPROC macro (but with no 
following  comma).  These macros defined the correct type of argument to 
pass the perl context.  I can do more investigation and expand on this 
if necessary.

Regards,
Rob.

[perl-win32-gui-hackers] Perl contexts [Was: Fixing Scintilla]

[Robert May <rmay@po...>]

>> 2) I'm confused about the Perl context and how it's used within 
>> Win32-GUI - and how Scintilla should handle things.
>>
>> I've got it working by doing:
>>
>> dTHX;       /* fetch context */
>>
>> in the Scintilla event handlers, but had to remove NOTXSCALL/NOTXSPROC 
>> in some functions where the context isn't used/needed.
>>
>> If it would help, I can check in what I've got, with the idea of 
>> fixing things once I've got my head around these issues?
> 
> 
> I'm not overly familiar with this, but the concept is that a call to a C 
>  function that doesn't require perl context should look like a regular C 
> call;  A call from an XS sub (or anywhere that has a perl context 
> pointer available) to a function that requires a perl context should be 
> defined with a first argument of NOTXACALL/NOTXSPROC macro (but with no 
> following  comma).  These macros defined the correct type of argument to 
> pass the perl context.  I can do more investigation and expand on this 
> if necessary.

OK, I've just scanned the perldoc perlguts section entitled "How 
multiple interpreters and concurrency are supported".  I think it's 
reasonable to assume the following:

You can always define PERL_NO_GET_CONTEXT - all modern perl's will 
behave with this regardless of whether PERL_IMPLICIT_CONTEXT is defined 
or not (although it is defined for current ActiveState Perl builds - 
it's required for ithread support).  Defining pTHX, pTHX_, aTHX, aTHX_ 
and dTHX are dealt with in perl.h and XSUB.h

In Win32::GUI NOTXSPROC and NOTXSCALL are aliases for pTHX_ and aTHX_ 
respectively.  Any other code in GUI.h is purely for backwards 
compatibility with older perl's.  Even the "\n*** Using an implicit Perl 
context.\n" section should not be necessary, as pTHX_ and aTHX_ should 
deal with non-thread-capable perl builds.

To understand how to use pTHX_ and aTHX_ correctly read perlguts - but 
for consistency with Win32::GUI I would recommend using NOTXSPROC and 
NOTXSCALL macros in their place.

[ I've added a note to myself to look for all the places that we use 
dTHX and to ensure that they are necessary - I think they are used when 
we get a win32 api callback such that we can't use the NOTXSPROC calling 
convention ]

I hope this helps.

Rob.

[perl-win32-gui-hackers] RE: Perl contexts [Was: Fixing Scintilla]

[Jeremy White <jez_white@ho...>]

>[ I've added a note to myself to look for all the places that we use dTHX 
>and to ensure that they are necessary - I think they are used when we get a 
>win32 api callback such that we can't use the NOTXSPROC calling convention 
>]

I'm still not 100% sure how the context works within Win32-GUI but there 
might be a problem when Win32-GUI is used in a threaded app, where you want 
different threads to deal with different windows (see AttachThreadInput 
method in the docs).

As I understand things, we're storing the context in PERLWIN32GUI_USERDATA 
via the macro PERLUD_DECLARE and we're fetching the context via the macro 
PERLUD_FETCH. As we're fetching the context everytime we're doing an event, 
we are *always* using the perl interpreter that created the window, rather 
than the context that is associated with the current running thread.

I'm not to sure what the solution could be, as I understand things you can't 
pass sub refs around perl threads, which means that NEM would be broken 
(i.e., thread A creates the window and the sub refs for the events, but you 
need thread B to respond to events for the window created by A). From very 
bitter experience, Perl 5.6 and most 5.8.x (5.8.7 seems ok) threading at the 
deep perl level is broken.

Cheers,

jez.

[perl-win32-gui-hackers] Re: [win32gui] RE: Perl contexts [Was: Fixing Scintilla]

[Robert May <rmay@po...>]

Jeremy White wrote:
>> [ I've added a note to myself to look for all the places that we use 
>> dTHX and to ensure that they are necessary - I think they are used 
>> when we get a win32 api callback such that we can't use the NOTXSPROC 
>> calling convention ]
> 
> 
> I'm still not 100% sure how the context works within Win32-GUI but there 
> might be a problem when Win32-GUI is used in a threaded app, where you 
> want different threads to deal with different windows (see 
> AttachThreadInput method in the docs).
> 
> As I understand things, we're storing the context in 
> PERLWIN32GUI_USERDATA via the macro PERLUD_DECLARE and we're fetching 
> the context via the macro PERLUD_FETCH. As we're fetching the context 
> everytime we're doing an event, we are *always* using the perl 
> interpreter that created the window, rather than the context that is 
> associated with the current running thread.
> 
> I'm not to sure what the solution could be, as I understand things you 
> can't pass sub refs around perl threads, which means that NEM would be 
> broken (i.e., thread A creates the window and the sub refs for the 
> events, but you need thread B to respond to events for the window 
> created by A). From very bitter experience, Perl 5.6 and most 5.8.x 
> (5.8.7 seems ok) threading at the deep perl level is broken.

So long as you don't use AttachThreadInput, then I think we're OK, as 
messages get posted to the thread that creates the window, and the 
message loops only pull messages from that thread's message loop. 
You're correct that as soon as you call AttachThreadInput the whole 
thing is broken.

That said, Win32::GUI needs some significant work to be thread-safe (and 
even more work to be thread-friendly).

I've got some ideas of the direction to take this, and think that as a 
first step simply adding a CLONE_SKIP method to each class, that 
prevents the objects getting cloned when a new thread is started is the 
easiest direction.  This would certainly cater for (what I can see is a 
very common case of) wanting to start a thread to do some work, whilst 
keeping all the GUI code in a separate thread.   At least we'd be able 
to use threads (even if in a somewhat limited way) without crashing 
every time we do a thread->join().

I'll tuck your comments away for further though when I get around to 
considering threads in more detail.

Regards,
Rob.

[perl-win32-gui-hackers] Re: Fixing Scintilla

[Jeremy White <jez_white@ho...>]

>At first glance these errors look like C++ vs. C syntax problems.  I 
>haven't looked at the scintilla code, but it would be my guess that the XS 
>files is being compiled as C (this is the standard Makefile.PL behaviour) 
>whereas Win32::GUI (for reasons that escape me) is built using a C++ 
>compiler (see the override in Win32::GUI's Makefile.PL (sub xs_c) that 
>builds .cpp files from the XS rather than .c files).

ARH! Yes, you are right. I've always wondered why win32-gui used a C++ 
compiler, as there is no C++ code.

>I thin that (2) if preferable, and it is one of my longer-term goals to get 
>Win32::GUI to use standard 'C' too.

I changed GUI.h (and GUI_MessageLoops.cpp) to be "C" safe.

Cheers,

jez.

[perl-win32-gui-hackers] Re: [win32gui] Re: Fixing Scintilla

[Robert May <rmay@po...>]

Jeremy White wrote:
>> At first glance these errors look like C++ vs. C syntax problems.  I 
>> haven't looked at the scintilla code, but it would be my guess that 
>> the XS files is being compiled as C (this is the standard Makefile.PL 
>> behaviour) whereas Win32::GUI (for reasons that escape me) is built 
>> using a C++ compiler (see the override in Win32::GUI's Makefile.PL 
>> (sub xs_c) that builds .cpp files from the XS rather than .c files).
> 
> ARH! Yes, you are right. I've always wondered why win32-gui used a C++ 
> compiler, as there is no C++ code.
> 
>> I thin that (2) if preferable, and it is one of my longer-term goals 
>> to get Win32::GUI to use standard 'C' too.
> 
> I changed GUI.h (and GUI_MessageLoops.cpp) to be "C" safe.

Nice work on Scintilla - I haven't tried building it yet, but can 
confirm that Win32::GUI still builds and tests ok with these changes and 
your TYPEMAP changes.

Regards,
Rob.