Win 7 x64 mingw.mak - cannot find -lpq

[Carlo Stonebanks]

I am trying to build on Win 7 x64 as per your recommendation, getting this
error:

d:/devtools/mingw/bin/../lib/gcc/mingw32/4.6.2/../../../../mingw32/bin/ld.ex
e: cannot find -lpq

collect2: ld returned 1 exit status

mingw32-make: *** [dll] Error 1

Problem appears to come from line 40 of mingw.mak:

PG_LIBS = -L"$(PGSQL)/lib" -lpq

Can you please advise on how to fix this?

Thanks,

Carlo

[lbayuk]

Did you edit the Makefile "mingw.mak" as per the INSTALL file? You have to set the two symbols PGSQL and TCL at the top to point to the directories where PostgreSQL and Tcl are installed. The PostgreSQL one in particular needs to be the path to the folder which contains the "lib" folder, in which libpq.dll can be found.

I think it can handle spaces in those pathnames, but I am not 100% sure. (MinGW itself is not happy with spaces in its own install paths, I know that.)

If that doesn't help, make should have echoed the command line it tried, just before the "cannot find -lpq". If so, could you copy and past that to see what it tried?

By the way, in reference to [bugs:#6] - and it might be related to the problem building it - I am pretty much clueless on 32-bit vs 64-bit issues, if any. All I know so far is that MinGW is available only in 32-bits (but runs fine on 64-bit Windows), and both Tcl and PostgreSQL are available for Windows in both 32-bit and 64-bit versions. Which do you have? I'm wondering if mixed 32-bit and 64-bit versions of these products would cause problems.

[Carlo Stonebanks]

Yes, I had made the changes as advised in the install, and below is the full GCC command and the result. I have solved THIS particular problem and now I have another one. I am including the following info because I want to share what I have done, and I am concerned whether this make has ever been tried.

You can see that GCC is trying to call ld.exe with a lib file (-l parameter) - it appears to be trying to load a lib file called "pq"

D:\Projects\PgTcl\pgtcl2_0_0>mingw32-make -f my_mingw.mak
gcc -shared -o libpgtcl.dll -Wl,-enable-runtime-pseudo-reloc-v2 -Wl,-enable-auto-import pgtcl.o pgtclCmds.o pgtclId.o pgtclres.o -L"d:/DevTools/Tcl/lib" -ltclstub85 -L"d:/projects/postgresql-9_1_8/src/lib" -lpq

d:/devtools/mingw/bin/../lib/gcc/mingw32/4.6.2/../../../../mingw32/bin/ld.exe: cannot find -lpq
collect2: ld returned 1 exit status
mingw32-make: *** [dll] Error 1

I figured that this was just a syntax error, and in fact -lpq was supposed to be -llibpq. I modified the MAKE and added a path to the folder for D:\Program Files\PostgreSQL\9.1\lib where there is a 64-bit PG installation.

PG_LIBS = -L"$(PGSQL)/lib" -L"D:\Program Files\PostgreSQL\9.1\lib" -llibpq

Now the make produces a call to GCC that looks like this:

gcc -shared -o libpgtcl.dll -Wl,-enable-runtime-pseudo-reloc-v2 -Wl,-enable-auto-import pgtcl.o pgtclCmds.o pgtclId.o pgtclres.o -L"d:/DevTools/Tcl/lib" -ltclstub85 -L"d:/projects/postgresql-9_1_8/src/lib" -L"D:\Program Files\PostgreSQL\9.1\lib" -llibpq

This created several hundred new undefined reference errors. Below is a synopsis with repeated errors removed (still long, but at least a bit shorter).

I have included a copy of my_mingw.mak, perhaps it will help. I am afraid I am way out of my comfort zone with this stuff, if you have any idea why these are undefined references, I would really appreciate it.

Thanks,

Carlo

Undefined references from:
   pgtcl.o:pgtcl.c:
   pgtclCmds.o:pgtclCmds.c
   pgtclId.o:pgtclId.c

To:
   lo_close
   lo_creat
   lo_export
   lo_import
   lo_lseek
   lo_open
   lo_read
   lo_tell
   lo_truncate
   lo_unlink
   lo_write
   PQbackendPID
   PQclear
   PQcmdStatus
   PQcmdTuples
   PQconndefaults
   PQconnectdb
   PQconninfoFree
   PQconsumeInput
   PQdescribePortal
   PQdescribePrepared
   PQencryptPassword
   PQerrorMessage
   PQescapeBytea
   PQescapeByteaConn
   PQescapeString
   PQescapeStringConn
   PQexec
   PQexecParams
   PQexecPrepared
   PQfformat
   PQfinish
   PQfmod
   PQfname
   PQfreemem
   PQfsize
   PQftable
   PQftablecol
   PQftype
   PQgetCopyData
   PQgetisnull
   PQgetlength
   PQgetResult
   PQgetvalue
   PQisBusy
   PQisnonblocking
   PQnfields
   PQnotifies
   PQnparams
   PQntuples
   PQoidValue
   PQparameterStatus
   PQparamtype
   PQputCopyData
   PQputCopyEnd
   PQrequestCancel
   PQresStatus
   PQresultErrorField
   PQresultErrorMessage
   PQresultStatus
   PQsendQuery
   PQsendQueryParams
   PQsendQueryPrepared
   PQserverVersion
   PQsetClientEncoding
   PQsetdbLogin
   PQsetnonblocking
   PQsetNoticeProcessor
   PQsocket
   PQstatus
   PQtransactionStatus
   PQunescapeBytea
   Tcl_InitStubs
   tclStubsPtr   

[lbayuk]

Of course it has been tested. How do you think I build it for Windows? The gcc compiler in MinGW uses -lXXX to mean link with a library called "libXXX.EXT". The "lib" prefix is added by the linker, and .EXT is based on the platform. (It's an old Unix convention. The math library is "libm" or "libm.so", and linked with -lm, for example.) So "-L... -lpq" means add ... to the search path, then find and link with "libpq.dll".

Now, I don't know your exact install paths, but I am suspicious about this:

PGSQL=d:/projects/postgresql-9_1_8/src

That looks like a source directory, and we need the parent folder above the "lib" folder which has libpq.dll. I think what you want (based on the information you provided) is:

PGSQL="d:/Program Files/PostgreSQL/9.1"

(I believe either / or \ will work in this Makefile. But do use the quotes, because there are spaces in the path.)

Start from the original mingw.mak, or reset your other changes to the PG_LIBS, PG_INCLUDES etc, then try that instead.

Don't worry about the number of undefined references. They are all from libpq (except the last 2), and once we get the linking to libpq.dll working they should all go away.

[Carlo Stonebanks]

"How do you think I build it for Windows?"

My apologies - since I was having such a hard time, I thought perhaps you had built with BCC. Sorry, I am unaware of the 'NIX linker conventions.

I made the changes as per your recommendation, and ld failed with the following error:

D:/Program Files/PostgreSQL/9.1/lib/libpq.dll: file not recognized: File format not recognized
collect2: ld returned 1 exit status
mingw32-make: *** [dll] Error 1

I had downloaded a 64-bit version of PG, and I assume this was the problem. So I downloaded the 32 bit version of 9.1.8 and changed the make file accordingly:

PGSQL=D:/Program Files (x86)/PostgreSQL/9.1

I deleted all the .o files to force a complete recompile. I now get the following results with errors:

gcc -O2 -Wall -I"D:/Program Files (x86)/PostgreSQL/9.1/include" -I"D:/DevTools/Tcl/include" -DPACKAGE_VERSION=\"2.0.0\" -DHAVE_LO_TRUNCATE=1 -DHAVE_PQDESCRIBEPREPARED=1 -DHAVE_PQENCRYPTPASSWORD=1 -DHAVE_PQESCAPEBYTEACONN=1 -DHAVE_PQESCAPESTRINGCONN=1 -DUSE_TCL_STUBS   -c -o pgtcl.o pgtcl.c
gcc -O2 -Wall -I"D:/Program Files (x86)/PostgreSQL/9.1/include" -I"D:/DevTools/Tcl/include" -DPACKAGE_VERSION=\"2.0.0\" -DHAVE_LO_TRUNCATE=1 -DHAVE_PQDESCRIBEPREPARED=1 -DHAVE_PQENCRYPTPASSWORD=1 -DHAVE_PQESCAPEBYTEACONN=1 -DHAVE_PQESCAPESTRINGCONN=1 -DUSE_TCL_STUBS   -c -o pgtclCmds.o pgtclCmds.c
gcc -O2 -Wall -I"D:/Program Files (x86)/PostgreSQL/9.1/include" -I"D:/DevTools/Tcl/include" -DPACKAGE_VERSION=\"2.0.0\" -DHAVE_LO_TRUNCATE=1 -DHAVE_PQDESCRIBEPREPARED=1 -DHAVE_PQENCRYPTPASSWORD=1 -DHAVE_PQESCAPEBYTEACONN=1 -DHAVE_PQESCAPESTRINGCONN=1 -DUSE_TCL_STUBS   -c -o pgtclId.o pgtclId.c
windres pgtclres.rc pgtclres.o
gcc -shared -o libpgtcl.dll -Wl,-enable-runtime-pseudo-reloc-v2 -Wl,-enable-auto-import pgtcl.o pgtclCmds.o pgtclId.o pgtclres.o -L"D:/DevTools/Tcl/lib" -ltclstub85 -L"D:/Program Files (x86)/PostgreSQL/9.1/lib" -lpq

pgtcl.o:pgtcl.c:(.text+0x1d): undefined reference to `Tcl_InitStubs'
pgtcl.o:pgtcl.c:(.text+0x2a): undefined reference to `tclStubsPtr'
pgtcl.o:pgtcl.c:(.text+0x8a): undefined reference to `tclStubsPtr'
pgtcl.o:pgtcl.c:(.text+0xb8): undefined reference to `tclStubsPtr'
pgtcl.o:pgtcl.c:(.text+0xe6): undefined reference to `tclStubsPtr'
pgtcl.o:pgtcl.c:(.text+0x114): undefined reference to `tclStubsPtr'
pgtcl.o:pgtcl.c:(.text+0x142): more undefined references to `tclStubsPtr' follow
collect2: ld returned 1 exit status
mingw32-make: *** [dll] Error 1

This appears to be more than a warning, as no .dll is generated. I have included the console make output (make.txt) to reduce the verbosity of this post.

I Googled the errors, and someone had suggested changing the lib order so that the Tcl libs were last would solve the problem, so I changed the following lines following "dll: $(OBJS)":

BEFORE
    $(CC) -shared -o libpgtcl.dll $(LDFLAGS) $(OBJS) $(TCL_LIBS) $(PG_LIBS)
AFTER
    $(CC) -shared -o libpgtcl.dll $(LDFLAGS) $(OBJS) $(PG_LIBS) $(TCL_LIBS) 

Same errors. So I returned the order to what it was. I confirmed the tclstub85 was in the folder pointed to by the GCC parameters -L"D:/DevTools/Tcl/lib" -ltclstub85

This was the result:

dir "D:\DevTools\Tcl\lib\tclstub85.*"

Directory of D:\DevTools\Tcl\lib

04/02/2011  05:41 AM             5,770 tclstub85.lib
               1 File(s)          5,770 bytes
               0 Dir(s)  1,275,660,480,512 bytes free

Sorry for the high level of detail, I wanted to do as much up front work as possible before coming back to you.

The new make file is attached.

As always, Thanks,

Carlo

[Carlo Stonebanks]

UPDATE! After a kind response from a German developer who was also having trouble with Tcl stubs, I found out there was a MinGW-64; fortunately, someone had built a Windows binary and I downloaded it and compiled with that.

At first it errored out because of the 32-bit lib (ld error "skipping incompatible D:/Program Files (x86)/PostgreSQL/9.1/lib/libpq.dll when searching for -lp" and then eventually "ld.exe: cannot find -lpq")

So I changed back to 64-bit PG:

PGSQL=D:/Program Files (x86)/PostgreSQL/9.1

I was then able to recompile with only warning errors:

gcc -O2 -Wall -I"D:/Program Files/PostgreSQL/9.1/include" -I"D:/DevTools/Tcl/include" -DPACKAGE_VERSION=\"2.0.0\" -DHAVE_LO_TRUNCATE=1 -DHAVE_PQDESCRIBEPREPARED=1 -DHAVE_PQENCRYPTPASSWORD=1 -DHAVE_PQESCAPEBYTEACONN=1 -DHAVE_PQESCAPESTRINGCONN=1 -DUSE_TCL_STUBS   -c -o pgtcl.o pgtcl.c
gcc -O2 -Wall -I"D:/Program Files/PostgreSQL/9.1/include" -I"D:/DevTools/Tcl/include" -DPACKAGE_VERSION=\"2.0.0\" -DHAVE_LO_TRUNCATE=1 -DHAVE_PQDESCRIBEPREPARED=1 -DHAVE_PQENCRYPTPASSWORD=1 -DHAVE_PQESCAPEBYTEACONN=1 -DHAVE_PQESCAPESTRINGCONN=1 -DUSE_TCL_STUBS   -c -o pgtclCmds.o pgtclCmds.c
gcc -O2 -Wall -I"D:/Program Files/PostgreSQL/9.1/include" -I"D:/DevTools/Tcl/include" -DPACKAGE_VERSION=\"2.0.0\" -DHAVE_LO_TRUNCATE=1 -DHAVE_PQDESCRIBEPREPARED=1 -DHAVE_PQENCRYPTPASSWORD=1 -DHAVE_PQESCAPEBYTEACONN=1 -DHAVE_PQESCAPESTRINGCONN=1 -DUSE_TCL_STUBS   -c -o pgtclId.o pgtclId.c
windres pgtclres.rc pgtclres.o
gcc -shared -o libpgtcl.dll -Wl,-enable-runtime-pseudo-reloc-v2 -Wl,-enable-auto-import pgtcl.o pgtclCmds.o pgtclId.o pgtclres.o -L"D:/DevTools/Tcl/lib" -ltclstub85 -L"D:/Program Files/PostgreSQL/9.1/lib" -lpq
pgtclId.c: In function 'PgSetConnectionId':
pgtclId.c:238:54: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
Warning: .drectve `/DEFAULTLIB:"MSVCRT" /DEFAULTLIB:"OLDNAMES" ' unrecognized

Is the cast error from this line?

connid->notifier_channel = Tcl_MakeTcpClientChannel((ClientData) PQsocket(conn));

If so, is this a Tcl stub 32-bit pointer being put into a libpq 64 bit pointer, and is it safe?

[lbayuk]

Sorry, I don't know what happened - I worked on this Fri 2/8 and I am sure I replied to your message (2nd above), but my reply isn't there and I guess it got lost somewhere or never posted.

I can't remember what I was writing then, and I see you took another path anyway in the above message so maybe it is moot. Anyway, you did succeed then in getting it to link to PostgreSQL by switching to the 32-bit version. So I think the message "file not recognized" refers to the 32-bit vs 64-bit library files. (But I see you got a different message when it went the other way: 64-bit linking to 32-bit library.)

But then it failed on Tcl, and I don't know why. There is no error message about the library - just not finding the symbols. So I was going to ask where your Tcl came from, and what version, 32-bit or 64-bit. Your tclstub85.lib is not the same size as mine, which is from ActiveState ActiveTcl 8.5.13 (32-bit version).

Now it looks like you are going back to trying 64-bit versions. On Fri I also read about Mingw-w64, which is not maintained by the MinGW project but has apparently been used successfully by people to build 64-bit programs.

Re: that cast warning from the compiler (why does it follow the link command, and not the compile command?)

If so, is this a Tcl stub 32-bit pointer being put into a libpq 64 bit pointer, and is it safe?

It is an 'int' returned by PostgreSQL as a socket handle, being stored in a Tcl 'opaque' type. I believe this is one of those things that gets people in trouble when moving to 64-bits... The cast itself should be safe because it won't lose bits. But I would not trust the whole thing in 64 bits without testing, and I don't have any way to test on 64-bit Windows yet.

[Carlo Stonebanks]

Re: that cast warning from the compiler (why does it follow the link command, and not the compile command?)

In order to create the file, I redirected err and stdout devices in separate files as it was the only way to get all the output. That's why the compiler progress messages (stdout) aren't immediately followed by their error messages (err).

I reproduced your environment to the letter. 32 bit ActiveState Tcl 8.5.13, PG 32 bit (9.1) and then loaded the original Pgtcl 2.0.0 project. I can now confirm that if everything is 32 bit, from Tcl to PostgreSQL that it is possible to run on a Windows 7 x64 machine. Interestingly, absolutely no one on the postgres-general mailing list responded with any meaningful help on this topic.

So, although I am disappointed to find out that we will always have to worry about 64 bit installations of Tcl or 64-bit libpq.dll being first in the search path, at least I am relieved to be able to say, "Yes - it IS possible to use Windows 64 bit PC's.

I appreciate the help and I am sorry that I could not help the project by building a 64-bit version of the DLL. If you would like me to still try, I am at your disposal, but I will need very specific instructions - as you can tell, this is a very different development environment for me.

No matter what you decide, I very much appreciate the help you have given.

[lbayuk]

You've help a lot by persisting and finding out about the 32-bit Tcl + 32-bit PostgreSQL on 64-bit Windows. Eventually I hope to have access to a 64-bit Windows 7 / Linux box, for more testing, but for now just the information you found is very valuable.

I understand about the DLL search paths. I spent some time trying to figure out why I was getting 5 new Pgtcl-ng test failures after updating my ActiveState Tcl and PostgreSQL client, only to discover I had PHP in my search path early, and PHP for Windows ships a really old version of libpq.dll (from PostgreSQL-8.3.6). Pgtcl-ng was using that version instead of the 9.2.x one.

Another apology: I used to follow pgsql-general looking for pgtcl issues, but have neglected it. (There used to be a pgsql-interfaces list for issues like this, which was much lower traffic and easier to follow, but no more.) I just checked and see your message... well, they were a bit rough, weren't they? Had I seen it and replied, it might have gone better.

To answer your question on the list about "Pg & Tcl - is it dying out?" - Other than the 64-bit issue, I wouldn't say it is dying, but there isn't very much activity on any of the 3 projects. At most, I've been retesting when new PostgreSQL versions (or Tcl) come out. My goal has always been more in line with the original libpgtcl interface: to provide a LibPQ interface from Tcl. So I'm not interested in adding new, higher-level features. Unfortunately, I haven't really kept up with improvements in LibPQ that well either.

One more thing: I will probably take the bug report you opened, and change it to cover need for 64-bit support. So it will stay open as a record of something that needs to be fixed.