last heavy bug: crashes of 68k modules when used from pps builds (mos/os4)

[kas1e]

So, we at rev 390. I test now all 3 builds: os3/os4/mos library/binary, them all works as expected, all is fine, but other than that we have one weird problem for ppc builds: some weird-strange crashes when we use 68k modules from ppc builds (os4 or morphos does not matter, problems the same).

The bug happens not in one single module, but in some of them. At least we catch 2 default one, and one 3dparty.

1. filetype.module.

To reproduce:

-- go to any directory contain .guide file
-- dbl-click on it, it will spawn a dopus-requester with buttons "sniff!/read/chancel"
-- press on "Sniff!".

Now, on os4 that window just disappear. I.e. shows and auto-closes. No crash, nothing on serial, just auto-closes. On morphos, it still shows, but show the empty action window. I.e. field Pri ID Filetype are empty. While in os3 version they fill by some stuff. Now, if we press "create" then it crashes. As on os4 we even didn't have that window visibly and it exits, then there is only crashlog from morphos:

http://kas1e.mikendezign.com/misc/dopus5/modules_bug/filetype_snifer_crash_mos.txt

Maybe its just somehow related to "strings" which we pass between ppc builds / os3 modules (at least, that can explain why on mos window is empty, and on os4 it autocloses like there is nothing).

2. ftp.module.

To reproduce:

-- run dopus5
-- press on FTP in buttons , or just choice FTP Connect from UserMenu.

And you immediately have crash which always and 100% reproducable on both: os4 and mos:

http://kas1e.mikendezign.com/misc/dopus5/modules_bug/ftp_connect_crash_mos.txt
http://kas1e.mikendezign.com/misc/dopus5/modules_bug/ftp_connect_crash_os4.txt

3. xadopus.module from aminet.

Crashes on morphos when you tried to open archive with it.

http://kas1e.mikendezign.com/misc/dopus5/modules_bug/xad_modules_crash_mos.txt

On os4 we create xadopus.m.main and it just works without crashes. So seems mos only related too.

I already build native versions of filetype.module for os4/mos, and while on os4 it start to works in native form straight away, on morphos it bring bugs and crashes (seems related to CALLBACKs). While fixing of native version for morphos is one thing, still, we need to make it all works in 68k form too (to avoid such kind of problems with any 3d party 68k modules later).

[kas1e]

@Ikka
I already port filetype.module to os3/os4.

That what i do:

-- add libfunc/libstub/libprotos
-- replace crap on REGs/ASM/SAVEDS/STDARGS
-- add in 2 functions (finder_creator_proc_init() and finder_editor_proc_init() ): REG(a2, int skip), so it will no crashes empty when call those ones
-- mix normally libinit.c from library with libinit.c from module
-- makefiles

OS3 build works fine with os3 library/binary, and OS4 build works fine with os4 library/binary, but crashes when os4 module is being used with os3 library/binary and crashed when filetype-sniffer window spawns , and stack trace point out on [filetype.c:1256] L_Module_Entry(), which are:

func_callback( EXTCMD_GET_SOURCE, IPCDATA(ipc), data->path );

So imho is that "ppc-68k" calling problems, and we imho need to make func_callback works there too ? (and it will be also good and for mos build, and for all native ppc modules if anyone will write them someday).

For morphos build, i already put there makefile, all what you need it fix it to builds on morphos + create morphos_stubs for those 2 module functions (entry and identify).

Also we need to fix there __alignment again. There is only few of them, but still.

Anyway, what it mean, is that i have native working build of filetype.module which works fine with os4 binary/library. I assume it will also fix the problems for morphos build too, but we anyway need to find out way to make 68k modules works from ppc builds imho..

Btw, what i found is that in code of the Program, in file command.c , we have:

// Ask module to identify itself
if (info=Module_Identify(-1))

For me it looks somehow related (as it exactly that -1 crash which we have).

[kas1e]

@ikka
I compile filetype_module for morphos as well. Just only with those changes:

1. I create mos_ppc_libstub.c file with need it functions
2. remove from makefile -DDOPUS5_LIBRARIES as it mess compiling of modules
3. in the code add:

void * malloc(size_t size) { return AllocVec(size, MEMF_ANY); }
void free(void *ptr) { return FreeVec(ptr); }

because if not then, morphos linker scream about needs for libnix_mempool when tryed to link that module (as on morphos channel says, its because when nostartup and co uses we cant use that stuff).

And sadly to say, but its crashes when open filetype window when i dbl-click on unknown filetype and press "sniff". I.e. window showns, but then crashes with the same -1. While as i say on os4 the same 1:1 code (with those malloc/free addons as well) works ok with native version. But on morphos its still -1 , what mean its can be in morphos related parts something again.

I will create in our branch now a directory "ported_modules", where will put all the modules which i port and/or on which some work in that terms done.

[Ilkka Lehtoranta]

func_callback( EXTCMD_GET_SOURCE, IPCDATA(ipc), data->path );

So imho is that "ppc-68k" calling problems, and we imho need to make func_callback works there too ? (and it will be also
good and for mos build, and for all native ppc modules if anyone will write them someday).

This func_callback() must be called via EmulateTags() / EmulCallDirect68k() function and arguments placed to correct registers respectively.

[Ilkka Lehtoranta]

Btw about those ported modules. It would be lot easier if all modules used the same framework to build a module. The base structure for all modules is same. Only name and copyrights have changed.

We could create small link library which is linked against modules to get it built.

[xenic]

@kas1e

but we anyway need to find out way to make 68k modules works from ppc builds

The modules are libraries. We need to generate a seperate interface file for each OS3 module just like you need to generate an interface file for any OS3 library that you want to call from an OS4 program. The module interfaces would probably be the same except for some name changes since they use (as likka pointed out in the previous message) the same library structure.

[kas1e]

@xenic
We are much futher than that. 2 weeks ago i even upload for you in some thread all those stubs for modules (of course they are libraries, how else i can port them ?:) ) and i of course test os3 gcc version with those stub-interfaces. problem now is not how to use os3 libs/modules (for them we already have those interface-stubs which i upload for you some time ago), but problem is when we just use os4 module/library with os3 dopus5.library/binary : that why we need to add there emulation calls. Its not big deal of course, as in general no one should use ppc modules with 68k library/binary, but if we want to make all cool and good, then we need add that as well.

@ikka
do you need any help with -1 crash ? i do all what we planned: now you need to fix filetype.module native mos crash and i can help with any debug. just lets finish first native release and then we can relax ) at least its very close, but alone i cant do anything about those -1 problems.

@all
can anyone have a look at ftp.module in ported dir ? all compiles but fail to link because of some amitcp hardcore. They define their own functions. Check ftp_extern_pragmas, there is bunch of amitcp_ and as225_ functions, which i dunno how to replace normally.

[Ilkka Lehtoranta]

I am busy couple of days from now on due to other commitments but I take a look at it after the weekend. Until then I cant do other than write few short comments here...

as225_#? functions is IIRC this old and obsolete Commodore networking library which was replaced by superior bsdsocket.library by 3rd party vendors. We can drop it.

amitcp_#? prefixed calls are just standard bsdsocket.library calls that were prefixed with "amitcp_" and words in function names were changed to start with uppercase letters. Remove this prefix and then change names to lower case except incalls which were added for Amiga only: CloseSocket(), WaitSelect() and SetErrnoPtr().

[kas1e]

@ikka

as225_#? functions is IIRC this old and obsolete Commodore networking library which was replaced by superior bsdsocket.library by 3rd party vendors. We can drop it.

Yes, but then what we can do with that:

int as225_accept (int, struct sockaddr *, int *);
int as225_bind (int, struct sockaddr *, int );
void as225_cleanup_sockets ( void ) ;
int as225_connect (int, struct sockaddr *, int);
struct hostent *as225_gethostbyname ( char * );
int as225_gethostname (char *, int);
struct servent *as225_getservbyname ( char *, char * );
struct servent *as225_getservbyport ( int, char * );
int as225_getsockname ( int, struct sockaddr *, int * );
u_long as225_inet_addr ( char * );
int as225_listen (int , int);
int as225_recv(int, char *, int, int );
int as225_selectwait (int, fd_set *, fd_set *, fd_set *, struct timeval *, long *);
int as225_send (int, char *, int, int );
int as225_setsockopt( int, int, int, char *, int );
ULONG as225_setup_sockets ( UWORD, int * );
int as225_shutdown (int, int);
int as225_socket( int, int, int );

#pragma libcall SocketBase as225_setup_sockets 1e 8102
#pragma libcall SocketBase as225_cleanup_sockets 24 0
#pragma libcall SocketBase as225_socket 2a 21003
#pragma libcall SocketBase as225_s_close 30 001
#pragma libcall SocketBase as225_gethostname 72 0802
#pragma libcall SocketBase as225_gethostbyname 8a 801
#pragma libcall SocketBase as225_inet_addr 96 901
#pragma libcall SocketBase as225_accept ba 98003
#pragma libcall SocketBase as225_bind c0 19003
#pragma libcall SocketBase as225_connect c6 19003
#pragma libcall SocketBase as225_listen d2 1002
#pragma libcall SocketBase as225_recv d8 218004
#pragma libcall SocketBase as225_selectwait f0 21A98006
#pragma libcall SocketBase as225_send f6 918004
#pragma libcall SocketBase as225_shutdown 108 1002
#pragma libcall SocketBase as225_setsockopt 10e 3821005
#pragma libcall SocketBase as225_getservbyname 168 9802
#pragma libcall SocketBase as225_getservbyport 16e 8002
#pragma libcall SocketBase as225_getsockname 19e 98003

I of course comment pragma libcalls, but then on linking it all want all those as225_x functions. How we should replace them ?

amitcp_#? prefixed calls are just standard bsdsocket.library calls that were prefixed with "amitcp_" and words in function names were changed to start with uppercase letters. Remove this prefix and then change names to lower case except incalls which were added for Amiga only: CloseSocket(), WaitSelect() and SetErrnoPtr().

That what we have for amitcp:

LONG amitcp_Accept(LONG s, struct sockaddr *addr, LONG *addrlen);
LONG amitcp_Bind(LONG s,  struct sockaddr *name, LONG namelen);
LONG amitcp_CloseSocket(LONG d);
LONG amitcp_Connect(LONG s,  struct sockaddr *name, LONG namelen);
struct hostent  *amitcp_GetHostByName( UBYTE *name);
LONG amitcp_GetHostName(STRPTR hostname, LONG size);          /* V3 */
struct servent  *amitcp_GetServByName( UBYTE *name,  UBYTE *proto);
struct servent  *amitcp_GetServByPort( LONG,  UBYTE *proto);
LONG amitcp_GetSockName(LONG s, struct sockaddr *name, LONG *namelen);
ULONG amitcp_Inet_Addr( UBYTE *);
LONG amitcp_Listen(LONG s, LONG backlog);
LONG amitcp_Recv(LONG s, UBYTE *buf, LONG len, LONG flags); /* V3 */
LONG amitcp_Send(LONG s,  UBYTE *msg, LONG len, LONG flags);
LONG amitcp_SetErrnoPtr(void *errno_p, LONG size);
LONG amitcp_SetSockOpt(LONG s, LONG level, LONG optname,
        const void *optval, LONG optlen);
LONG amitcp_Socket(LONG domain, LONG type, LONG protocol);
LONG amitcp_Shutdown(LONG s, LONG how);
LONG amitcp_WaitSelect(LONG nfds, fd_set *readfds, fd_set *writefds, fd_set *exeptfds,
        struct timeval *timeout, ULONG *maskp);

#pragma libcall SocketBase amitcp_Socket 1E 21003
#pragma libcall SocketBase amitcp_Bind 24 18003
#pragma libcall SocketBase amitcp_Listen 2A 1002
#pragma libcall SocketBase amitcp_Accept 30 98003
#pragma libcall SocketBase amitcp_Connect 36 18003
#pragma libcall SocketBase amitcp_Send 42 218004
#pragma libcall SocketBase amitcp_Recv 4E 218004
#pragma libcall SocketBase amitcp_Shutdown 54 1002
#pragma libcall SocketBase amitcp_SetSockOpt 5A 3821005
#pragma libcall SocketBase amitcp_GetSockName 66 98003
#pragma libcall SocketBase amitcp_CloseSocket 78 001
#pragma libcall SocketBase amitcp_WaitSelect 7E 1BA98006
#pragma libcall SocketBase amitcp_SetErrnoPtr A8 0802
#pragma libcall SocketBase amitcp_Inet_Addr B4 801
#pragma libcall SocketBase amitcp_GetHostByName D2 801
#pragma libcall SocketBase amitcp_GetServByName EA 9802
#pragma libcall SocketBase amitcp_GetServByPort F0 8002
#pragma libcall SocketBase amitcp_GetHostName 11A 0802

I of course comment pragma libcalls, and then tryed just that:

#define amitcp_Accept accept
#define amitcp_Bind bind
#define amitcp_CloseSocket closesocket
#define amitcp_Send send

But then, on linking it undefs to all of them. If i add -lc to linking, then it want ixemulbase , but as we in library with -nostartup and -noixemul, then it undefs. What to do with it ?

[Ilkka Lehtoranta]

We either delete all code using as225 or we create as225 inlines. But I dont know where to find as225 SDK... maybe it is on Amiga Developer CD or Aminet, must check it later.

Some object code must be compiled without -noixemul switch if it is wanting ixemulbase... anyway another solution would be renameing all amitcp_#? calls in source code to standard bsdsocket calls (like amitcp_Accept to accept).

[kas1e]

@ikka
i already deal with ftp.module for os4: it links, but crashes when i just tryed to open addressbook. it is in svn now,try to build mos version plz, i assume fixes will be the same. and once we will have working os4/mos version then can deal with os3 one. changes which i do mostly in that exter_socket_pragma.h file