#510 standalone plugin not seen in linux

[Sam Tansy]

I tried to make standalone plugin and got few obstacles.
Linux 7zz i doesn't see it, no matter where I put it (whether `Codecs' directory is in same directory or in `/usr/local/lib/p7zip/Codecs'), while windows 7z.exe i sees it, no problem.

$ wine 7z.exe i

7-Zip 21.07 (x86) : Copyright (c) 1999-2021 Igor Pavlov : 2021-12-26

Libs:
 0  Z:\mnt\dev2\src\pak\7z_codec\src\7z2107_win\7z.dll
 1  Z:\mnt\dev2\src\pak\7z_codec\src\7z2107_win\Codecs\ByteSwap8_Codec.dll

Codecs:
(...)
 0  EDF    20302 Swap2
 0  EDF    20304 Swap4
(...)
 1  EDF    20308 Swap8

$ ./7zz i

7-Zip (z) 21.07 (x86) : Copyright (c) 1999-2021 Igor Pavlov : 2021-12-26

Formats:
(...)

Codecs:
(...)
    EDF    20302 Swap2
    EDF    20304 Swap4

At first I thought it was because in linux `Codecs.def' doesn't work - gcc doesn't recognize it, in windows (mingw) it works but then I realized that Linux version doesn't look for plugins in Codecs directory. If that's the case, how to convince 7z to look for plugins in Codecs directory?

Second thing is - there seem to be some major change in plugins API in 7z-23. When I tried to compile my code (in attachment) I got whole bunch of errors (ibid.). Would you mind to explain what that is and put in README?

[Ninimu]

Can you please try building 7z from CPP/7zip/UI/Console/makefile.gcc and 7z.so from CPP/7zip/Bundles/Format7zF/makefile.gcc, and use the compiled 7z binary instead of 7zz?

[Sam Tansy]

Can you please try building 7z from CPP/7zip/UI/Console/makefile.gcc

Notice I use original source, only my plugin is separate, to not mess up with original source code tree. I have many, many reasons to not mix up with main source, like updates, for example. If main source changes, which it does quite often recently, I would have to edit that new source, edit new Makefiles and deal with changing tens, possibly hundreds of lines of code in case of complex plugins. I don't see a reason to do that if I can do it better.
There is nothing wrong with separate plugins. They can even be compiled separately and linked statically into executable at final linkage stage.
I don't see a benefits of doing what you propose.

[Igor Pavlov]

7z + 7z.dll (+ plugins) - is main way to use 7-Zip in Windows.
But it could be more difficult to provide dynamic linking in linux.
That is why I used standalone 7zz in linux as main binary.
But if you need plugins in linux, then you must compile code with dynamic linking:

CPP/7zip/UI/Console/makefile.gcc
CPP/7zip/Bundles/Format7zF/makefile.gcc

[Igor Pavlov]

7zz - is standalone version. It doen't look plugins.
7z + 7z.so must work.

About changes in API.
Old plugins for p7zip are not compatible with new 7-zip for linux because we have changed IUnknown:

/*
  p7zip and 7-Zip before v23 used virtual destructor in IUnknown,
  if _WIN32 is not defined.
  It used virtual destructor, because some compilers don't like virtual
  interfaces without virtual destructor.
  IUnknown in Windows (_WIN32) doesn't use virtual destructor in IUnknown.
  We still can define Z7_USE_VIRTUAL_DESTRUCTOR_IN_IUNKNOWN here,
  if we want to be compatible with old plugin interface of p7zip and 7-Zip before v23.

v23:
  In new 7-Zip v23 we try to be more compatible with original IUnknown from _WIN32.
  So we do not define Z7_USE_VIRTUAL_DESTRUCTOR_IN_IUNKNOWN here,
*/
// #define Z7_USE_VIRTUAL_DESTRUCTOR_IN_IUNKNOWN


DEFINE_GUID(IID_IUnknown,
0x00000000, 0x0000, 0x0000, 0xC0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x46);
struct IUnknown
{
  STDMETHOD(QueryInterface) (REFIID iid, void **outObject) =0;
  STDMETHOD_(ULONG, AddRef)() =0;
  STDMETHOD_(ULONG, Release)() =0;
 #ifdef Z7_USE_VIRTUAL_DESTRUCTOR_IN_IUNKNOWN
  virtual ~IUnknown() {}
 #endif
};

Also there is some additional function in API to get
version of plugin API and version of module:

GetModuleProp
NModulePropID::kInterfaceType
NModulePropID::kVersion
NModuleInterfaceType::k_IUnknown_VirtDestructor_Yes

[Sam Tansy]

About changes in API

Understand. So the request is to put this in README; it's important change that noone knows about. At least something like

p7zip and 7-Zip before v23 used virtual destructor in IUnknown,
if _WIN32 is not defined.
Details in LoadCodecs module

And put that in LoadCodecs.cpp rather than in MyWindows.h, where would be reference to `LoadCodecs'.

---

7zz - is standalone version. It doen't look plugins.

Would you mind to add such option, so 7zz would be compelled to look for plugins?
Or, shall I make a formal 'feature request'?

[Igor Pavlov]

There is macro

// #define Z7_USE_VIRTUAL_DESTRUCTOR_IN_IUNKNOWN

that allows to get code compatible with old code.
And we must define that macro in MyWindows.h or in makefile.
It changes all com interfaces everywhere in 7-zip (not only LoadCodecs.cpp).
So it was described in place where it can be defined - MyWindows.h.

Did you read history.txt in source code ?

- 23.01          2023-06-20
- Linux/macOS versions of 7-Zip: IUnknown interface in new code doesn't use 
  virtual destructor that was used in previous 7-Zip and p7zip:
     // virtual ~IUnknown() {}
  So 7-Zip's dynamically linked shared libraries (codecs) are not compatible 
  between new 7-Zip for Linux/macOS and old 7-Zip (and p7zip).

[Sam Tansy]

There is macro
// #define Z7_USE_VIRTUAL_DESTRUCTOR_IN_IUNKNOWN
that allows to get code compatible with old code.

I will test it.

So it was described in place where it can be defined - MyWindows.h.

Still in:

// LoadCodecs.cpp
/*
Z7_EXTERNAL_CODECS

Could be reference to `MyWindows.h', like:

Plugin interface is defined here, in LoadCodecs.cpp module and in MyWindows.h.

And in README reference to plugins, or at least where are they defined.

External Plugins can be placed in `Codecs' and/or `Formats' directories.
Definition of interface is placed in LoadCodecs module
Please note that interface changed in v23. For details refer to above module.

---

Did you read history.txt in source code?

I did. I missed that somehow. I guess, because of 'Linux/MacOS'. I don't have Mac.

It states:

Linux/macOS versions of 7-Zip:

Does it mean that Windows plugins are still compatible between versions?

[Sam Tansy]

Tried to make it work with with v23 and my simple example of Swap8 seems to work. With more complex XorFilter from here I couldn't figure out how exactly these macros are suppose to work. AFAIU Z7_COM7F_IMF/Z7_COM7F_IMP replace STDMETHOD/STDMETHODIMP, and don't know what to do with MY_UNKNOWN_IMP2.
For both 7zip source is to be in `./7z2301' directory. It can be adjusted in scripts.

[Igor Pavlov]

Just look 7-zip codecs in source code (old in new).
look in old source code of 7-zip the lines where MY_UNKNOWN_IMP2 was used, and then look to what lines it was replaced.

[Sam Tansy]

XorFilter v23 attached here as I couldn't attach two files.

[Igor Pavlov]

Would you mind to add such option, so 7zz would be compelled to look for plugins?

Maybe that is good feature, but maybe is not good in some cases.
Maybe some users want the code that never use any plugins.

[Sam Tansy]

Maybe some users want the code that never use any plugins.

And it would not work by default without overtly specifying it. If someone makes an effort to specify an option they mean to use it.