#99 WASM build

[Danial Chitnis]

I have managed to successfully compile and build Ngspice for WASM with Emscripten. I have made a Docker container which encapsulates all the required components and procedures. Details can be found here:
https://github.com/danchitnis/ngspice

@h_vogt However, few problems do exist:
1. I had to remove the following section from misc_time.c due to memset.

#ifdef HAVE_GETRUSAGE
    int ret;
    struct rusage ruse;

    memset(&ruse, 0, sizeof(ruse));
    ret = getrusage(RUSAGE_SELF, &ruse);
    if(ret == -1) {
      perror("getrusage(): ");
      return 1;
    }
    return ((double)ruse.ru_utime.tv_sec + (double) ruse.ru_utime.tv_usec / 1000000.0);
#else

I am not sure what this function does? It doesn't seem to have any implications when running ngspice.
2. During compilation emmake make, there is a warning being generated that warning: unknown warning option '-Wno-unused-but-set-variable'; did you mean '-Wno-unused-const-variable'? [-Wunknown-warning-option]. Where is the option to change the CC compiler flags in the ngspice codebase?
3. OPENMP is not supported neither in WASM nor Emscripten yet. However this might change by the introduction SIMD instructions for WASM.
4. The shared library option doesn't seem to work well with Emscripten at the moment.

It would be nice to add a WASM build to ngspice build options.

[Holger Vogt]

memset is used in more than 20 places throughout the ngspice source code. So I do not understand why this one here creates problems, and the others do not.

CC compiler flags are set in configure.ac (see e.g. line 238).

OpenMP may be deselected by not giving --enable-openmp during the configure step.

What does 4. mean exactly? Any error messages during compile? Any strange behavior or results?

[Holger Vogt]

The previous discussion in #96 said that it is not memset, but getrusage (may in combination with memset) which does not do.

So we simply have to exclude getrusage. This should be done during the configure step. In configure.ac there is a check for operating system at compile time (lines 307 ff.) Could there be an entry for Emscripten as well? We then could set a flag to exclude getrusage.

[Danial Chitnis]

Hi Holger,
1. yes it is getrusage, the errors happen in memset, otherwise memset is fine. We can add a enable_wasm flag there.
2. I found the CC flags, I have to test it now. These warnings happen on every file and clutter the compiler output.
3. This is fine for now.
4. I am not sure where the problem is. It seems that emccis not strict enough when compiling shared libraries, and doesn't give any errors. Then, the code actually doesn't run and hardly gets past the first few lines without a fatal exit. The problem might be in the linking process. Getting the shared library working doesn't have a significant benefit over the interactive version. The reason is both run in the virtual WASM machine in a similar way.

Few more step to get a clean WASM version of ngspice without any hacks!

[Holger Vogt]

Again looking at some blogs from internet: It is not the memset function written by ngspice people, but a _memset function that is part of the getrusage function. So getrusage seems to be basically flawed in emcc.

You may try replacing
#ifdef HAVE_GETRUSAGE
by
#if defined(HAVE_GETRUSAGE) && !defined(__EMSCRIPTEN__)

[Danial Chitnis]

1. I changed the flag in configure.ac line 237 to CFLAGS="$CFLAGS -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Wold-style-definition -Wredundant-decls -Wconversion -Wno-unused-const-variable". This removed the previous unknown option warning. Can we make a separate CC flag for emscripten?
2. The getrusage bug has disappeared! I am not sure this was due to ngspice 33 or Emscripten 2.0.7 releases. There is no need for patches or flags here.

I have attached the config and make log here. Is the build process as expected? The generated wasm seems to run correctly. I will update my Docker file with these new changes.

[Holger Vogt]

Can we make a separate CC flag for emscripten?

Yes, as soon as we know how to safely detect emscripten or emcc in configure.ac .

[Holger Vogt]

Please check if
-Wno-unused
is o.k. instead of
-Wno-unused-but-set-variable

[Danial Chitnis]

-Wno-unused-const-variable
instead of
-Wno-unused-but-set-variable
works fine.

Currently, I am doing: sed -i 's/-Wno-unused-but-set-variable/-Wno-unused-const-variable/g' ./configure.ac

I have updated the demo to show ngspice version and build timestamp

[Holger Vogt]

emcc seems to be not up-to-date concernig possible error message flags (compared to gcc).

In an ngspice release I will not tinker around with sed or so.

If there is a means to safely auto-detect the emscripten environment in configure.ac, I would consider adding a branch to avoid the offensive flag. Unfortunately I do not have emscripten (and I am not considering to install it locally), so I have to rely on your input here.

[Danial Chitnis]

Hi Holger, this seems to work in configure.ac

if test "x$GCC" = xyes; then
        CFLAGS="$CFLAGS -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Wold-style-definition -Wredundant-decls -Wconversion -Wno-unused-but-set-variable"
    fi

    # Set flags for Emscripten and emcc
    AC_MSG_CHECKING([for Emscripten])
    AS_IF([test "x$EMSCRIPTEN" != "x"], [
    AC_MSG_RESULT([yes])
    CFLAGS="-g -O2 -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Wold-style-definition -Wredundant-decls -Wconversion -Wno-unused-const-variable"], [
    AC_MSG_RESULT([no])
    ])

It has to be placed after GCC test, otherwise $CFLAGS will be overwritten.

I am developing inside containers, so no local installation is needed. I have made a Github mirror, which eventually enables some automatic building and testing in the cloud.

[pe^cjo]

I have been trying to compile ngspice to wasm as a shared library, which requires -sSIDE_MODULE=1 to be passed as a linker flag. But it appears that the build system ignores this completely. How would I do this?

[pe^cjo]

For the record, the pyodide pull request: https://github.com/pyodide/pyodide/pull/5601

[Giles Atkinson]

Wasm has shared libraries? How does that work?

In any case, I think you need to give a more detailed description of what you want to do, how you are trying to do it, and what goes wrong, before anyone could help.

[pe^cjo]

Idk, emscripten has a page about it: https://emscripten.org/docs/compiling/Dynamic-Linking.html

The linked PR should have a pretty good summary of what I'm trying to do and what goes wrong:
I want to make an ngspice package for Pyodide you you can use it with libraries like PySpice/InSpice that load libngspice.so with cffi.

It is currently giving an error:
The following error occurred while loading ngspice:
need the dylink section to be first

Which the Pyodide maintainers say is because you need to pass -sSIDE_MODULE=1 as a linker flag to create a shared library without system libraries. I have added the suggested flags from libmagic to my recipe but am still getting the same error.

I've been trying to debug the ngspice build system but it appears that the linker flags aren't making it through so it still gives warnings about -shared and the resulting wasm library doesn't have the required dylink section.

[Danial Chitnis]

I tried this back in 2020 and wasn't successful. My feeling was that some code in ngspice needed to be changed for it to be used correctly as a shared library in WASM. If your goal is to use ngspice as a library in JS/TS, then just use this:

https://github.com/eelab-dev/EEcircuit-engine

[pe^cjo]

Yes I saw your project, which is really impressive. I want to use it as a python library though.

Are you using ngspice as a binary? InSpice also supports the binary interface so if that works I may just abandon the shared library approach and use the more primitive binary interface.

But I'd like to think with some build system massaging it should be possible to make a working libngspice.so like any other well behaved C library.

[Danial Chitnis]

Yes, it uses the binary in interactive mode. It is hacked quite a bit to achieve a library-like functionality. There are some advantages to doing it this way as you load once, and then simulations run faster.

Personally, I don't see the benefit of running WASM in Python. Perhaps security? EEcircuit-engine works with nodeJS, so you could wrap a Python function around it. It would be better to compile ngspice as a binary or shared library and call it in phyton since you run it locally, not inside a browser.

Maybe with some effort, we could compile ngpsice shared library into WASM, but it is also an issue with maintenance as Emscripten's update rate is very high.

[pe^cjo]

When I said python, I meant Pyodide, so Python compiled to WASM, so you can run simulations in a WASM notebook like Marimo. I can run ImSpice in Pyodide, which supports both the shared library interface as well as the binary interface. The advantage of the shared library is that it allows for streaming results. But if compiling as a binary is significantly easier than as a shared library I could use that.

[Danial Chitnis]

Okay, I see. Pyodide is an excellent project, and I was looking at what you are trying to do now a while back but didn't have time to work on it. An easier approach is to call eecircuit-engine as an external JS function inside Pyodide. I believe this should be possible according to their documentation:
https://pyodide.org/en/stable/usage/type-conversions.html

I will be interested in how you would get on whichever method you choose. Keep me posted.

[Giles Atkinson]

What part of Ngspice might be modified to fix this problem? What is the procedure for building? I have never knowingly used WASM, and the same may be true for other maintainers/developers here.

[pe^cjo]

The basic procedure is to wrap your build steps, which sets up the build with extra variables and parameters to use the emscripten compiler. From my Pyodide recipe:

emconfigure ../configure "${configure_args[@]}" CFLAGS="${SIDE_MODULE_CFLAGS}"
emmake make -j ${PYODIDE_JOBS:-3} LDFLAGS="${SIDE_MODULE_LDFLAGS}"

The complication seems to be that Emscripten has a unusual way of dealing with system libraries and preferring static linking combined with aggressive dead code elimination, and by default generates statically linked code in for dynamic libraries as well.

For dynamic linking, as explained in the above link, you have to pass the MAIN_MODULE and SIDE_MODULE linker flags, so that only the main module includes the system libraries. As far as I can tell the problem is that the ngspice build system overwrites LDFLAGS, so the shared library is not actually compiled as a shared library.

I have been trying unsuccessfully to modify the build system to pass LDFLAGS down, which seems to be at least the first hurdle to making this work.

[Giles Atkinson]

Pass " --disable-silent-rules" to configure. Then you will see that the LDFLAGS are passed to libtool and ignored! The only way I can find to get the library built with linker options is to log the build, then extract and edit the link command from the log so that "ld" can be called directly. Libtool is not your friend! I do not know how that translates with a WASM build, but the same method may work.

[pe^cjo]

i indeed tried the same change to pass cflags and ldflags like you mentioned with no effect.
are you suggesting to do a dry-run build to essentially generate a script, and then sed the script to inject the linker flag? that sounds properly hacky.

i don't know anything about libtool but you'd think surely there is a better way.

[pe^cjo]

it looks like libtool should have dedicated support for emscripten that should hide all the linking details as intended:
https://git.savannah.gnu.org/cgit/libtool.git/commit/?id=d8a934458b251b465b7d31da9ba9148a128a146d
so maybe the problem is rather that we're not using emscripten libtool?
this might be related:
https://sourceforge.net/p/ngspice/patches/100/

[Giles Atkinson]

Yes, I am suggesting that hacky approach. Ugly, but the only way I found to modify the ngspice library's linking.

That newer libtool sounds as if it might well help. However, all my testing was done on a standard build, no WASM involved. It seems like a libtool bug, or Ngspice is somehow misusing it. But nothing unusual is attempted.

My impression is that libtool is a solution for a problem that no longer exists: building on many commercial Unix variants with their own toolchains, all subtly different. I looked inside once and did not like it.

Finally a question: looking around the web, I did not find examples or screenshots of Pyodide use. I have the impression that is something like a command-line OS in your browser. Is that correct? And is there a "Try Me" link somewhere?

Thanks, G.