Re: [Modules] tcl module load slowness debug

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Thank you for the response Xavier

we have 2 environments :
old environment based on RHEL 7)
~]$ time module load R-bundle-Novartis

real    0m2.327s
user    0m1.677s
sys     0m0.177s
~]$ module --version
VERSION=3.2.10
DATE=2012-12-21

AUTOLOADPATH=undef
BASEPREFIX="/usr/share"
BEGINENV=99
CACHE_AVAIL=undef
DEF_COLLATE_BY_NUMBER=undef

new environment based on RHEL 9.5)
module version:
~]$ module --version
Modules Release 5.3.0 (2023-05-14)

Yes, Module files are located over the network in both the situations.
That’s why we believe that either the latest version of module is unable to handle the module loads efficiently
or there is something within the module files which causes the loads to take more time.

I tried using –time and –debug, and I got a lot of information. Will try to see if I am able to understand the root cause
……….
TIMER charEscaped 2.1.12 (0.018 ms)
TIMER charEscaped /CHBS/apps/EB/software/NASM/2.15.05-GCCcore-11.2.0/easybuild/NASM-2.15.05-GCCcore-11.2.0-easybuild-devel (0.013 ms)
TIMER charEscaped /CHBS/apps/EB/software/OpenSSL/1.1 (0.011 ms)
TIMER charEscaped 2.38 (0.010 ms)
TIMER charEscaped 1.12 (0.012 ms)
TIMER charEscaped {echo -e "cmdstan THREADS" && make -f /apps/EB-DEV/software/cmdstan/2.32.2-gomkl-2022a/makefile -C /apps/EB-DEV/software/cmdstan/2.32.2-gomkl-2022a "${PWD}/$1"} (0.062 ms)
TIMER charEscaped {R --no-restore-data --no-save} (0.016 ms)
TIMER renderTrue (1.653 ms)
TIMER renderSettings (162.068 ms)
TIMER module load R-bundle-organozation (163945.261 ms)
TIMER renderFlush (2.275 ms)

TIMER Total execution took 163951.384 ms

real    2m44.000s
user    0m38.132s
sys     0m15.587s
[user@rhel9server ~]$ ls -lrth out
-rw-------. 1 user ug 93M Dec  3 04:46 out
[user@rhel9server ~]$

Regards
Puneet

From: Xavier Delaruelle <xav...@gm...>
Sent: 01 December 2025 23:51
To: Singh, Puneet-2 (Ext) <pun...@no...>
Subject: Fwd: [Modules] tcl module load slowness debug

Le lun. 1 déc. 2025 à 19: 19, Xavier Delaruelle <xavier. delaruelle@ gmail. com> a écrit : Hi, To debug such situation you may add the "--timer --debug" options to your module commands. You will get precise list of operations with
ZjQcmQRYFpfptBannerStart
This Message Is From an Untrusted Sender
You have not previously corresponded with this sender.
ZjQcmQRYFpfptBannerEnd

Le lun. 1 déc. 2025 à 19:19, Xavier Delaruelle <xav...@gm...<mailto:xav...@gm...>> a écrit :
Hi,

To debug such situation you may add the "--timer --debug" options to your module commands. You will get precise list of operations with their timing.

What version of Modules are you using ?
Is the modulefiles located on a network filesystem? Have you tried building a module cache to speed-up the reads?

Regards,
Xavier

Le lun. 1 déc. 2025 à 19:09, Singh, Puneet-2 (Ext) via Modules-interest <mod...@li...<mailto:mod...@li...>> a écrit :

Hi ,
We are currently in a situation where in a module load and module purge are taking more than 15 seconds.
I f this was a code, I would have attached a debugger to see which line of code is taking longer.  Could you please help me to understand how and thich tools  can I debug the root cause of slowness here ?
Here is the command I use to reproduce the slowness -

So far we have onserved slowness issues with only this particular module.

Example:
[singhpuv@server ~]$ time module load  R-bundle-Novartis

real    0m16.866s
user    0m15.105s
sys     0m0.082s
[singhpuv@Pserver~]$ module list
Currently Loaded Modulefiles:
1) GCCcore/11.2.0                               83) Mesa/22.0.3-GCCcore-11.2.0
2) zlib/1.2.11-GCCcore-11.2.0                   84) libGLU/9.0.2-GCCcore-11.2.0
3) binutils/2.37-GCCcore-11.2.0                 85) nodejs/18.17.1-GCCcore-11.2.0
4) GCC/11.2.0                                   86) MPFR/4.1.0-GCCcore-11.2.0
5) numactl/2.0.14-GCCcore-11.2.0                87) OpenJPEG/2.5.0-GCCcore-11.2.0
6) XZ/5.2.5-GCCcore-11.2.0                      88) Pandoc/3.1.2
7) libxml2/2.9.13-GCCcore-11.2.0                89) NSPR/4.34-GCCcore-11.2.0
8) libpciaccess/0.16-GCCcore-11.2.0-BuildNAOS   90) NSS/3.79-GCCcore-11.2.0
9) hwloc/2.7.1-GCCcore-11.2.0                   91) double-conversion/3.2.0-GCCcore-11.2.0
10) OpenSSL/1.1                                  92) graphite2/1.3.14-GCCcore-11.2.0
11) libevent/2.1.12-GCCcore-11.2.0               93) DBus/1.14.0-GCCcore-11.2.0
12) UCX/1.12.1-GCCcore-11.2.0                    94) snappy/1.1.9-GCCcore-11.2.0
13) libfabric/1.15.1-GCCcore-11.2.0              95) Qt5/5.15.5-GCCcore-11.2.0-MEISSA
14) PMIx/4.1.2-GCCcore-11.2.0                    96) Boost/1.81.0-GCC-11.2.0
15) UCC/1.0.0-GCCcore-11.2.0                     97) poppler/22.01.0-GCC-11.2.0-MEISSA
16) OpenMPI/4.1.4-GCC-11.2.0                     98) OpenPGM/5.2.122-GCCcore-11.2.0
17) gompi/2022a                                  99) libsodium/1.0.18-GCCcore-11.2.0
18) imkl/2022.1.0-gompi-2022a                    100) ZeroMQ/4.3.4-GCCcore-11.2.0-BuildNAOS
19) gomkl/2022a                                  101) FFTW/3.3.10-gompi-2022a
20) OracleInstantClient/21.12-el9                102) Szip/2.1.1-GCCcore-11.2.0
21) ncurses/6.2-GCCcore-11.2.0                   103) HDF5/1.12.1-gompi-2022a
22) libreadline/8.1-GCCcore-11.2.0               104) libogg/1.3.5-GCCcore-11.2.0
23) bzip2/1.0.8-GCCcore-11.2.0                   105) FLAC/1.3.4-GCCcore-11.2.0
24) PCRE2/10.40-GCCcore-11.2.0                   106) libvorbis/1.3.7-GCCcore-11.2.0
25) cURL/7.78.0-GCCcore-11.2.0                   107) libopus/1.3.1-GCCcore-11.2.0
26) M4/1.4.20-GCCcore-11.2.0                     108) LAME/3.100-GCCcore-11.2.0
27) expat/2.4.8-GCCcore-11.2.0                   109) libsndfile/1.1.0-GCCcore-11.2.0
28) DB/18.1.40-GCCcore-11.2.0                    110) libpipeline/1.5.7-GCC-11.2.0
29) Perl/5.34.0-GCCcore-11.2.0-BuildNAOS         111) man-db/2.11.2-GCC-11.2.0
30) Autoconf/2.71-GCCcore-11.2.0-BuildNAOS       112) NLopt/2.7.1-GCCcore-11.2.0
31) Automake/1.16.5-GCCcore-11.2.0-BuildNAOS     113) UDUNITS/2.2.28-GCCcore-11.2.0
32) libtool/2.4.6-GCCcore-11.2.0                 114) netCDF/4.9.0-gompi-2022a
33) Autotools/20220317-GCCcore-11.2.0            115) GEOS/3.11.1-GCC-11.2.0
34) libpng/1.6.37-GCCcore-11.2.0                 116) nlohmann_json/3.11.2-GCCcore-11.2.0
35) Brotli/1.0.9-GCCcore-11.2.0                  117) PROJ/9.0.0-GCCcore-11.2.0
36) freetype/2.12.1-GCCcore-11.2.0               118) libgeotiff/1.7.1-GCCcore-11.2.0
37) pixman/0.40.0-GCCcore-11.2.0                 119) pybind11/2.10.3-GCCcore-11.2.0
38) libffi/3.4.4-GCCcore-11.2.0                  120) hypothesis/6.14.6-gomkl-2022a
39) gettext/0.21-GCCcore-11.2.0-BuildNAOS        121) numpy/1.23.5-gomkl-2022a
40) PCRE/8.45-GCCcore-11.2.0                     122) SciPy-bundle/2022.05-gomkl-2022a
41) util-linux/2.38-GCCcore-11.2.0               123) libtirpc/1.3.2-GCCcore-11.2.0
42) GLib/2.72.1-GCCcore-11.2.0                   124) HDF/4.2.15-GCCcore-11.2.0
43) fontconfig/2.14.0-GCCcore-11.2.0             125) Eigen/3.4.0-GCCcore-11.2.0
44) xorg-macros/1.19.3-GCCcore-11.2.0-BuildNAOS  126) arpack-ng/3.8.0-gomkl-2022a
45) X11/20220504-GCCcore-11.2.0                  127) Armadillo/11.4.3-gomkl-2022a-MEISSA
46) cairo/1.17.4-GCCcore-11.2.0                  128) CFITSIO/4.2.0-GCCcore-11.2.0
47) Java/17.0.6                                  129) giflib/5.2.1-GCCcore-11.2.0
48) ICU/74.1-GCCcore-11.2.0                      130) json-c/0.16-GCCcore-11.2.0
49) NASM/2.15.05-GCCcore-11.2.0                  131) Xerces-C++/3.2.4-GCCcore-11.2.0
50) libjpeg-turbo/2.1.3-GCCcore-11.2.0           132) Imath/3.1.5-GCCcore-11.2.0
51) jbigkit/2.1-GCCcore-11.2.0                   133) OpenEXR/3.1.5-GCCcore-11.2.0
52) gzip/1.12-GCCcore-11.2.0                     134) Highway/1.0.3-GCCcore-11.2.0
53) lz4/1.9.3-GCCcore-11.2.0                     135) Brunsli/0.1-GCCcore-11.2.0
54) zstd/1.5.2-GCCcore-11.2.0                    136) Qhull/2020.2-GCCcore-11.2.0
55) LibTIFF/4.3.0-GCCcore-11.2.0                 137) LERC/4.0.0-GCCcore-11.2.0
56) HarfBuzz/8.2.2-GCCcore-11.2.0-MEISSA         138) libwebp/1.3.1-GCCcore-11.2.0
57) FriBidi/1.0.12-GCCcore-11.2.0                139) GDAL/3.6.2-gomkl-2022a-MEISSA
58) Pango/1.50.7-GCCcore-11.2.0-MEISSA           140) unixODBC/2.3.11-gomkl-2022a
59) Tcl/8.6.12-GCCcore-11.2.0                    141) GObject-Introspection/1.72.0-GCCcore-11.2.0
60) Tk/8.6.12-GCCcore-11.2.0                     142) pkg-config/0.29.2-GCCcore-11.2.0
61) Zip/3.0-GCCcore-11.2.0                       143) librsvg/2.54.2-gomkl-2022a-MEISSA
62) CMake/3.25.0-GCCcore-11.2.0                  144) phantomjs/2.1.1-gomkl-2022a
63) cmdstan/2.32.2-gomkl-2022a                   145) MPC/1.2.1-GCCcore-11.2.0
64) ATK/2.38.0-GCCcore-11.2.0                    146) gmpy2/2.1.2-GCC-11.2.0
65) SQLite/3.39.4-GCCcore-11.2.0                 147) sympy/1.9-gomkl-2022a
66) GMP/6.2.1-GCCcore-11.2.0                     148) PostgreSQL/14.4-GCCcore-11.2.0
67) Python/3.10.7-GCCcore-11.2.0-BuildNAOS       149) libpq/13.20-el9_5
68) Gdk-Pixbuf/2.42.8-GCCcore-11.2.0             150) texlive/20210325-GCCcore-11.2.0-MEISSA
69) six/1.16.0-gomkl-2022a                       151) rpmrebuild/2.11
70) GTK2/2.24.33-GCCcore-11.2.0-MEISSA           152) shinycannon/1.1.3-linux-x86_64
71) Ghostscript/9.56.1-GCCcore-11.2.0-MEISSA     153) MariaDB-connector-c/3.3.5-gomkl-2022a
72) JasPer/4.0.0-GCCcore-11.2.0                  154) quartocli/1.6.40
73) LittleCMS/2.13.1-GCCcore-11.2.0              155) tbb/2018_U5-GCCcore-11.2.0
74) ImageMagick/7.1.0-4-GCCcore-11.2.0-MEISSA    156) libarchive/3.7.7-GCCcore-11.2.0
75) GLPK/5.0-GCCcore-11.2.0                      157) texinfo/7.1-GCCcore-11.2.0
76) GSL/2.7-GCC-11.2.0                           158) hiredis/1.3.0-GCCcore-11.2.0
77) JAGS/4.3.2-gomkl-2022a                       159) libxslt/1.1.43-GCCcore-11.2.0
78) libgit2/1.4.3-GCCcore-11.2.0                 160) Abseil/20250512.1-GCCcore-11.2.0
79) libdrm/2.4.110-GCCcore-11.2.0                161) protobuf/31.1-GCCcore-11.2.0
80) libglvnd/1.4.0-GCCcore-11.2.0                162) R/4.3.1-gomkl-2022a-0.4
81) libunwind/1.6.2-GCCcore-11.2.0               163) R-bundle-Novartis/0.5-gomkl-2022a-R-4.3.1
82) LLVM/12.0.1-GCCcore-11.2.0

Key:
auto-loaded
[singhpuv@server  ~]$ time module purge

real    0m18.310s
user    0m17.923s
sys     0m0.045s
[singhpuv@server ~]$
_______________________________________________
Modules-interest mailing list
Mod...@li...<mailto:Mod...@li...>
https://lists.sourceforge.net/lists/listinfo/modules-interest<https://urldefense.com/v3/__https:/lists.sourceforge.net/lists/listinfo/modules-interest__;!!N3hqHg43uw!sNArrmBrygltUfMOX-dS-7baQKtka4F4PDBxyH5NTJHzy3uikm_0JVJeqyxhWicpzr1mzedkU-bkZ9oPYA87zKIMWC0_MMGL6nv16Q$>

Re: [Modules] tcl module load slowness debug

Manage your shell environment variables and aliases

Re: [Modules] tcl module load slowness debug