using parallelization with a custom JAGS module

[František Bartoš]

Hi,

I'm developing a package that uses runjags and I wanted to add chain parallelization. However, I'm getting an error because my JAGS module that I distribute with the package as can't be found. I found an argument that is dedicated for the modules specification, but I'm not able to make it work.

There is a reproducible example (the non-parallel alternative works perfectly) :

library("RoBMA")

model_syntax <- '
model{
for(j in 1:2){
  eta[j] ~ dgamma(1, 1)
}
for(j in 1:2){
  std_eta[j]  = eta[j] / sum(eta)
  omega[j]    = sum(std_eta[1:j])
}
for(i in 1:10){
  t[i] ~ dwt_1s(10, 0, crit_t[i,], omega)
}
}'

data <- list(
  t      = rt(10, 10),
  crit_t = matrix(1.96, ncol = 1, nrow = 10)
)


runjags::autorun.jags(
  model           = model_syntax,
  data            = data,
  monitor         = "omega",
  n.chains        = 2,
  method          = "rjparallel",
  modules         = "RoBMA"
)

[Matt Denwood]

Hi František

Your example works fine for me on R 3.6.3 for macOS (CRAN build). But I am behind with updating R (and a lot of other things) due to uexpected commitments working on COVID so it might be that this is an issue under R 4.x

Can you confirm on what version of R (and what platform) you see the error and also describe the problem in more detail? Then I will try and look into it when other duties permit.

Note: I would NOT expect the "parallel" or "background" methods to work (and they don't) but the "rjparallel" method should work (and does for me).

Cheers,

Matt

[František Bartoš]

Hi Matt,

thank you for the quick reply! I'm also running 3.6.3 but on Windows. I quickly checked with friends and they verify that it runs on MacOS and Linux, but also have problems with Windows.

Also, the modules argument does not seem to be neccessary.

I get the following error on Windows:

> runjags::autorun.jags(
+   model           = model_syntax,
+   data            = data,
+   monitor         = "omega",
+   n.chains        = 2,
+   method          = "rjparallel",
+   modules         = "RoBMA"
+ )

Auto-run JAGS

Running a pilot chain...
Compiling rjags model...
Starting 2 rjags simulations using a PSOCK cluster with 2 nodes on host 'localhost'
Error: The following error was encountered while attempting to run the JAGS model:  
   One or more rjags sessions failed with the following error:
Error in checkForRemoteErrors(val) : 
  2 nodes produced errors; first error: Failed to load the module 'RoBMA'

The worker log file (which may also help with debugging) is:
starting worker pid=24984 on localhost:11940 at 10:40:41.995
starting worker pid=6320 on localhost:11940 at 10:40:42.234
Loading required package: runjags
Loading required package: runjags
Error in rjags::load.module(extra.options$modules[[i]][1]) : 
  File not found: C:\Program Files\JAGS\JAGS-4.3.0/x64/modules/RoBMA.dll
Error in rjags::load.module(extra.options$modules[[i]][1]) : 
  File not found: C:\Program Files\JAGS\JAGS-4.3.0/x64/modules/RoBMA.dll
Have you remembered to specify all required modules and factories?
In addition: Warning message:
No initial values were provided - JAGS will use the same initial values for all chains 
> 

I suppose that the problem is that rjags is looking for the module in the JAGS installation directory (C:\Program Files\JAGS\JAGS-4.3.0/x64/modules/RoBMA.dll) instead of the R package directory (C:\Users\fbart\Documents\R\win-library\3.6\RoBMA\libs\x64/RoBMA.dll).

The runjags.dll is also installed in the runjags R package dirrectory, however, the following example using the runjags JAGS module runs without a problem.

m <- "
model{
L ~ dlomax(1,1)
}
"
results <- run.jags(
  model    = m,
  monitor  = "L",
  n.chains = 2,
  method   = "rjparallel"
)

I guess that I need to add the path to the RoBMA module to a module search path for runjags, but I'm not aware of any way of doing this.

Cheers,
Frantisek

[Matt Denwood]

OK thanks - this reminded me that there is a difference between a Fork cluster (default on macOS/Linux) and PSOCK cluster (the only thing available on Windows) in that the latter does not share the full environment of the parent process. By forcing my machine to use a PSOCK cluster I was able to replicate your issue.

The solution is to set up the cluster manually and make sure your package is loaded before running the model:

library("parallel")
cl <- makePSOCKcluster(2)
clusterCall(cl, function(x) require("RoBMA"))

runjags::autorun.jags(
  model           = model_syntax,
  data            = data,
  monitor         = "omega",
  n.chains        = 2,
  method          = "rjparallel",
  modules         = "RoBMA",
  cl = cl
)

Hopefully that will work for you as well (I think PSOCK clusters work the same on macOS and Windows but I might be wrong). Also, you are right that the modules argument is not needed here as the module is not one of the standard JAGS modules.

Cheers,

Matt

[František Bartoš]

Dear Matt, thank you very much! It works on Windows as well.
Thank you for your time and have a nice rest of the day.
Best,
Frantisek

[Nasser Najibi]

Hi Matt,

I had a sort of similar problem with the dgenpareto module in R jags. Recently, I have updated my R, Rstudio, and rjags packages. Below are the platform/versions I am running my R scripts on/with:

version _
platform x86_64-w64-mingw32
arch x86_64
os mingw32
crt ucrt
system x86_64, mingw32
status
major 4
minor 2.0
year 2022
month 04
day 22
svn rev 82229
language R
version.string R version 4.2.0 (2022-04-22 ucrt)
nickname Vigorous Calisthenics

library(runjags)
Warning message:
package ‘runjags’ was built under R version 4.2.1

library(rjags)
Loading required package: coda
Linked to JAGS 4.3.1
Loaded modules: basemod,bugs
Warning message:
package ‘rjags’ was built under R version 4.2.1

library(R2jags)
Attaching package: ‘R2jags’
The following object is masked from ‘package:coda’: traceplot
Warning message:
package ‘R2jags’ was built under R version 4.2.1

I am running a jags model that contains the generalized Pareto distribution (GPD) as 'dgenpareto'. I use the following command lines to run my jags model:

jags.parallel(model.file = bayes_gpd,data = jags.data,parameters.to.save = jags.params,
inits = jags.inits,n.iter=50000,n.chains = 4,
jags.module='runjags')

But I get the following Error message each time I try to evaluate my jags model:

Error in checkForRemoteErrors(val) : 4 nodes produced errors; first error: File not found: C:\Program Files\JAGS\JAGS-4.3.1/x64/modules/runjags.dll

While I had no issue with running my jags model (the same code) before updating the R and jags packages (~ early 2022).

Could you please provide some hints on how I can resolve this issue? I also tried to post this question to StackOverFlow. Thanks!