MCMC convergence with strong autocorrelation

[Simone Tenan]

Dear all,
I'm fitting a rather complex model to a large data set and for some parameters there is strong autocorrelation in the MCMC draws. I'm using the function parCodaSamples(), from the "dclone" R package, to generate posterior samples and the code works. Suppose I get the first bunch of posterior samples, called "o1" using the following code:

timer <- proc.time()
cl <- makePSOCKcluster(chains)
parLoadModule(cl, 'glm')
parLoadModule(cl, 'lecuyer')
parLoadModule(cl, 'dic')
# adaptative phase
parJagsModel(cl = cl, name = 'ipm_Agull_5C_v4a', file = "ipm_Agull_5C_v4a.txt", data = bugs.data, n.chains = chains, n.adapt = n.adapt)
# burnin
parUpdate(cl = cl, object = 'ipm_Agull_5C_v4a', n.iter = n.update)
# generate posterior samples
o1 <- parCodaSamples(cl = cl, model = 'ipm_Agull_5C_v4a', variable.names = parameters, n.iter = n.iter, thin = thin)

but autocorrelation is still strong for some parameters in the output "o1". In addition, "o1" is a large object which takes a lot of memory.

My question is: Can I remove object "o1" to release some memory and run the function again in the same R session in hopes of achieving convergence?

o2 <- parCodaSamples(cl = cl, model = 'ipm_Agull_5C_v4a', variable.names = parameters, 
                     n.iter = n.iter, thin = thin)

If yes, does this new run start from where o1 left (even if "o1" is no more present)?

Thank you!
Regards,
Simone

[Martyn Plummer]

You might ask the maintainer of the package, but my first impression is yes. This is how it works in the rjags package and it looks like the dclone package works in the same way with added parallelism.

To save memory when you have high autocorrelation, you should increase the thinning interval.

If you really want to remove memory in an R session then you should run gc() to force garbage collection.