Re: [micro-manager-general] Saving images at high speed
Status: Beta
Brought to you by:
nicost
Menu
▾
▴
Re: [micro-manager-general] Saving images at high speed
|
From: PAVAK S. <pavak@g.ucla.edu> - 2020-04-09 17:23:02
|
This conversation led me to do a little bit of low stakes testing. Turns
out that by default, the JRE on my system selects the parallel GC (it seems
that this is an automated choice based on core counts and OS). The parallel
GC supposedly supports the maximum throughput in memory turnover, but can
cause long-ish stop-the-world pauses during this. The "blips" I observed
during high framerate acquisitions were probably these events.
Switching to a low-latency GC explicitly when launching the JVM,
specifically the Concurrent Mark Sweep Collector, completely eliminated
this behavior for me with small frames. With large frames (2100 columns x
400 rows) acquiring at the same 280 effective FPS does cause some frames to
pile up in the sequence buffer, but far more slowly than with the parallel
GC and a 3,600 volume acquisition at 4 volumes per second (x 2 cameras x 35
planes per volume) can be acquired with ~ 5 GB of sequence buffer without
completely filling it up.
It seems that it should be possible to further tune the parallel GC, for
example by increasing the number of threads it's allowed to use (normally
this is a fixed fraction of the total available) and by setting a maximum
pause time. Based on my ~3 hour old understanding, the main advantage of
parallel collection is more efficient heap utilization, but with memory as
cheap as it is these days for folks that are really trying to push the
limits of acquisition rates, maybe it's not so essential. Time permitting
I'll play around a bit and report back if any findings become more
definitive.
I did start getting a few of these errors during the acquisition, but I'm
not positive yet that they're completely related the choice of GC:
2020-04-09T10:09:34.628451 tid10392 [IFO,App] Error: Error reading image
metadata from file
2020-04-09T10:09:34.628451 tid10392 [IFO,App]
[ ]
org.micromanager.utils.MMScriptException: MMScript error: Can't figure out
pixel type in Thread[AWT-EventQueue-0,6,main]
[ ] at
org.micromanager.utils.MDUtils.getPixelType(MDUtils.java:240)
[ ] at
org.micromanager.utils.MDUtils.isGRAY8(MDUtils.java:349)
[ ] at
org.micromanager.utils.MDUtils.isGRAY(MDUtils.java:389)
[ ] at
org.micromanager.utils.MDUtils.isGRAY(MDUtils.java:397)
[ ] at
org.micromanager.imagedisplay.AcquisitionVirtualStack.getPixels(AcquisitionVirtualStack.java:146)
[ ] at
org.micromanager.imagedisplay.AcquisitionVirtualStack.getProcessor(AcquisitionVirtualStack.java:164)
[ ] at
ij.CompositeImage.updateImage(CompositeImage.java:252)
[ ] at
org.micromanager.imagedisplay.MMCompositeImage.superUpdateImage(MMCompositeImage.java:136)
[ ] at
org.micromanager.imagedisplay.MMCompositeImage.updateAndDraw(MMCompositeImage.java:172)
[ ] at
ij.ImagePlus.updateAndRepaintWindow(ImagePlus.java:316)
[ ] at
ij.ImagePlus.setSlice(ImagePlus.java:1510)
[ ] at
ij.gui.StackWindow.setPosition(StackWindow.java:277)
[ ] at
ij.ImagePlus.setPosition(ImagePlus.java:1377)
[ ] at
org.micromanager.imagedisplay.HyperstackControls.onSetImage(HyperstackControls.java:436)
[ ] at
sun.reflect.GeneratedMethodAccessor31.invoke(Unknown Source)
[ ] at
sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
[ ] at
java.lang.reflect.Method.invoke(Unknown Source)
[ ] at
com.google.common.eventbus.EventSubscriber.handleEvent(EventSubscriber.java:74)
[ ] at
com.google.common.eventbus.SynchronizedEventSubscriber.handleEvent(SynchronizedEventSubscriber.java:47)
[ ] at
com.google.common.eventbus.EventBus.dispatch(EventBus.java:322)
[ ] at
com.google.common.eventbus.EventBus.dispatchQueuedEvents(EventBus.java:304)
[ ] at
com.google.common.eventbus.EventBus.post(EventBus.java:275)
[ ] at
org.micromanager.imagedisplay.VirtualAcquisitionDisplay.doShowImage(VirtualAcquisitionDisplay.java:724)
[ ] at
org.micromanager.imagedisplay.VirtualAcquisitionDisplay.access$700(VirtualAcquisitionDisplay.java:87)
[ ] at
org.micromanager.imagedisplay.VirtualAcquisitionDisplay$2.run(VirtualAcquisitionDisplay.java:695)
[ ] at
java.awt.event.InvocationEvent.dispatch(Unknown Source)
[ ] at
java.awt.EventQueue.dispatchEventImpl(Unknown Source)
[ ] at
java.awt.EventQueue.access$500(Unknown Source)
[ ] at
java.awt.EventQueue$3.run(Unknown Source)
[ ] at
java.awt.EventQueue$3.run(Unknown Source)
[ ] at
java.security.AccessController.doPrivileged(Native Method)
[ ] at
java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown
Source)
[ ] at
java.awt.EventQueue.dispatchEvent(Unknown Source)
[ ] at
java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
[ ] at
java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
[ ] at
java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
[ ] at
java.awt.EventDispatchThread.pumpEvents(Unknown Source)
[ ] at
java.awt.EventDispatchThread.pumpEvents(Unknown Source)
[ ] at
java.awt.EventDispatchThread.run(Unknown Source)
Best,
Pavak
On Wed, Apr 8, 2020 at 5:37 PM JonD <jo...@as...> wrote:
> PAVAK SHAH-3 wrote
> > I understand completely and appreciate the explanation. It's helpful to
> > have an idea of the scope of the endeavor and what it may take to make it
> > possible.
>
> We "just" have to clone Mark and Nico and secure funding for said clones
> ;-)
> Or maybe suitable clones can be found somewhere if funding became
> available.
>
>
>
> --
> Sent from: http://micro-manager.3463995.n2.nabble.com/
>
>
> _______________________________________________
> micro-manager-general mailing list
> mic...@li...
> https://lists.sourceforge.net/lists/listinfo/micro-manager-general
>
--
Pavak K Shah
Assistant Professor
Molecular, Cell and Developmental Biology
Institute for Quantitative and Computational Biosciences
5000C Terasaki Life Sciences Building
University of California, Los Angeles
www.connectiontofunction.org
|
