Print Capture 'void' during iter distribution
Status: Alpha
Brought to you by:
kellrott
During the creation of an array of data from a user 'iter' object, user code is run. At this point, if the user prints statements, those will be captured in the buffer, but not returned to the master process until the clear buffer command is called at a later time (usually after the next work load is completed). In one example, debug STDOUT produced during a very large iter process produced enough data to cause a memory error before the buffer was cleared.
Easy fix: add a buffer clear after ever call to the user iter. Remember, during the array construction, the master is waiting for a completion signal from the worker. So a buffer clear would have to occur on both the master and worker nodes.