zmq_term.vi causes error in zmq_send.vi even though the former hangs on the open socket

[Stobber]

Here's a little example application I wrote that dynamically launches a server VI and two client VIs in the same app context. The VIs are launched using the ACBR node so they will all be running in their own VI hierarchies while using the same 0MQ context.
https://dl.dropboxusercontent.com/u/19403476/REQ-REP%20Dynamic%20Example_lv12.zip

When the clients are stopped (by typing "stop" into the message box on each client), the second client reports an ETERM error in the zmq_send VI. The exact sequence of actions among the four running VIs is:
1. Send "stop" from client 1 to the server.
2. Server replies with "stop ACK" and exits, closing its socket.
3a. (Racing with 3b) Client 1 exits, closing its socket.
3b. Main VI sees server has stopped, calls zmq_term.vi, which hangs because client sockets are not all closed.
4. Send "stop" from client 2. Client 2 throws error in zmq_send.vi and exits, closing its socket.
5. Main VI's call to zmq_term.vi returns without error.

Setting "LINGER = -1" on every socket doesn't prevent the error.

If the zmq_term VI is hanging and unable to close the context because sockets are open, why does the zmq_send VI return an ETERM error when its socket is still open? I expect everything to work in client 2 until it closes its socket, at which point I expect zmq_term.vi to return without error.

[Martijn Jasperse]

This is expected behaviour from the underlying ZMQ library. Calling term signals that you're done with the context and everything associated with it, so threads using it should clean up. The hanging is a contract between threads to call zmq_close after context termination, so that when ctx_destroy returns you know the clean-up is "done". This turns out to be important for complicated messaging topologies.

So ETERM is the error message that lets you know a call failed because the context is gone, and the next call should (must?) be a socket close. It makes no sense for the call to return without an error because the call you tried to make did fail.

Potentially the error handler for zmq_close could remove ETERM from the error stack if encountered, but there's no way to know that the socket that raised ETERM is the one being closed, so this is left to the user.

Hence ETERM is a normal part of ZMQ flow, so checking for/removing ETERM before/after zmq_close is a sensible thing to do.