Proposed IPC and shared memory architecture

[Charles Anthony]

I want to migrate dps8m towards a model where the CPUs and I/O devices run as independent processes. This will allow true asynchronous I/O and CPU execution.

The DPS8M h/w features a main memory architecture that allows multiple CPUs and I/O devices direct access to main memory. This suggests that a model where the DPS8M main memory would be implemented as shared memory would be a natural component of the emulator.

DPS8M memory accesses

Disk drives <---+---> IOM <----> SCUs <----> CPUs
Tape drives <---|
Console <-------|
FNP <-----------|

The main memory resides in the SCUs (System Control Units); four megawords per SCU, with a maximum configuration of four SCUs.

(The SCUs are simple; they hold memory, and route connect and interrupt signals among the CPUs and IOMs. I suspect that the correct architecture is that the SCUs live in the "main process".)

Features needed:

* Session name space   

    The architecture should support more then one instance of the emulator running on a
    system; the IPC and shared memory need to include some form of session ID, allowing
    peaceful coexistence.   

* Shared memory   

    * Main memory in shared memory   

    * Possibly: Cabling and configuration in shared memory. This would keep the
      processing of the configuration data (base_system.ini, 20184.ini) in the "main"
      process, as CPU and device processes are started, they look in this shared memory
      for configuration data.  

    * Possibly: CPU and peripheral state data in shared memory. This greatly simplifies
      blinkenLights; rather then having code scattered throughout the CPU emulator
      harvesting data, it just looks in the shared memory segment. Arguably, the
      gathering up of all of the CPU state data into a structure is good s/w engineering
      practice.  

* IPC (this list is off the top of my head; not definitive)  

    The CPUs send "connect" messages to the SCUs.  

    The CPUs send "write register" messages to the SCUs.  

    The CPUs send "read register" messages and expect a synchronous reply.  

    The SCUs sends "connect" messages to the IOMs and CPUs.  

    The SCUs sends "interrupt" messages to the CPUs  

    The IOMs send I/O commands to the peripherals.  

    The peripherals send "list service" messages to the IOMs  

    The peripherals send "indirect list service" messages to the IOMs  

    (The "list service" is low-bandwidth; it is a request to be sent the next I/O command. "Indirect list service" is potentially high bandwidth as it is a DMA request.)  

* Portability  

     The targeted platforms are Fedora, Ubuntu, MacOS, Windows (7?).  

[Harry Reed]

I like your ideas. After producing a more-or-less stable release we may want to also ponder .....

Do we want to stick with SIMH? or roll our own? Perhaps we would want to extend SIMH to add process management, IPC, SHM, etc? (all in a platform agnostic manner.) Not that the SIMH folks would want to accept our contributions, but we could come up with a SIMH extention library that has the missing bits and pieces we need and make that available for others.

Do we want to stay with C? C++ is wonderful for encapsulation, but alas, SIMH is in C, but we could work around that. Other languages?

Other playforms -- 32-bit? Android? IOS? Windows (Native and/or Cygwin.)?

And there's always optimizing what we have. Surely our 5,000 MIP / core CPUs can emulate a 1.8MIP machine, shall we say, a tad faster than what were getting?

Also don't forget about FNP <=> FNP for Multics internetworking.

What libraries to we want to keep or abandon?

Thoughts?

[Charles Anthony]

Caution: these are my personal opinions and thoughts, and are not necessarily
representative of the project members, or for that matter, reality.

What does simh offer?

The set/show state management

    The code is currently heavily dependent on this for configuration
    management (base_system.ini, 20184.ini, but it's limitations are
    showing up as a significant amount of helper code (cfg_*).

    Replacing this with in-house or a 3rd party interpreter should
    be quite feasible and should be considered.

The "activate" mechanism for emulation of asynchronous events

    By breaking the emulator up into actually independent processes, we
    lose all reliance on this mechanism.

The sim_instr() loop.

    There is an interesting philosophical distinction between simh's 
    view of CPUs and the DSP8M CPU. Simh implicitly believes that CPUs
    have a run/stop state; the DPS8M CPU is always running -- at power
    on with zero-initialized memory, it goes into a trouble fault cascade;
    the boot program "starts" the boot process by dropping the entry 
    address of the boot program into the trouble fault vector location.

    In simh, the simh command processor is always running; as a side effect
    of some simh commands, it will decide to start the sim_instr loop. To
    execute more simh commands, the CPU is paused by sim_instr's decision 
    to return to the simh command processor.

    This worked very well during development, but is a bit of a broken 
    paradigm for a multi-core, always running mainframe OS.

    As an example, simh would have you "mount a tape" on the tape drive
    with the attach command, but pausing the CPU in a multiuser OS to 
    type in the command is not attractive. We synthesize the command,
    but we lose all of the nice command parsing, and have to reach into
    the simh internals to do so. Awkward. There are soon going to many
    more examples of this kind of code, and I wonder if we should 
    reconsider the mechanism.

Breakpoints/instruction tracing/state examination

    This has been invaluable during development, but is rapidly becoming
    vestigial; the infrastructure is just not up to dealing with the
    asynchronous, multiprocessing, segmented, dynamically linked, and
    long running OS.

The terminal multiplexor.

    I do not trust the terminal multiplexor; it is tied intricately 
    into the simh architecture, and it's code is unreadable by me.
    I fully intend to replace the terminal multiplexor code with
    libtelnet and/or ssh.

Clock management.

    I have always been more interested in a maximum performance
    implementation as opposed to an accurately timed emulation, so
    I have no interest in the clock management.

I am of the belief that simh is of decreasing importance to the emulator, and
should be allowed to wither on the vine.

I am contemplating that for the CPU process, shedding simh entirely. The cable and configuration management would be done with simh in the main process, with the data communicated to the CPU processes through shared memory.

Language:

I am contemplating the value of http://ooc-lang.org/

I have an rather strong dislike for C++; I think they have lost the narrative thread. (Also, I have philosophical issues with languages that generate more efficient code for "++p" then "p++").

The language feature that I think would help the most is a strictly enforced way of doing 15/18/36/72 signed unsigned math -- specifically no automatic
up-casting.

word18 foo, bar;
int baz;
foo = bar + baz;
            ^ error: type mixing.

This is probably doable in C by making the subtypes opaque structure, and defining math functions across the structures, but I fear for the readability of the code, and the possibility of the generated code being inefficient.

This is certainly doable in C++; but I lack both the skills to write the classes, and the trust that anyone else would do it right.

Optimization: Always start any optimization by (re)reading "Programming Pearls" and "More Programming Pearls"

IPC worries me. I like ZMQ in principle, but they do seem to be having their fair share of windows issues, and I think that the zyre chat mechanism was a good first pass at IPC. I think I am searching for a "more tightly coupled" mechanism -- it is too easy for nodes to drop in and out. When the CPU is running, it needs to know that it is connected to a particular IOM, and that its dropping out and back in is not acceptable behavior. ZMQ does have other libraries built on it; one of them may have a good approach to IPC.

Also, an automatic launch mechanism might be nice; when the main process sends its first message to CPU-B, if the process is not running, it should be launch and connected automagically. I am uncertain about this; it may be that the
main process should explicitly launch everything; but then you get into issues of child management. I need to think a look more about this.