LTK emulation patch - adds Lt. Kernal host adaptor cartridge support.
I also reordered some of the case statements in c64cart.c so the cartridges were in the standard alphabetical order.
generally the code looks ok to me, i can't comment on any of the LT.Kernal specific things. One thing i am not sure about is - if this is a "SCSI Host Controller", then isnt it possible to use other SCSI devices than HDs with it? If so, the actual HD emulation should be splitted off, so other devices can be added some day eventually. Also, doesnt the cartridge contain a MC68B21P? Where is this emulated? Note that "chip" emulation should go into a seperate file in "cores" and work generically.
There are mostly style issues and a few missing things:
+ ltkernal_io_device.start_address = 0xdd00 + ltk_io * 256;
shouldnt this be 0xde00 ?
note that this should be the "official" name, as written on the actual product and/or it's documentation.
at a couple places there are "magic values" used in the code, followed by a comment explaining what it is. i suggest using defines for those.
includes in ltkernal.h should be moved to ltkernal.c
then there are a couple codestyle issues:
there are some // comments, please replace them by regular c-style comments
there are many other cases of missing spaces, missing curly braces, indention style different from what we use, etc. please read coding-guidelines.txt - if something isnt clear, please ask so we can explain it better in that file. For example:
for(i=0;i<512;i++) ltk_scsi_data_buf[i]=0;
should be
for(i=0; i < 512; i++) {ltk_scsi_data_buf[i] =0;
}
Things that are missing:
callbacks for the .crt format in ltkernal.c
changes to cartconv.c to actually handle that format
documentation for all of the above in vice.texi (commandline options, resources, crt format)
...that should give you something to work on :)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I didn't want to abstract the device down to individual parts. I realize this is an ideal, but it certainly complicates the implementation and I was striving for functional at this point since we have nothing. I can always add this in the future, but I think getting something out there that people can use would be good. I have the full DOS code and have disassembled it so I was able to determine which chip features are used; the basically just use the MC68B21P as parallel IO. No interrupts or other features are being used. That being said, I finally was able to aquire a Lt. Kernal; all my code up to this point was arrived at by code inspection. I plan to examine the circuit to see if there is anything else to gain from it, but they do have some PLDs on there which may prove difficult to reverse engineer. Some has done all this work (mytec) but they refuse to share anything for the emulation project (they are happy reproducing old hardware and selling it).
I also have a CMD HD emulation (SCSI) which is my next task to intergrate into SVN, but again the SCSI is integrated (right now). Yes, I would like to separate them, but I feel it is important to get something into VICE as I'm wasting time patching around things that are changing. I started this work on 3.3, and spent the last 2 weeks moving it over to 3.4. Now, it turns out, my vdrive improvements (native partition support) need a lot of changes since dual drive support was added in the SVN (it was quite a hack from what I can see). That will take some time to fix.
I can certainly make those other changes you mentioned. I wasn't aware the group & freeze thing had something to do with the UI. Adapter/Adaptor - tomato/tomatoe. :) I'll fix it.
I wasn't sure about the CRT thing. The ROM is just a dump. There is no CRT type for it. How do we go about setting one up? Do I just pick an "ID" for it?
Is there a place for extended docs? (d64 images, etc?) It turns out the main source for lt. kernal info on the web is wrong in a lot of cases when it comes to configuration, and the author passed away a while ago. So no one is there to maintain it. I could make my own website, but I would prefer something with more permanence. I noticed that a lot of the cartridges refer to pokefinder.org. Is that an option?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
As far as other scsi devices, I've hear "rumors" about tape drives, but I haven't found anything. The device was limited to 20MB and 40MB hard discs. I was going to ask on lemon64 to see if anyone has anything they can contribute.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I totally forgot - we already have a 6821 emulation core, see cores/mc6821core.c - currently it is used by dolphindos3, magicformel and formel64 (so you can look at those as an example). the emulation core isn't complete (no interrupts iirc), but if you need that, it should work..
As for seperating SCSI - if you are planning to add CMD-HD too, i definetly suggest doing this. I am very reluctant to merge things when it is known they have to get reworked like this - because the past tells us that once it is in, it wont get fixed.
About the CRT type - you already picked it - this is what the +#define CARTRIDGE_LT_KERNAL 72 /* ltkernal.c */
is about :) your new crt id is 72. If you look at cartconv.c, you will find a list of all types around the start of the file, just make a new entry at the bottom of the list with the respective type (copypaste it from a cartridge that comes close to what you need and edit it accordingly) and it should work. then you only need to add the two hooks to your cartridge implementation at add the type to crt.c, should be straightforward.
extended docs are out of the scope of VICE - we only document the emulator specific things (pokefinder is very "freezer cartridge" centric, i think lt.kernal is out of scope there too)
as for "getting it into VICE", you should just keep developing in the svn trunk, then there shouldnt be problems with merging. we are not in a hurry :)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
If I separate the SCSI into another module, then there will be more additions. The CMD HD requires a number of patches as I'm introducing a new image type. This also requires modifications of the image handler and vdrive as well. The current hack to vdrive to add "drive 1" has to be reworked to support up to 255 partitions on CMD HD images. That will take longer. It ends up being a lot of changes and I will be concerned that you won't want to integrate that many changes at once (my last count is about 30 files). So we have to have a compromise here. I can guarantee you that I will be working on this stuff for a while; I started these modifications 2 years ago. I wrote the REL file handler for vdrive about 15 years ago. The only thing that paused my contributions were having kids (they take a lot of time from you), but they are older now and I have more time.
In the meantime I will work on the CRT and docs and see if I can add in the 6821 or separate the SCSI system.
You can always classify this as experimental/evolving as it won't affect anything unless you turn it on.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
well the drive number isnt a partition, partitions need to be handled seperatly - it has nothing to do with scsi emulation either, vdrive is something totally different. "SCSI Emulation" should be only about emulating SCSI devices, vdrive is unrelated to this. for "SCSI emulation" we'd want an API similar to the IEC emulation, ie emulate the bus in a way that will allow to hook up arbitrary SCSI devices to it. You may want to look at how clockport devices are being handled at the moment as an example.
Last edit: gpz 2020-10-03
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
In 3.4, vdrive parsed the drive number in the file name (or would have, but it wasn't complete), eg. "0:$" or "1:$". From what I can see now, that parsing has to be moved to a higher level as each vdrive instance isn't a drive, but a drive/unit now (eg. 8,0 is different than 8,1). This isn't proper as drives should be treated as a whole, with all their units. So I should be able to do the Duplicate command, or Copy command between units (for the CMD, that is up to 255 of them all possibly being different image types). That doens't work now in the svn, and to make it work would be hard. The proper solution would have been to add a second (or more) image to each vdrive and handle it all in there. So the current version it is kind of a hack and will definately need to be reworked.
Reworking it will take some time as I have to track it all back up the chain of functions.
As far as SCSI goes, it seems like you are proposing that each drive ends up being a separate processing system, like TDE. This is far from optimal. Right now I don't treat the host and drives as separate units; I streamlined the communications to avoid the headaches of shared buses. I know what you are saying, but it is needlessly complicated for what these devices need. Both CMD and Xetec were using some of the first SASI/SCSI drives and they had their issues (not completely standard). I can already tell you their DOS's were designed with specific drives in mind and abstracting this to some generic unit will likely have problems. CMD had to change their boot rom each time a new drive was introduced. Lt. Kernal doesn't work with large drives as they keyed the filing system around the number of heads and sectors per cylinder. These are all special considerations to make things work. As much as I'd love to go and fix their individual DOS's to be more accepting of other drives, I can't. I'm making it work for their latests DOS's which are over 25 years old. Maybe, in the future, I can improve those, but this is the priority now.
As I see it there is a clear lack of large volume storage support in VICE. Sure, you have fsdevice, but it doens't support REL files (maybe a little now). So any one who wants to use that is limited to TDE on that standard drives. Both CMD and LTK solved this, and my updates to vdrive add to that. So I'm trying to get those out there.
I know you're not in a rush to add things, but everytime you release something new, it usually ends up effecting some part of what I'm working on. (eg. vdrive - no one touched it until a few months ago, and it seriously needs some love).
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
as said, vdrive is unrelated here, and needs different considerations that should be discussed first. (and not here, lets keep the ticket clean from this) When you are planning larger changes to existing code, it should be discussed first how we want to do it, and if what you have in mind doesnt clash with what other people are already working on.
as for SCSI, the implementations of cartridges (or other hardware) should never be only for a specific software, but in theory work for whatever software could be run on a device/cartridge. i know its not needed right now and seems like overengineering first - but it will pay off long term. even if there is no other software right now, and even if the implementation only works with the existing software right now, the seperation should be done and prepared for when someone wants to extend this further. ie the long term goal should be that IF someone will ever implement "TDE" style CMD-HD emulation, the same SCSI emulation code can be used for that. in a similar vein, we also have "ATA" emulation, which IDE64 uses (see cores/ata.c). the resason is a similar one as why the code should use the m6821 core - we want to make sure to use generic shared code whenever possible, and not duplicate any code anywhere. this was a huge problem in the past, and we are just slowly coming to the point where its mostly cleaned up.
Last edit: gpz 2020-10-03
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have begun to look at your MC68B21 code, and there is a FIXME part that I need done, but I don't understand what needs to be fixed other than explaining what "CA2state" is and how it differentiates between the two modes. It looks like it works for the MC6821_CTRL_C2_STROBE_E, but not the MC6821_CTRL_C2_STROBE_C mode and there is no clear difference in the code for them so it seems incomplete. I don't know what your plans are for this.
As far as the SCSI system goes, I have implemented CMD HD TDE, but I didn't need a full SCSI backend system to do it. The same can be said for the LTK. I emulated the hardware so that the software can co-exist with it and behave correctly.
The point I'm tyring to make is VICE is an emulator, not a simulator. And emulator provides functionality at the expense of loss some in timing accuracy, where as simulation is the ultimate desired goal, but we can't make VICE work like a transistor level simulation; we just don't have all the information, and it would never be finished, and therefore useless - not to mention too slow. We have to compromise, and we did all over.
I can generilize the SCSI system to work with both devices for 99.9999% of the use cases, but it isn't worth trying to implement support for all devices. There are far too many. Basic disks are fine, and if we need more we can go from there. The code is easy to add to.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Actually the original definition of "emulator" was that it was a better recreation of the original system than a mere simulator...
VICE does try to be as timing-exact as possible, that's why x64sc exists. Talking about a transistor-level simulation is just a false dilemma in this context, for the purposes of VICE it does not gain you anything that you could not recreate using a higher-level emulation approach - the well-known visual6502 model is in fact less accurate than the CPU emulation in VICE for some illegal opcodes.
There is a big argument why it would be a good idea to implement a real SCSI layer in VICE: You can interface it to real SCSI devices of the host, so a user could for example take the HDD from his real CMD HD, connect it to his PC and access it from VICE.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I can access a real CMDHD connected to the SCSI bus now. The disk access is facilitated by the OS, not me. I just use fopen, fseek, fread, fwrite, fclose and all is handled. I use ftell to report the drives size.
Is it necessary to pass the SCSI commands, un-altered, to the scsi drive?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I don't understand what needs to be fixed other than explaining what "CA2state" is and how it differentiates between the two modes. It looks like it works for the MC6821_CTRL_C2_STROBE_E, but not the MC6821_CTRL_C2_STROBE_C mode and there is no clear difference in the code for them so it seems incomplete. I don't know what your plans are for this.
the plan is to emulate the chip as accurately as needed, because that makes it generic and can be used in all pieces of the emulator that use that chip - and the answer to what is missing and needs to be done is in the datasheet of the chip. i haven't looked at it since i implemented whats there, so i cant really tell what exactly is missing or needs to be done.
and yes, we do indeed for aim for very high accurracy these days, and want to implement things like they really work, not as some blackbox approach. this aim was different back in the days when the project was started.
I can generilize the SCSI system to work with both devices for 99.9999% of the use cases, but it isn't worth trying to implement support for all devices. There are far too many. Basic disks are fine, and if we need more we can go from there. The code is easy to add to.
you don't have to support "all devices" at all. just like the clockport stuff, it can be splitted into a "bus" API, and the actual emulation of a device. when we have that, additional devices can always be added without touching any of the other code. or even made to talk to actual real scsi devices, as ingo suggested.
Last edit: gpz 2020-10-04
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
the plan is to emulate the chip as accurately as needed, because that makes it generic and can be used in all pieces of the emulator that use that chip - and the answer to what is missing and needs to be done is in the datasheet of the chip. i haven't looked at it since i implemented whats there, so i cant really tell what exactly is missing or needs to be done.
My point is I don't see what is missing about MC6821_CTRL_C2_STROBE_E. That part seems complete, but you have FIXME messages about it.
re: scsi bus, to clarify, you want to be able to, in the end, send CDB commands directly to real SCSI devices? As it is right now, you should be able to specify a disk image device of "/dev/sdf" to use a full scsi disk. It won't pass the CDB commands, the OS will do that, as we just read/write to data mostly. It reports the "size" of the disk, but that is it. I only handle the basic SCSI commands and the ones I know the DOS's use.
If this is the case, then why doesn't core/ata.c do this? It emulates the drives behaviour and responses, but doesn't pass any of the ATA commands along to a real drive.
Can the emulated or TDE 1581 read and write to real CBM disks using a 3.5" PC floppy? I don't think it does.
If this is the case, then why are we expecting this level of operation from SCSI when we know it most likely will be only ever used with disk images?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
(aside about vdrives etc: my current thoughts on multiple images in a vdrive are that vdrives should grow some over-arching vunit, like TDE drives where a diskunit_context_t contains 1 or more drive_ts. This would handle multiple drives, and partitions. But I haven't dared to restructure this. I am still too scared of the LED handling for the GUI.)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
My changes to vdrive correct a lot of issues. it supports DHD images (CMD HD) with multiple parititions, sub directories, full paths, duplication/copying between partitions. etc.
I only have to make a dew changes for "drive 0/1" support on the CBM dual drives.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Please, this discussion is getting out of hand. Move the vdrive stuff into #vice-dev, it's 100% offtopic here.
Also, the fact something else isnt (yet) working as we'd want it to be today is irrelevant - a lot of terrible things exist in the VICE code. We want to make it better than what is there. And we don't want to add more stuff of which we know it doesnt match how we want it to be.
My point is I don't see what is missing about MC6821_CTRL_C2_STROBE_E. That part seems complete, but you have FIXME messages about it.
as said, i cant tell. perhaps it even works and just was never tested if it does, because none of the currently emulated hardware uses it. you should be able to find that out once your emulation uses it :)
re: scsi bus, to clarify, you want to be able to, in the end, send CDB commands directly to real SCSI devices?
i want the API that connects scsi.c with scsi-hardrive.c to be based around scsi commands. then in theory what you said could be done. and it'd be easy to add whatever other scsi devices. it will make the whole code more logical, more structured, easier to expand and maintain.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Re: MC68B21P
Here is my latest update.
Tell me what you think. I'm using the 6821 code, added a SCSI subsystem, added the docs, and cleaned things up.
SCSI, on a modern system, has an API which tells it the expected and maximum size of the CDB and response. On the LTK and CMD, we don't know this. They depend completely on the drive. So when they ask for a data block, they keep receiving info until the device says there is none left. So I can't really "pass" commands onto SCSI devices unless I completely take them apart. And since I don't know what command they can send (other than READ/WRITE), this is difficult to foresee. So I basically have to translate them, but I don't know what they will send me. I've taken apart the LTK DOS and I know they only use 4 SCSI commands. I'd done the same to the CMD DOS and utilities and found all the commands they use. So right now, it just handle disk read/write and those basic other SCSI commands.
in src/core/Makefile.am there seems to be no TAB before the added lines
remove // comments in src/core/scsi.c. also a few TABs used for indention and trailing spaces at the end of lines (i recommend an editor that displays those...)
the disk image names shouldnt be some arbitrary names by default (we only use default names when we provide the respective files)
a few statements are missing spaces, eg scsi_image_attach(<k_scsi,i,ltk_disk[i]);should be scsi_image_attach(<k_scsi, i, ltk_disk[i]);
there is still that switch statement with magic numbers and then comments
+case0x00:/* TEST UNIT READY */+case0x01:/* REZERO UNIT */+case0x03:/* REQUEST SENSE */+case0x04:/* FORMAT */+case0x07:/* REASSIGN */+case0x08:/* READ */
use defined constant for these, and remove the comments (basically for all SCSI commands in the code)
also, since you have changed/fixed the 6821 core, which is shared with other code - please do some brief testing with magic formel/formel 64 and check if you see obvious problems (i doubt it, but still).
also note to myself: commit the 6821 fixes seperately :)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
ok, some more nitpicking (sorry, didnt see this before)
there are some very long lines, in particular in ltkernal.c. generall try to insert linebreaks:
- before position 80 in block comments
- before position 80 in code, IF the total line is longer than ~100 (so somewhat longish lines are acceptable, but please not much longer. some lines are >130, thats a bit much)
also, please use some common prefix for the SCSI command defines - since they are in a "public" header file, they might clash with some other things.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Ok, i am applying it now. There have been some minor things left which i have fixed while testing:
the vice.texi should also have linebreaks ~ column 80. i fixed that
i am not sure if the very verbose/detailed hardware description belongs into vice.texi. i left it in there for the time being, but you might want to move it elsewhere (the doc should really only contain whats relevant to the crt format)
integer resources use uint8 variable cast to int, thats a bug waiting to happen. i changed those to int variables. (generally try to avoid casting anything, eg the casts in the snapshot saving code are unnecessary and potentially hide bugs - i have removed them)
likewise, integer resources should be saved as such into the snapshot (avoiding the cast)
missing things (please add them):
- a resource/cmdline option to make the disk images read only (yes, it is missing in IDE64 code too)
- the code should not use the romh_banks array as RAM - please use the export_ram0 array for that
- please add io_dump function(s)
and a question: when the ltk cartridge image is attached without attaching a disk image, VICE shows a black screen. does the same happen on the real hardware (when no disk is attached)? this should perhaps get mentioned in the documentation then.
also, why is the "LTKio" resource (1=$DExx, 2=$DFxx=default) ? why not 0 and 1? That seems more logical, unless there is a good reason to have 0 for "not set" (makes no sense to me)
that said, applied in r38814 - thanks!
Last edit: gpz 2020-10-15
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Sorry about the vice.texi file. I assumed a new line was the start of a new paragraph, but I guess it is two new lines.
Would it be good to leave a "txt" file in the source directroy that gives more details, or should it all just be in the source? The LTK is a weird unicorn and there isn't a lot of reliable information out there so I think it is best to have it somewhere.
Perhaps we should have snapshot macros that support all the data types. They seem to only support unsigned, but no signed. Also, a lot of the other modules have casting in those macros.
The LTK won't work in read only mode. The DOS swaps memory to disk and back so it would fail almost immediately. There really isn't a way to isolate it either as the locations it uses for this swapping isn't predicatable.
The LTK has two separate 8KiB RAMs. I didn't want to setup another RAM location; I just thought I would use what was available. Recommendations?
I guess io_dump is new. Will do.
Yes, the LTK will go black if the "drive" doesn't respond; this is documented in the manual. The emulaion of the drive properly responds as "not present" if the image file doesn't exist. This original "feature" is to stop users from trying to reinstall their DOS on a failing drive. I will add in an error message if the first disk isn't available on access and tell the user to attach something. I can also add it to the texi.
The LTK docs always refer to "port 1" and "port 2". It is labeled on the PCB as well. Port 1 being $DE00 and port 2 being $DF00. So I thought it would be best to stick with what the official docs say.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The form removed the asterisk in the c64cart star .c note in the original report.
ok some quick review:
generally the code looks ok to me, i can't comment on any of the LT.Kernal specific things. One thing i am not sure about is - if this is a "SCSI Host Controller", then isnt it possible to use other SCSI devices than HDs with it? If so, the actual HD emulation should be splitted off, so other devices can be added some day eventually. Also, doesnt the cartridge contain a MC68B21P? Where is this emulated? Note that "chip" emulation should go into a seperate file in "cores" and work generically.
There are mostly style issues and a few missing things:
+ ltkernal_io_device.start_address = 0xdd00 + ltk_io * 256;shouldnt this be 0xde00 ?
in c64cart.c you have
+ { CARTRIDGE_NAME_LT_KERNAL, CARTRIDGE_LT_KERNAL, CARTRIDGE_GROUP_UTIL | CARTRIDGE_GROUP_FREEZER},this should not be done, it should be only in one group (i suggest "util"), else they might appear twice in the GUI, which would be very confusing.
in cartridge.h the name of the cartridge:
#define CARTRIDGE_NAME_LT_KERNAL "Lt. Kernal Host Adapter"note that this should be the "official" name, as written on the actual product and/or it's documentation.
at a couple places there are "magic values" used in the code, followed by a comment explaining what it is. i suggest using defines for those.
includes in ltkernal.h should be moved to ltkernal.c
then there are a couple codestyle issues:
there are some // comments, please replace them by regular c-style comments
there are many other cases of missing spaces, missing curly braces, indention style different from what we use, etc. please read coding-guidelines.txt - if something isnt clear, please ask so we can explain it better in that file. For example:
for(i=0;i<512;i++) ltk_scsi_data_buf[i]=0;should be
Things that are missing:
...that should give you something to work on :)
I didn't want to abstract the device down to individual parts. I realize this is an ideal, but it certainly complicates the implementation and I was striving for functional at this point since we have nothing. I can always add this in the future, but I think getting something out there that people can use would be good. I have the full DOS code and have disassembled it so I was able to determine which chip features are used; the basically just use the MC68B21P as parallel IO. No interrupts or other features are being used. That being said, I finally was able to aquire a Lt. Kernal; all my code up to this point was arrived at by code inspection. I plan to examine the circuit to see if there is anything else to gain from it, but they do have some PLDs on there which may prove difficult to reverse engineer. Some has done all this work (mytec) but they refuse to share anything for the emulation project (they are happy reproducing old hardware and selling it).
I also have a CMD HD emulation (SCSI) which is my next task to intergrate into SVN, but again the SCSI is integrated (right now). Yes, I would like to separate them, but I feel it is important to get something into VICE as I'm wasting time patching around things that are changing. I started this work on 3.3, and spent the last 2 weeks moving it over to 3.4. Now, it turns out, my vdrive improvements (native partition support) need a lot of changes since dual drive support was added in the SVN (it was quite a hack from what I can see). That will take some time to fix.
I can certainly make those other changes you mentioned. I wasn't aware the group & freeze thing had something to do with the UI. Adapter/Adaptor - tomato/tomatoe. :) I'll fix it.
I wasn't sure about the CRT thing. The ROM is just a dump. There is no CRT type for it. How do we go about setting one up? Do I just pick an "ID" for it?
Is there a place for extended docs? (d64 images, etc?) It turns out the main source for lt. kernal info on the web is wrong in a lot of cases when it comes to configuration, and the author passed away a while ago. So no one is there to maintain it. I could make my own website, but I would prefer something with more permanence. I noticed that a lot of the cartridges refer to pokefinder.org. Is that an option?
As far as other scsi devices, I've hear "rumors" about tape drives, but I haven't found anything. The device was limited to 20MB and 40MB hard discs. I was going to ask on lemon64 to see if anyone has anything they can contribute.
I totally forgot - we already have a 6821 emulation core, see cores/mc6821core.c - currently it is used by dolphindos3, magicformel and formel64 (so you can look at those as an example). the emulation core isn't complete (no interrupts iirc), but if you need that, it should work..
As for seperating SCSI - if you are planning to add CMD-HD too, i definetly suggest doing this. I am very reluctant to merge things when it is known they have to get reworked like this - because the past tells us that once it is in, it wont get fixed.
About the CRT type - you already picked it - this is what the
+#define CARTRIDGE_LT_KERNAL 72 /* ltkernal.c */is about :) your new crt id is 72. If you look at cartconv.c, you will find a list of all types around the start of the file, just make a new entry at the bottom of the list with the respective type (copypaste it from a cartridge that comes close to what you need and edit it accordingly) and it should work. then you only need to add the two hooks to your cartridge implementation at add the type to crt.c, should be straightforward.
extended docs are out of the scope of VICE - we only document the emulator specific things (pokefinder is very "freezer cartridge" centric, i think lt.kernal is out of scope there too)
as for "getting it into VICE", you should just keep developing in the svn trunk, then there shouldnt be problems with merging. we are not in a hurry :)
If I separate the SCSI into another module, then there will be more additions. The CMD HD requires a number of patches as I'm introducing a new image type. This also requires modifications of the image handler and vdrive as well. The current hack to vdrive to add "drive 1" has to be reworked to support up to 255 partitions on CMD HD images. That will take longer. It ends up being a lot of changes and I will be concerned that you won't want to integrate that many changes at once (my last count is about 30 files). So we have to have a compromise here. I can guarantee you that I will be working on this stuff for a while; I started these modifications 2 years ago. I wrote the REL file handler for vdrive about 15 years ago. The only thing that paused my contributions were having kids (they take a lot of time from you), but they are older now and I have more time.
In the meantime I will work on the CRT and docs and see if I can add in the 6821 or separate the SCSI system.
You can always classify this as experimental/evolving as it won't affect anything unless you turn it on.
well the drive number isnt a partition, partitions need to be handled seperatly - it has nothing to do with scsi emulation either, vdrive is something totally different. "SCSI Emulation" should be only about emulating SCSI devices, vdrive is unrelated to this. for "SCSI emulation" we'd want an API similar to the IEC emulation, ie emulate the bus in a way that will allow to hook up arbitrary SCSI devices to it. You may want to look at how clockport devices are being handled at the moment as an example.
Last edit: gpz 2020-10-03
In 3.4, vdrive parsed the drive number in the file name (or would have, but it wasn't complete), eg. "0:$" or "1:$". From what I can see now, that parsing has to be moved to a higher level as each vdrive instance isn't a drive, but a drive/unit now (eg. 8,0 is different than 8,1). This isn't proper as drives should be treated as a whole, with all their units. So I should be able to do the Duplicate command, or Copy command between units (for the CMD, that is up to 255 of them all possibly being different image types). That doens't work now in the svn, and to make it work would be hard. The proper solution would have been to add a second (or more) image to each vdrive and handle it all in there. So the current version it is kind of a hack and will definately need to be reworked.
Reworking it will take some time as I have to track it all back up the chain of functions.
As far as SCSI goes, it seems like you are proposing that each drive ends up being a separate processing system, like TDE. This is far from optimal. Right now I don't treat the host and drives as separate units; I streamlined the communications to avoid the headaches of shared buses. I know what you are saying, but it is needlessly complicated for what these devices need. Both CMD and Xetec were using some of the first SASI/SCSI drives and they had their issues (not completely standard). I can already tell you their DOS's were designed with specific drives in mind and abstracting this to some generic unit will likely have problems. CMD had to change their boot rom each time a new drive was introduced. Lt. Kernal doesn't work with large drives as they keyed the filing system around the number of heads and sectors per cylinder. These are all special considerations to make things work. As much as I'd love to go and fix their individual DOS's to be more accepting of other drives, I can't. I'm making it work for their latests DOS's which are over 25 years old. Maybe, in the future, I can improve those, but this is the priority now.
As I see it there is a clear lack of large volume storage support in VICE. Sure, you have fsdevice, but it doens't support REL files (maybe a little now). So any one who wants to use that is limited to TDE on that standard drives. Both CMD and LTK solved this, and my updates to vdrive add to that. So I'm trying to get those out there.
I know you're not in a rush to add things, but everytime you release something new, it usually ends up effecting some part of what I'm working on. (eg. vdrive - no one touched it until a few months ago, and it seriously needs some love).
as said, vdrive is unrelated here, and needs different considerations that should be discussed first. (and not here, lets keep the ticket clean from this) When you are planning larger changes to existing code, it should be discussed first how we want to do it, and if what you have in mind doesnt clash with what other people are already working on.
as for SCSI, the implementations of cartridges (or other hardware) should never be only for a specific software, but in theory work for whatever software could be run on a device/cartridge. i know its not needed right now and seems like overengineering first - but it will pay off long term. even if there is no other software right now, and even if the implementation only works with the existing software right now, the seperation should be done and prepared for when someone wants to extend this further. ie the long term goal should be that IF someone will ever implement "TDE" style CMD-HD emulation, the same SCSI emulation code can be used for that. in a similar vein, we also have "ATA" emulation, which IDE64 uses (see cores/ata.c). the resason is a similar one as why the code should use the m6821 core - we want to make sure to use generic shared code whenever possible, and not duplicate any code anywhere. this was a huge problem in the past, and we are just slowly coming to the point where its mostly cleaned up.
Last edit: gpz 2020-10-03
I have begun to look at your MC68B21 code, and there is a FIXME part that I need done, but I don't understand what needs to be fixed other than explaining what "CA2state" is and how it differentiates between the two modes. It looks like it works for the MC6821_CTRL_C2_STROBE_E, but not the MC6821_CTRL_C2_STROBE_C mode and there is no clear difference in the code for them so it seems incomplete. I don't know what your plans are for this.
As far as the SCSI system goes, I have implemented CMD HD TDE, but I didn't need a full SCSI backend system to do it. The same can be said for the LTK. I emulated the hardware so that the software can co-exist with it and behave correctly.
The point I'm tyring to make is VICE is an emulator, not a simulator. And emulator provides functionality at the expense of loss some in timing accuracy, where as simulation is the ultimate desired goal, but we can't make VICE work like a transistor level simulation; we just don't have all the information, and it would never be finished, and therefore useless - not to mention too slow. We have to compromise, and we did all over.
I can generilize the SCSI system to work with both devices for 99.9999% of the use cases, but it isn't worth trying to implement support for all devices. There are far too many. Basic disks are fine, and if we need more we can go from there. The code is easy to add to.
Actually the original definition of "emulator" was that it was a better recreation of the original system than a mere simulator...
VICE does try to be as timing-exact as possible, that's why x64sc exists. Talking about a transistor-level simulation is just a false dilemma in this context, for the purposes of VICE it does not gain you anything that you could not recreate using a higher-level emulation approach - the well-known visual6502 model is in fact less accurate than the CPU emulation in VICE for some illegal opcodes.
There is a big argument why it would be a good idea to implement a real SCSI layer in VICE: You can interface it to real SCSI devices of the host, so a user could for example take the HDD from his real CMD HD, connect it to his PC and access it from VICE.
I can access a real CMDHD connected to the SCSI bus now. The disk access is facilitated by the OS, not me. I just use fopen, fseek, fread, fwrite, fclose and all is handled. I use ftell to report the drives size.
Is it necessary to pass the SCSI commands, un-altered, to the scsi drive?
the plan is to emulate the chip as accurately as needed, because that makes it generic and can be used in all pieces of the emulator that use that chip - and the answer to what is missing and needs to be done is in the datasheet of the chip. i haven't looked at it since i implemented whats there, so i cant really tell what exactly is missing or needs to be done.
and yes, we do indeed for aim for very high accurracy these days, and want to implement things like they really work, not as some blackbox approach. this aim was different back in the days when the project was started.
you don't have to support "all devices" at all. just like the clockport stuff, it can be splitted into a "bus" API, and the actual emulation of a device. when we have that, additional devices can always be added without touching any of the other code. or even made to talk to actual real scsi devices, as ingo suggested.
Last edit: gpz 2020-10-04
(aside about vdrives etc: my current thoughts on multiple images in a vdrive are that
vdrives should grow some over-archingvunit, like TDE drives where adiskunit_context_tcontains 1 or moredrive_ts. This would handle multiple drives, and partitions. But I haven't dared to restructure this. I am still too scared of the LED handling for the GUI.)My changes to vdrive correct a lot of issues. it supports DHD images (CMD HD) with multiple parititions, sub directories, full paths, duplication/copying between partitions. etc.
I only have to make a dew changes for "drive 0/1" support on the CBM dual drives.
Please, this discussion is getting out of hand. Move the vdrive stuff into #vice-dev, it's 100% offtopic here.
Also, the fact something else isnt (yet) working as we'd want it to be today is irrelevant - a lot of terrible things exist in the VICE code. We want to make it better than what is there. And we don't want to add more stuff of which we know it doesnt match how we want it to be.
as said, i cant tell. perhaps it even works and just was never tested if it does, because none of the currently emulated hardware uses it. you should be able to find that out once your emulation uses it :)
i want the API that connects scsi.c with scsi-hardrive.c to be based around scsi commands. then in theory what you said could be done. and it'd be easy to add whatever other scsi devices. it will make the whole code more logical, more structured, easier to expand and maintain.
Re: MC68B21P
Here is my latest update.
Tell me what you think. I'm using the 6821 code, added a SCSI subsystem, added the docs, and cleaned things up.
SCSI, on a modern system, has an API which tells it the expected and maximum size of the CDB and response. On the LTK and CMD, we don't know this. They depend completely on the drive. So when they ask for a data block, they keep receiving info until the device says there is none left. So I can't really "pass" commands onto SCSI devices unless I completely take them apart. And since I don't know what command they can send (other than READ/WRITE), this is difficult to foresee. So I basically have to translate them, but I don't know what they will send me. I've taken apart the LTK DOS and I know they only use 4 SCSI commands. I'd done the same to the CMD DOS and utilities and found all the commands they use. So right now, it just handle disk read/write and those basic other SCSI commands.
Looks a lot better now - almost there
Just a few minor quirks left:
scsi_image_attach(<k_scsi,i,ltk_disk[i]);should bescsi_image_attach(<k_scsi, i, ltk_disk[i]);use defined constant for these, and remove the comments (basically for all SCSI commands in the code)
also, since you have changed/fixed the 6821 core, which is shared with other code - please do some brief testing with magic formel/formel 64 and check if you see obvious problems (i doubt it, but still).
also note to myself: commit the 6821 fixes seperately :)
Okay. This is all of it.
I will email you the supporting files.
ok, some more nitpicking (sorry, didnt see this before)
there are some very long lines, in particular in ltkernal.c. generall try to insert linebreaks:
- before position 80 in block comments
- before position 80 in code, IF the total line is longer than ~100 (so somewhat longish lines are acceptable, but please not much longer. some lines are >130, thats a bit much)
also, please use some common prefix for the SCSI command defines - since they are in a "public" header file, they might clash with some other things.
Okay. Made those changes. I used, primarily the linux SCSI defines, but I understand.
Ok, i am applying it now. There have been some minor things left which i have fixed while testing:
missing things (please add them):
- a resource/cmdline option to make the disk images read only (yes, it is missing in IDE64 code too)
- the code should not use the romh_banks array as RAM - please use the export_ram0 array for that
- please add io_dump function(s)
and a question: when the ltk cartridge image is attached without attaching a disk image, VICE shows a black screen. does the same happen on the real hardware (when no disk is attached)? this should perhaps get mentioned in the documentation then.
also, why is the "LTKio" resource (1=$DExx, 2=$DFxx=default) ? why not 0 and 1? That seems more logical, unless there is a good reason to have 0 for "not set" (makes no sense to me)
that said, applied in r38814 - thanks!
Last edit: gpz 2020-10-15
Sorry about the vice.texi file. I assumed a new line was the start of a new paragraph, but I guess it is two new lines.
Would it be good to leave a "txt" file in the source directroy that gives more details, or should it all just be in the source? The LTK is a weird unicorn and there isn't a lot of reliable information out there so I think it is best to have it somewhere.
Perhaps we should have snapshot macros that support all the data types. They seem to only support unsigned, but no signed. Also, a lot of the other modules have casting in those macros.
The LTK won't work in read only mode. The DOS swaps memory to disk and back so it would fail almost immediately. There really isn't a way to isolate it either as the locations it uses for this swapping isn't predicatable.
The LTK has two separate 8KiB RAMs. I didn't want to setup another RAM location; I just thought I would use what was available. Recommendations?
I guess io_dump is new. Will do.
Yes, the LTK will go black if the "drive" doesn't respond; this is documented in the manual. The emulaion of the drive properly responds as "not present" if the image file doesn't exist. This original "feature" is to stop users from trying to reinstall their DOS on a failing drive. I will add in an error message if the first disk isn't available on access and tell the user to attach something. I can also add it to the texi.
The LTK docs always refer to "port 1" and "port 2". It is labeled on the PCB as well. Port 1 being $DE00 and port 2 being $DF00. So I thought it would be best to stick with what the official docs say.
Okay. I think I've made all the necessary changes.