#459 ZXMMC emulation does not work.

[Paul Osmialowski]

ZXMMC emulation does not work. I tried to use HDF image prepared with raw2hdf which is recognized by DivIDE emulation in the +3e mode (e.g. CAT TAB shows some geometry for it and I can use it as rootfs for Fuzix). The same image is not visible at all when used with ZXMMC emulation. Fuse compiled from git master commit 8deafe0e.

[Sergio Baldoví]

I don't see any problem with ZXMMC emulation. Please, check that you are using the proper +3e ROMs for ZXMMC and options similar to:

fuse -m plus3e --zxmmc --zxmmc-file image.hdf --rom-plus3e-0 mmcen3e0.rom --rom-plus3e-1 mmcen3e1.rom --rom-plus3e-2 mmcen3e2.rom --rom-plus3e-3 mmcen3e3.rom

In case it fails, maybe it's something specific to your hdf file? Do you mind sharing (privately) with me to reproduce this error?

[Paul Osmialowski]

I do not have ROM files from your list. The only ROM files I've got are those shipped along with Fuse package in Gentoo Linux:

-rw-r--r-- 1 root root 16384 Dec 17  2018 128-0.rom
-rw-r--r-- 1 root root 16384 Dec 17  2018 128-1.rom
-rw-r--r-- 1 root root 16384 Dec 17  2018 48.rom
-rw-r--r-- 1 root root  8192 Dec 17  2018 disciple.rom
-rw-r--r-- 1 root root 16384 Dec 17  2018 plus2-0.rom
-rw-r--r-- 1 root root 16384 Dec 17  2018 plus2-1.rom
-rw-r--r-- 1 root root 16384 Dec 17  2018 plus3-0.rom
-rw-r--r-- 1 root root 16384 Dec 17  2018 plus3-1.rom
-rw-r--r-- 1 root root 16384 Dec 17  2018 plus3-2.rom
-rw-r--r-- 1 root root 16384 Dec 17  2018 plus3-3.rom
-rw-r--r-- 1 root root 16384 Dec 17  2018 plus3e-0.rom
-rw-r--r-- 1 root root 16384 Dec 17  2018 plus3e-1.rom
-rw-r--r-- 1 root root 16384 Dec 17  2018 plus3e-2.rom
-rw-r--r-- 1 root root 16384 Dec 17  2018 plus3e-3.rom
-rw-r--r-- 1 root root  8192 Dec 17  2018 plusd.rom
-rw-r--r-- 1 root root 16384 Dec 17  2018 se-0.rom
-rw-r--r-- 1 root root 16384 Dec 17  2018 se-1.rom
-rw-r--r-- 1 root root  8192 Dec 17  2018 speccyboot-1.4.rom
-rw-r--r-- 1 root root 16384 Dec 17  2018 tc2048.rom
-rw-r--r-- 1 root root 16384 Dec 17  2018 tc2068-0.rom
-rw-r--r-- 1 root root  8192 Dec 17  2018 tc2068-1.rom

Also, the images I'm using are substantially big (4GB, 8GB), it can be hard to share. Yet it is easy to prepare one as the content does not really matter. Take one SD card with regular DOS partition table (as Fuzix understands only DOS paritition tables), grab entire card's content (partition table and partitions) using dd command into a raw image file and use raw2hdf from fuse-utils to create HDF image out of it.

[Sergio Baldoví]

I do not have ROM files from your list. The only ROM files I've got are those shipped along with Fuse package in Gentoo Linux:

That plus3e roms distributed by default are for the Simple 8-bit IDE interface. The plus3e author make specific ROMs for different interfaces. You should grab the mmc files:
http://www.worldofspectrum.org/zxplus3e/p3eroms.html

You could change the Fuse options to load that new roms, or rename to the same filename and replace them.

Also, the images I'm using are substantially big (4GB, 8GB), it can be hard to share.

The 4 Gb image should work good. The 8 Gb image could fail. Could you at least specify the exact size of the hdf files? It should be trivial to rebuild that empty images.

[Paul Osmialowski]

Using ROM images downloaded from that page indeed helped a bit and now CAT TAB shows geometry of some empty ~512MB image I've prepared. Some problems unfortunately emerged still. I tried to make use of ~256MB image of real SD card, unfortunately, Fuse refused to start, stating the size of the card is unsupported (while DivIDE emulation had no problems working with the very same image). You can replicate the problem with empty image too:

dd if=/dev/zero of=256MBzeros.rawimg bs=252968960 count=1
raw2hdf 256MBzeros.rawimg 256MBzeros.hdf

Then I tried to make use of ~3GB real SD card (3904897024 bytes raw image, which resulted in 3904897558 bytes hdf image) of Workbench +3e (with all of its substantial number of partitions its installer have created). Unfortunately, as I tried to use it, +3DOS has gone haywire and the +3e startup screen started to look as in attached image. As a result, no drives (either floppies or MMC) were accessible.

[Sergio Baldoví]

Thank you for the info! AFAIK you have these real cards:

~256 Mb SDSC card
252968960 bytes = 494080 blocks (not multiple of 2^10)

~3 Gb SDHC card
3904897024 bytes = 7626752 blocks (multiple of 2^10)

Some problems unfortunately emerged still. I tried to make use of ~256MB image of real SD card, unfortunately, Fuse refused to start, stating the size of the card is unsupported

MMC emulation use a virtual SDHC card between 512 Kb and 32 Gb. That's not realistic as SDSC card sizes ranges between 256 Kb and 2 Gb and SDHC card sizes ranges between 2 Gb and 32 Gb.

The number of blocks of SDHC cards should be multiple of 2^10 (512 Kb) and Fuse/libspectrum are strictly validating this size. Hence the problem with your 256 Mb card image.

Fuse should autodetect the card type and adjust the MMC emulation. That's doable but requires coding/testing. As a workaround, we could relax this validation and allow images not multiple of 2^10. With your 256 Mb card image, the last 512 blocks (256 Kb) would not be accesible.

Then I tried to make use of ~3GB real SD card (3904897024 bytes raw image, which resulted in 3904897558 bytes hdf image) of Workbench +3e (with all of its substantial number of partitions its installer have created). Unfortunately, as I tried to use it, +3DOS has gone haywire and the +3e startup screen started to look as in attached image. As a result, no drives (either floppies or MMC) were accessible.

Have you tested the HDF image available in Workbench +3e site? We would need concise steps to reproduce this error to know what's going on.

[Paul Osmialowski]

The first issue has an easy workaroud, it's sufficient to extend size of the raw image and fill it with zeros, e.g.:

dd if=/dev/zero of=256MB.rawimg bs=15466496 count=1 oflag=append conv=notrunc
raw2hdf 256MB.rawimg 256MB.hdf

The other issue, yes, I did try to use Workbench +3e preinstalled .hdf image and it seems to work properly. Also I managed to install Workbench +3e on empty ~2GB image formatted in Fuse.
The only problems that remain are:
1. SD card image copied from the real thing can't be read properly (result seen on the screenshot attached previously)
2. The Fuzix bootloader can't find Fuzix partitions on SD card image grabbed from the real thing (where the same bootloader can find them; also they can be found with DivIDE emulation). The author states it's a bug in Fuse.

[Sergio Baldoví]

The first issue has an easy workaroud, it's sufficient to extend size of the raw image and fill it with zeros

That's even better.

The other issue, yes, I did try to use Workbench +3e preinstalled .hdf image and it seems to work properly. Also I managed to install Workbench +3e on empty ~2GB image formatted in Fuse.

My gut tells me that if Workbench +3e works with a formatted image, the MMC commands work good and the problem could be with the data contained in the card or how is organised in sectors.

How did you partitioned the card? Which partion IDs do you used? You mentioned regular DOS partition table but +3e requires IDEDOS. It would be good to compare the partition table of the physical card and the empty blank image formatted in Fuse (high level) or the blocks itself (low level).

The only problems that remain are:
1. SD card image copied from the real thing can't be read properly (result seen on the screenshot attached previously)

Could you share an image via dropbox/onedrive/google drive? I could debug the MMC commands if necessary.

2 . The Fuzix bootloader can't find Fuzix partitions on SD card image grabbed from the real thing (where the same bootloader can find them; also they can be found with DivIDE emulation). The author states it's a bug in Fuse.

I know nothing about Fuzix. If you cand share an image and provide detailed steps to reproduce the problem I could have a look.

Now you mention it, the partitions you need with a 48k machine + DivIDE are quite different than a +3e machine + ZXMMC.

[Paul Osmialowski]

How did you partitioned the card? Which partion IDs do you used? You mentioned regular DOS partition table but +3e requires IDEDOS.

Hey, this is the misconception that also made me stuck for a while. I am using extended IDEDOS scheme, as described on those two pages: http://www.worldofspectrum.org/zxplus3e/sharingdisks.html and http://zxvgs.yarek.com/en-idedos.html

What is not stated explicitely enough, is that we're dealing with two levels of partitioning here. Top level is the regular DOS partition table starting at sector 0, in which the first partition (of any type, as for +3DOS it is not relevant) is the placeholder for the lower level (ofIDEDOS) partitions with PLUSIDEDOS partition table. What is relevant is where placeholder for IDEDOS partitions starts. If +3e firmware finds the disk image starts with the DOS partition table, it looks for IDEDOS partitions at sector 128 (Head 1 in the C/H/S scheme), otherwise it expects IDEDOS partitions to start at the sector 0.
In order to force Linux's fdisk to place the IDEDOS placeholder partition at sector 128, it must be started in the DOS compatibility mode, e.g.: fdisk -c=dos /dev/sdb.

If you cand share an image and provide detailed steps to reproduce the problem I could have a look.

I'd cover next message with that, later today.

A new issue I've found is that mass storage driver for CP/M Plus by otivax was supposed to support ZXMMC interface since its 2017's version 1.4 (a.k.a driver v0.7, the versioning of this thing is messy and distribution model is hopeless). I tried the only two versions I could possibly find (1.4 and 1.5) and all they both say is Interface: Not found.

[Paul Osmialowski]

2 . The Fuzix bootloader can't find Fuzix partitions on SD card image grabbed from the real thing (where the same bootloader can find them; also they can be found with DivIDE emulation). The author states it's a bug in Fuse.

Installation procedure is like this:
1. Download boot disk: http://fuzix.org/downloads/0.3/zx+3-boot-0.3.dsk.gz
2. Unpack boot disk: gunzip zx+3-boot-0.3.dsk.gz, should leave zx+3-boot-0.3.dsk file.
3. Download 32MB filesystem: http://fuzix.org/downloads/0.3/rootfs-z80-32.gz
4. Unpack filesystem: gunzip rootfs-z80-32.gz, should leave rootfs-z80-32 file.
5. Create empty raw image (256MB should be sufficient):

dd if=/dev/zero of=256MB.rawimg bs=268435456 count=1

1. Create DOS partition table and partitions:

fdisk -c=dos 256MB.rawimg

Example fdisk session:

Welcome to fdisk (util-linux 2.33.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table.
DOS-compatible mode is deprecated.

Created a new DOS disklabel with disk identifier 0xfbeb3b95.

Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p): p
Partition number (1-4, default 1): 1
First sector (63-524287, default 63): 128
Last sector, +/-sectors or +/-size{K,M,G,T,P} (128-524287, default 524287): +419967

Created a new partition 1 of type 'Linux' and of size 205.1 MiB.

Command (m for help): t
Selected partition 1
Hex code (type L to list all codes): 19
Changed type of partition 'Linux' to 'unknown'.

Command (m for help): p
Disk 256MB.rawimg: 256 MiB, 268435456 bytes, 524288 sectors
Geometry: 255 heads, 63 sectors/track, 32 cylinders
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xfbeb3b95

Device        Boot Start    End Sectors   Size Id Type
256MB.rawimg1        128 420095  419968 205.1M 19 unknown

Command (m for help): n
Partition type
   p   primary (1 primary, 0 extended, 3 free)
   e   extended (container for logical partitions)
Select (default p): p
Partition number (2-4, default 2): 2
First sector (63-524287, default 63): 420096
Last sector, +/-sectors or +/-size{K,M,G,T,P} (420096-524287, default 524287): +65535

Created a new partition 2 of type 'Linux' and of size 32 MiB.

Command (m for help): t
Partition number (1,2, default 2): 2
Hex code (type L to list all codes): 7e

Changed type of partition 'Linux' to 'unknown'.

Command (m for help): p
Disk 256MB.rawimg: 256 MiB, 268435456 bytes, 524288 sectors
Geometry: 255 heads, 63 sectors/track, 32 cylinders
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xfbeb3b95

Device        Boot  Start    End Sectors   Size Id Type
256MB.rawimg1         128 420095  419968 205.1M 19 unknown
256MB.rawimg2      420096 485631   65536    32M 7e unknown

Command (m for help): n
Partition type
   p   primary (2 primary, 0 extended, 2 free)
   e   extended (container for logical partitions)
Select (default p): p
Partition number (3,4, default 3): 3
First sector (63-524287, default 63): 485632
Last sector, +/-sectors or +/-size{K,M,G,T,P} (485632-524287, default 524287): +8191

Created a new partition 3 of type 'Linux' and of size 4 MiB.

Command (m for help): t
Partition number (1-3, default 3): 3
Hex code (type L to list all codes): 7f

Changed type of partition 'Linux' to 'unknown'.

Command (m for help): p
Disk 256MB.rawimg: 256 MiB, 268435456 bytes, 524288 sectors
Geometry: 255 heads, 63 sectors/track, 32 cylinders
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xfbeb3b95

Device        Boot  Start    End Sectors   Size Id Type
256MB.rawimg1         128 420095  419968 205.1M 19 unknown
256MB.rawimg2      420096 485631   65536    32M 7e unknown
256MB.rawimg3      485632 493823    8192     4M 7f unknown

Command (m for help): w
The partition table has been altered.
Syncing disks.

What matters here is that there are two partitions, one of type 7e and size of 65536 (1 + 65535) sectors for the filesystem and the other of type 7f and size of 8192 (1 + 8191) sectors for a swap. I've created one more partition starting at sector 128 and unused type 19 that could be used as placeholder for IDEDOS partitions (just remember to limit the number of cylinders used when formatting from +3DOS).
Remember to write the new partition table before leaving fdisk.
7. Copy filesystem image to the 7e-typed partition:

dd if=rootfs-z80-32 of=256MB.rawimg bs=512 seek=420096 conv=notrunc

Note that this operation should NOT change the size of the 256MB.rawimg file.
8. Convert raw image to an HDF image:

raw2hdf 256MB.rawimg 256MB.hdf

1. Run fuse, e.g.:

fuse -m plus3e --zxmmc --zxmmc-file 256MB.hdf --rom-plus3e-0 $HOME/fuse/roms/mmcen3e0.rom --rom-plus3e-1 $HOME/fuse/roms/mmcen3e1.rom --rom-plus3e-2 $HOME/fuse/roms/mmcen3e2.rom --rom-
plus3e-3 $HOME/fuse/roms/mmcen3e3.rom zx+3-boot-0.3.dsk

1. Make it boot from A:.
2. You'll see that no SD cards were found by the bootloader.
3. Now try the same with DivIDE:

fuse -m plus3e --divide --divide-masterfile 256MB.hdf zx+3-boot-0.3.dsk

1. Now, with booted from A: you can see that the bootloader has found three IDEpartitions: hda1, hda2 and hda3. At the bootdev: prompt type hda2.
2. When asked, enter today's date and time, and type root at the login: prompt.
3. To turn Fuzix safely off, type shutdown command and wait until it says Halted., otherwise it will go through filesystem repair at subsequent boot time.
   On the real +3e with the ZXMMC interface, those partitions should be found as SD cards rather than IDE drives, yet still partitions should have names with hda prefix, namely, one should observe following output:

SD drive 0: hda: hda1 hda2 hda3 (swap)
SD drive 1: no card found
bootdev:

[Sergio Baldoví]

Thank you for the detailed explanation. I think the image creation and the partition layout are correct.

Booting a +3e machine with ZXMMC, block 0 (MBR) and block 128 (IDEDOS) are read and if there are IDEDOS partitions, blocks 129 and 130 are also read. I couldn't reproduce the error of the screnshot (+3DOS has gone haywire).

For debugging Fuzix, I've simplified the steps. Fuzix documentation states that supports +3 machines, so there is not need to mess with +3e machines:
1. Download boot disk: http://fuzix.org/downloads/0.3/zx+3-boot-0.3.dsk.gz
2. Download IDE image: http://fuzix.org/downloads/0.3/rc2014-0.3.ide.gz
3. Strip 1024 bytes from ide image:
dd if=0.3.ide of=0.3.raw ibs=1024 skip=1
4. Convert from raw image to HDF:
raw2hdf 0.3.raw 0.3.hdf
5. Start +3 machine:
fuse -m plus3 --zxmmc --zxmmc-file 0.3.hdf --plus3disk zx+3-boot.dsk

I see there is a problem initializing the SD card. CMD0 (GO_IDLE_STATE) command is issued but CMD 8 (SEND_IF_COND) never cames.

After the CMD0 command Fuzix does 21 reads. Fuse returns the R1 response and the latter 20 reads are floating bus.

The problem seems to be here:
https://github.com/EtchedPixels/FUZIX/blob/337485699af7f84a63bc73e6e3841c22b3ab47e0/Kernel/dev/devsd.c#L143
Fuzix is discarding the response of CMD0 - GO_IDLE command.

Previously this line was only active on CMD12 (STOP_TRANSMISSION) command, but was enabled by this refactoring:
https://github.com/EtchedPixels/FUZIX/commit/08a8d4a62165da1feca8529df033ca42978b54c8#diff-404eeab3bbef36bfc73d017fd814b76aL139

So it seems a bug in Fuzix. Although current code might work with real cards, IMO it is not safe to discard the first response.

[Paul Osmialowski]

Booting a +3e machine with ZXMMC, block 0 (MBR) and block 128 (IDEDOS) are read and if there are IDEDOS partitions, blocks 129 and 130 are also read. I couldn't reproduce the error of the screnshot (+3DOS has gone haywire).

Ok, mystery solved. I've written a simple program (attached) that prints IDEDOS partition table. For .hdf images it just needs to have 534 bytes of offset provided. The truth revealed itself: faulty image was not a card image, but the first partition on it (IDEDOS placeholder) that I took from one SD card to copy it to another. After some time (a week or so) I've forgotten about it and have made HDF image out of it. In effect, all IDEDOS partitions were pointed 128 sectors ahead.

I've introduced the change you've proposed on my local clone of the Fuzix repo and rebuilt the whole thing. It works in Fuse +3e with ZXMMC emulation now!

[Paul Osmialowski]

There's no bug then. This one can be closed now.

[Sergio Baldoví]

Ok, mystery solved. I've written a simple program (attached) that prints IDEDOS partition table.

Great. BTW, there is other useful tool for IDEDOS images:
https://github.com/ZXSpectrumMirrors/zx-3e/blob/master/3e.c

[Paul Osmialowski]

Just for a reference, the response I have from Fuzix authors:

No its a bug in the emulator. The first byte is floating bus so has to be ignored with most hardware adpaters. There's a reason it always works with real cards.

[Sergio Baldoví]

Thanks. Outstanding issues are:

• NCR timings, tracked by [bugs:#462].
• SDSC card size, tracked by [bugs:#463].