Windows 2.0 freeze
Brought to you by:
bartoldeman
Menu
▾
▴
#507 Windows 2.0 freeze
With the fix for bug ticket #506,
Windows 2.0 goes further and loads correctly.
The Executive operates normally,
but when a Windows application is run,
DOSEMU print the error msg:
"ERRROR: SMALLOC: bad pointer passed to smget_area_size()",
and Windows 2.0 seems to hang (with hourglass showed).
The msg error appears when win.com calls EMS AH=51h
reallocate pages. I guess then there's probably
a bug remaining in reallocate_pages() in emm.c ...
Here is the boot.log with only EMS
debug trace enabled.
...seems like Windows trying to reallocate the OS handle,
which is mapped into conventional memory,
thus producing the error message.
I reproduced the err msg with a small test program.
The OS handle is probably not treated properly
in reallocate_pages() in emm.c
(and for others funcs for that matter).
What's the purpose of that special handle anyway?
But I don't know if that would solve Windows
not responding if reallocate was changed.
Are the changes in devel affect this anyhow?
Please add another log in that case.
Also, I bet the log is incomplete.
Please do not truncate it.
Yes, I am pretty sure fixing ems will get
Windows to back work, because it used to work.
The purpose of the special handle is
to map the conventional memory. It is
described in the EMS specs.
I think if windows is trying to reallocate
it, then we have a bug, because I think
we shouldn't return that handle at first
place, and then windows will have no chance
touching it.
Here is the complete boot.log (with EMS trace).
According to the EMS spec,
the OS handle (0) is preallocated and
backfills the top 384K conventional memory
on initialization. The spec says the OS
may use the "reallocate pages"call
to allocate more pages.
The supplementary pages will be taken from
expanded memory then? I guess the "logical map"
for handle 0 is:
top conventional mem 384K: logical pages 0-23
expanded memory allocated: pages 24-...
Is that correct?
We should check with EMM386.exe to see how it behaves
for this special handle.
I started to experiment with EMM386.exe
and the OS handle.
But I need to leave town for 2 days,
and won't probably have Internet access.
I'll get back to it on Monday.
I think it doesn't matter where the
first 24 pages are allocated. They can
as well be allocated in an expanded
memory too.
I think your question comes from the
fact that dosemu is trying to keep the
memory block contiguous, which would
be impossible when you enlarge the os
handle. I actually think this is a bug.
Unless I am missing something, the space
for logical pages is not needed to be
virtually contiguous. You can do a per-page
allocations. Without this, not only
enlarging the os handle is impossible,
but I guess also many other reallocs
will step on each other's feets and fail
for no reason.
Hi Stas,
As for the first 24 allocated pages, they could be allocated in
expanded memory indeed but that would waste 384K of physical memory.
Also, in order not to corrupt DOS management of conventional memory
(or TSR already present), I guess the EMM manager on initialization
must either:
1) use real conventional memory;
2) or, make a copy of the 384k block into expanded memory before mapping.
I believe your correct for the logical space. The backing store
for the EMS logical space doesn't have to be virtually
contiguous (as is the case in dosemu), since EMS apps
only refer to individual logical page numbers,
without knowledge of the implementation mechanism.
Right now, dosemu can't enlarge the OS handle
because of the contiguity constraint. I'm not sure of
the best solution to this problem. At the top of my mind,
I can think quickly of two possibilities:
1) Treat the OS handle in a special way, but for other handles
nothing changes. For the special handle, make object ptr points to
supplementary allocated pages (>24) with the understanding that
the first logical pages (up to 24) must point into LOWMEM.
2) Remove the contiguity constraint altogether for all handles. Maybe have
a global EMS memory pool for all handles and allocate logical pages on
a per-page basis from it. That would imply though more management overhead.
Option 1 is probably less drastic but option 2 more uniform. OTOH, I don't see why you say multiple reallocs could interfere with each other. Doesn't realloc move the block around if necessary?
I did experiment with MS-DOS EMM386, and it does work as I thought.
The first 24 logical page (corresponding to backfilled conventional memory)
can be remapped elsewhere in conv mem (logical pages 0-23 coincide
with physical pages 4-27 initially) or in the page frame.
Extending the OS handle, allow to map more pages into the physical space
(in the 384K area or page frame).
...third possibility:
For the OS handle, allocate initially a 384K block
for object and copy LOWMEM into it. Would be even better
than option 1.
Yes, realloc can move, but that incurs
the fragmentation. When you are fragmented
enough, you may start having failures.
dosemu gets away with that by not enforcing
any limits on pool, which is not necessarily
a right way.
Also, when you realloc, you also need to
unmap and then map again, so I don't see how
the per-page allocations can have more
management overhead. Clearly it is a smaller
and faster code.
I think you have refuted your "third possibility"
already by correctly mentioning the waste of
384K.
Yes, of course the code can be optimized
to not require unmap/map cycles even with
realloc (provided that realloc is trying
to re-use the same physical store), but
for what, if the entire realloc can be
optimized out...
Ok, I didn't think realloc fragmentation was such a problem
since the virtual address space is so large and EMS memory
typically just uses a few megabytes at maximum.
I agree using a single memory pool with per-page allocation
would be cleaner and more in line with the EMS spec model.
If realloc/remap could be eliminated, yes, there would be
less dynamic overhead. But I think it would require a bit
more code in emm.c though.
Then, remains the question of the single memory
pool implementation. A few possibilities:
1) One global array allocation on init to the EMS mem maximum.
Simple, but wasteful of memory if EMS isn't used much.
2) A dynamic array. Less wasteful, but then we get back the
realloc fragmentation issue (though probably less severe
than multiple pools like the current implementation).
3) A dynamic pool implemented as a linked-list of memory
blocks with a reasonably coarse granularity for containing,
let's say for example, about a hundred ems logical pages per block.
Less wasteful of memory like 2 without the realloc overhead
like 1. The downside is that the code logic would become a bit more
complex.
Any suggestion?
Of course currently the fragmentation is
not a big deal at all. But if it can be
avoided alltogether with realloc and remap...
Why not (at least for the first time) to
not implement any pool, and just use a per-page
allocations? In the handle_record struct you'll
just transform "void object" into
"void logical[MAX_LOGICAL_PAGES_PER_HANDLE]",
or something along the lines. Looks like a
trivial change mostly, or am I missing something?
Ah yes, per-page allocation would work also.
It sure is the least invasive change.
MAX_LOGICAL_PAGES_PER_HANDLE can be computed with max EMS memory
and EMM_PAGE_SIZE. The logical array would require to be
dynamically allocated though, because ems_size is not
a compile-time constant.
The standard says:
BX = num_pages_alloc_to_emm_handle
Contains the number of logical pages allocated to the
specified EMM handle. This number never exceeds 2048
because the memory manager allows a maximum of 2048 pages
(32M bytes) of expanded memory.
Whether or not we should trust that part,
is however unclear... Surely the 65536 would
be a definite limit, but that's too much for
the static allocations.
If you don't want to trust in a 2048 limit,
then I will suggest to use glib... It was not
used in dosemu before AFAIK, but someone have
to start... :) I already had enough implementing
the hand-made containers of different kind for
dosemu.
Why shouldn't we trust the limit? The 32 MB max value
seems to be a hard limit of the standard. It's clearly
diagrammed at the beginning of the spec.
2048 doesn't appear to me too much costly
(16K on x64 and 8K on x32) per handle.
As for using glib, in this specific case, you mean
to make use of a dynamical array GPtrArray for example?
Does dosemu already honour the limit of 32Mb total?
If not, I would probably suggest against
enforcing it, unless there are a big reason to.
Is this limit enforced in emm386 and qemm?
Of course if dosemu already has this limit,
then no problems to keep it that way.
Yes, something like GPtrArray, or in fact
maybe GArray, because it is not unexpected
if someone would also like to save other
things, like, for instance, a number of the
mapped physical page, for a faster lookups.
Actually no, we can't do it statically,
no matter what. Your calculations are wrong
in that you forgot to multiple per MAX_HANDLES.
We need the size of the whole handle_info array.
I checked, and the 32 MB is not currently enforced in dosemu.
However, emm386 doesn't support mem > 32 MB.
I don't have qemm installed, so I can't check it.
I'd say then that dosemu is just not standard-conformant
at the moment.
OK, IMHO the easiest way is to just
realloc() the logicals array itself.
Or use glib. Whatever you prefer. :)
Ok, thanks for info.
In fact, enforcing 2048 logicals doesn't
limit us to 32Mb total, but to 32Mb per
page, which would be fine... if it would
help, but it doesn't.
per page = per handle, sorry
I started to change the code. Must of it is quite trivial.
However, I just realized that it will complicate quite a bit
the implementation for the Move/Exchange region EMS call,
if the region crosses multiple logical pages
(because of the lost of contiguity).
It will require some type of scatter/gather functionality...
Argh. Indeed.
Or you can create a temporary, contiguous
mapping while copying.
But I think s/g is not very difficult here,
so likely the temporary mapping is too much
just for that.