Re: [Alephmodular-devel] TS Draft: File abstraction

[Br'fin <br...@ma...>]

On Friday, January 24, 2003, at 10:55  AM, Woody Zenfell, III wrote:

> On Friday, January 24, 2003, at 01:39  AM, Br'fin wrote:
>
>> First there was portable_files.h and it was good in concept. But 
>> there  was only a Macintosh implementation. And even so, there was an 
>> evident  bios in the structure towards the Macintosh.
>
> And perhaps rightly so, at least for things like resource forks that 
> we have to deal with but which (obviously) impose a Mac bias.  (I 
> haven't reviewed portable_files.h, so maybe there is more Mac bias 
> than the "some" that needs to be there.)

The funny thing is, portable_files.h wasn't dealing with the resource 
forks. I admit it's probably the case that it was being cast to an 
FSSpec and then opened as a resource fork though. I don't have a 
problem with adding a open_resource_file to CFileDesc.

> So will there be a standard interface for things like "locate files 
> that match my filter" (which  may be used for populating a file 
> browsing dialog) etc.?  Such a thing would also be useful for a set of 
> paths to search for a given file.

I have not yet looked over find_files.h yet. Though it seems to me that 
find_files would be the place to expand upon for this kind of 
functionality.

> So far this is a comment, an observation, not a quibble or 
> recommendation etc., but... note that if you have multiple factories 
> at once, the above construction requires that the program know which 
> factory it should use at all times.

My own bad. I should have specified that CFileDescFactory is a 
singleton. Thus the application should always know which factory to use.

> If directory FileDescs that came from Factory A should always produce 
> FileDescs from Factory A (and never from Factory B), then it would be 
> better to have FileDescs know which factory they came from (either by 
> virtue of the class structure, or by explicitly keeping a reference to 
> the factory) and ask a FileDesc for a sub-FileDesc.  (Or ask a 
> FileDesc for its factory, and ask that for a sub-FileDesc, which maybe 
> should be an option but asking the FileDesc directly is provided as a 
> convenience method.)

There are some issues here I have been waffling on. CFileDesc could 
either be modfiable or constant. I've been leaning towards it being 
constant and that if you need a new path (a parent or a child) then you 
need to create a new CFileDesc.

Now I've been mixed as to whether this should be a function of the 
factory or of the file desc itself. GetParent definitely strikes me as 
a member of CFileDesc, but I've been hazier on children. I don't have 
any great problem with
CFileDesc *newDesc = preferencesDesc->Concatenate("FileName")

>> The factory is also the most reasonable place to provide access to  
>> CFileDescs for specific platform specific places. Ala preferences  
>> directory and application directory
>>
>> CFileDesc *foo =  FILE_DESC_FACTORY- 
>> >GetSpecificDir(CFileDescFactory::ApplicationsDirectory);
>> // It is an error to delete this foo
>>
>> Specific directories so far are
>> 	ApplicationDirectory - The directory the application has been run 
>> from
>> 			(Current dir if command line, directory of application executable  
>> otherwise)
>> 	PreferencesDirectory
>
> Note that some platforms may want to install and find data files 
> somewhere other than the application directory.  In general I think 
> having more "specific directories", even if they map to the same real 
> directory in practice, is going to be a Good Thing as it gives 
> different platforms the opportunity to put things where they want at a 
> finer granularity.

What other directories should I be including in the list? Looking at 
FileHandler.h in A1 also turns up the following:
	Save game directory
	Recording directory
	Resources directory - Directory for additional platform generic files 
required by A1
		ie, at one time this was the Pfhortran Language File in A1
		Though I suppose it could also cover SDL's required graphic files for 
its ui
		
> Maybe on the whole I'm not understanding the role this plays in a 
> larger system, but I have questions like:

>  what about searching multiple places? what _about_ support for file 
> browsing?  what about retrieving a directory's contents?
Some of these seem like uses for find_files.h.
Support for file browsing I didn't think of probably because I'm used 
to the various Mac file browsers. (StdFileDialog, NavServices) And so 
the need to support file browsing is kinda foreign. I can understand 
the troubles we're having with preferences implementation and SDL gui 
vs Mac GUI on that, but I'm not sure what to make of 'but our platform 
doesn't have a file browsing call'

That being said, it does seem like there's some cause to split up 
FileDesc's current functionality so that we have a DirectoryDesc and a 
FileDesc. The other option would be to add details like IsFile, IsDir, 
open_directory or what not on CFileDesc. Having separate classes does 
seem appropriate for clarity. Though when you do get a deeper entity 
from the directory, how do you know if it's a file or a directory? Hsm, 
one class for both has its merits...

>   etc.  I like centralizing (in a Factory) all the explicit paths etc. 
> - and since every FileDesc will come from one of these root FileDescs 
> (right?  well except for FileDescs that come from the OS-native 
> representation I guess, but the Factory could be used for those too),
I think it gets harder to have the factory create the FileDescs from 
the os-native representation. You can't put calls like 
CreateDesc(MacSpecificType) on the base factory declaration :)

I don't see a problem with CFileDesc *MacSaveDialog(), which needs to 
know the platform stuff anyways going new MacCFileDesc(MacSpecificType)


> the Factory gets to determine the C++ type of the FileDesc we end up 
> using - but I am not sure what other benefits it offers (e.g., why 
> _is_ it better to ask the Factory for a FileDesc every time we want a 
> file within a directory, instead of asking the directory's FileDesc?).

That is a very good question. And you're right, having to go through 
the factory for that does indeed seem awkward.

>> CFileRef represents an open file
>>
>> Operations include
>> 	get_fpos
>> 	set_fpos
>> 	set_eof
>> 	get_file_length
>> 	read_file
>> 	write_file
>> 	close_file
>
> And a CFileRef is, I hope, a subclass of some more-general "read and 
> write chunks of data" scheme that's also the basis for the "Packing" 
> (serialization) stuff, rather like SDL_RWops, that provides the same 
> interface for working with chunks of data in memory as it does for 
> working with chunks of data in files.  (SDL_RWops provides seek, tell, 
> get length, close, and various read and write operations, both "raw" 
> and packing- (and byte-swapping-) oriented.)
>
> A CFileRef would be allowed to return its CFileSpec on request of 
> course.  Memory chunks would return a special value (maybe NULL?) 
> instead to indicate they're not file-backed.  I'm not sure whether 
> this involves a dynamic_cast<> and a subclass-specific query, or a 
> "top-level" query to the CDataRef itself.  (There may wish to be a 
> corresponding method to get the base address of a CBufferRef also, for 
> example.)

Not really, portable_files itself was working at a low level. So it's 
not very intelligent. It's really not doing anything more than wrapping 
the stdio calls or platform specific elements. 'intelligence' comes 
from elements above that use it. Most of which tend to treat the file 
as a wad file, a resource file, or an ordered file with a header (ala 
shapes and sounds)

Now I agree that there should be a further abstraction over and above 
the wad file to specifically detail what needs to be saved and perhaps 
how.

> What about "safe saves"?  Should there be sort of built-in support for 
> that sort of thing at a fairly low level?

Now that seems pretty darn reasonable to try and work in. And would be 
appropriate for most things we're trying to save. (Not sure about 
replays, but I agree whole heartedly with respect to save games and 
preferences)

> Man all this reinventing-the-wheel stuff really does make me wish we 
> were sitting on top of libFoundation or something ;)  I guess the 
> important question to ask is "what flexibility/better 
> platform-dependent conformance/etc. are we buying with this extra 
> layer of wrapping?"  I mean, why NOT just use some kind of standard 
> mechanism like iostreams or stdio?  at least for most of the 
> interface?  (you could supplement those mechanisms with some kind of 
> abstraction only where needed.)  I mean, they're there, they're 
> familiar, they're tested, and they're not going to be going away any 
> time soon.  (I recognize that there are benefits to wrapping.  I would 
> just like to see them made explicit as part of the 
> reasoning/justification.)

A Very reasonable question. And I doubt it would be that hard to 
recreate CFileRef as 'a c-style file handle with extra goodies' that 
could self-cast itself so that you could go fprintf(CFileRef, "my 
string");

The initial reasons probably have to do with Marathon handling paths as 
FSSpecs. portable_files then handles the minimums Bungie required for 
accessing files. If we can encapsulate the FSSpec with CFileDesc so 
that the result of an open is always the result of stdio fopen, then we 
could very well buy ourselves a good deal.

> Anyway please don't take any of the above really as 'argument', I'm 
> glad to see a specific suggestion as a starting point and hope that a 
> discussion (including "devil's advocate" questions to expose more 
> detail and strengthen the underlying reasoning) will bring to light 
> any appropriate alterations.  Like I said, I don't have a good picture 
> of where CFileSpec fits into the bigger structure, so a lot of the 
> things I asked about could be irrelevant in the limited context of 
> CFileSpec itself - but if they're not handled there, then where?
>
And I thank you a great deal for the feedback. The next draft will be 
much improved for it. And if you didn't have a good idea as to where 
CFileSpec fit into the scheme of things, then its my bad for not 
writing it out clearer in the first place :)

-Jeremy Parsons