RE: [Modus-devs] advice on file-based persistence

[Nathan D. <na...@ch...>]

I am trying to avoid having to read files to do a get or a getAll, as
reading large files and parsing them would be very expensive relative to
listing out directory contents.

I'm also now thinking that if I am going to store them in one big directory
anyway, I might just store them named like:

objectID.objectType

I could then just use the filter attribute of CFDIRECTORY when getting all
of a certain type.

Most contenObjects will only be a few kb stored as flat files, and I figure
nobody will use the file storage for many many thousands of objects, so it
should work.

 - n

> -----Original Message-----
> From: mod...@li...
> [mailto:mod...@li...]On Behalf Of Brad Pauly
> Sent: Friday, September 13, 2002 3:57 PM
> To: modus devs
> Subject: RE: [Modus-devs] advice on file-based persistence
>
>
> I had another idea about storing objects. Using both the idea of keeping
> object types in separate directories and distributing the files into
> subdirectories. Then keeping a list of the directories in a file in the
> object type directory. So a path would look something like:
>
> <objectStoreRoot>/<objectType>/1/4/2/3/45740C9C-CCFF-20DE-D6C92C971A2A440B
> <objectStoreRoot>/<objectType>/1/4/4/1/45743C16-B5C3-75BC-CC9F7E3E7598099D
>
> <objectStoreRoot>/<objectType>/directoryList.txt
>
> <directoryList.txt>
>
> /1/4/2/3/
> /1/4/4/1/
>
> </directoryList.txt>
>
> When you did a getAll, you would have to read the file, then get a
> listing from each of the directories. I don't know if that is any better
> than reading a huge file of all the object files. It could just as
> easily be an xml file. That might be easier to handle. I haven't
> actually tried any of the new xml functions in cfmx yet. The other thing
> that might be useful is that you could change the distribution scheme
> and it would still work fine.
>
> It might be more work than it is worth. And it might be worth much, but
> I wanted to throw it out there. I may give it a test or two this
> evening. What do you guys think?
>
> Brad
>
>
> On Fri, 2002-09-13 at 08:43, Nathan Dintenfass wrote:
> > But, what happens at 10,000 files?  It's one thing to be slow,
> it would be
> > another to bring a machine to its knees.  I did a little searching, but
> > didn't find any hard and fast number on limits.
> >
> > Do you think it matters how big the file are?  I tend to doubt
> it, but if
> > anyone has better info . . .
> >
> >  - n
> >
> > > -----Original Message-----
> > > From: mod...@li...
> > > [mailto:mod...@li...]On Behalf Of Brian
> > > Ghidinelli
> > > Sent: Thursday, September 12, 2002 10:10 PM
> > > To: mod...@li...
> > > Subject: Re: [Modus-devs] advice on file-based persistence
> > >
> > >
> > >
> > > Nathan Dintenfass wrote:
> > > > Again, I don't think the file-based storage needs to be very
> > > scalable, as
> > > > anyone looking to run a scalable app will choose a
> different persistence
> > > > mechanism.  The file-based persister is just for "out-of-the-box"
> > > > implementation and simple sites that don't have access to a
> database.
> > >
> > > I think for this reason alone you should go all files in one
> directory.
> > > Even if you had a "big" site running off of file system
> persistence and
> > > you had 2500 files in a directory, you would probably be just
> fine on most
> > > file systems.
> > >
> > >
> > > Brian
> > >
> > >
> > > -------------------------------------------------------
> > > This sf.net email is sponsored by:ThinkGeek
> > > Welcome to geek heaven.
> > > http://thinkgeek.com/sf
> > > _______________________________________________
> > > Modus-devs mailing list
> > > Mod...@li...
> > > https://lists.sourceforge.net/lists/listinfo/modus-devs
> > >
> >
> >
> >
> > -------------------------------------------------------
> > This sf.net email is sponsored by:ThinkGeek
> > Welcome to geek heaven.
> > http://thinkgeek.com/sf
> > _______________________________________________
> > Modus-devs mailing list
> > Mod...@li...
> > https://lists.sourceforge.net/lists/listinfo/modus-devs
> >
> >
>
>
>
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Modus-devs mailing list
> Mod...@li...
> https://lists.sourceforge.net/lists/listinfo/modus-devs
>