RE: [Modus-devs] advice on file-based persistence

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

I don't think the files would get very large. The way I was playing
around with it was to have 4 directories each 4 deep with 4 directories
on each level. That gives (if my math is correct) 256 bottom level
directories. So directoyList would be at most 256 lines (short ones). It
does seem like this would be more steps than necessary. 

I do like your solution of appending the objectType to the file name. 

What about the storage of other content, like images?

Brad

On Fri, 2002-09-13 at 17:35, Nathan Dintenfass wrote:
> I am trying to avoid having to read files to do a get or a getAll, as
> reading large files and parsing them would be very expensive relative to
> listing out directory contents.
> 
> I'm also now thinking that if I am going to store them in one big directory
> anyway, I might just store them named like:
> 
> objectID.objectType
> 
> I could then just use the filter attribute of CFDIRECTORY when getting all
> of a certain type.
> 
> Most contenObjects will only be a few kb stored as flat files, and I figure
> nobody will use the file storage for many many thousands of objects, so it
> should work.
> 
>  - n
> 
> > -----Original Message-----
> > From: mod...@li...
> > [mailto:mod...@li...]On Behalf Of Brad Pauly
> > Sent: Friday, September 13, 2002 3:57 PM
> > To: modus devs
> > Subject: RE: [Modus-devs] advice on file-based persistence
> >
> >
> > I had another idea about storing objects. Using both the idea of keeping
> > object types in separate directories and distributing the files into
> > subdirectories. Then keeping a list of the directories in a file in the
> > object type directory. So a path would look something like:
> >
> > <objectStoreRoot>/<objectType>/1/4/2/3/45740C9C-CCFF-20DE-D6C92C971A2A440B
> > <objectStoreRoot>/<objectType>/1/4/4/1/45743C16-B5C3-75BC-CC9F7E3E7598099D
> >
> > <objectStoreRoot>/<objectType>/directoryList.txt
> >
> > <directoryList.txt>
> >
> > /1/4/2/3/
> > /1/4/4/1/
> >
> > </directoryList.txt>
> >
> > When you did a getAll, you would have to read the file, then get a
> > listing from each of the directories. I don't know if that is any better
> > than reading a huge file of all the object files. It could just as
> > easily be an xml file. That might be easier to handle. I haven't
> > actually tried any of the new xml functions in cfmx yet. The other thing
> > that might be useful is that you could change the distribution scheme
> > and it would still work fine.
> >
> > It might be more work than it is worth. And it might be worth much, but
> > I wanted to throw it out there. I may give it a test or two this
> > evening. What do you guys think?
> >
> > Brad
> >
> >
> > On Fri, 2002-09-13 at 08:43, Nathan Dintenfass wrote:
> > > But, what happens at 10,000 files?  It's one thing to be slow,
> > it would be
> > > another to bring a machine to its knees.  I did a little searching, but
> > > didn't find any hard and fast number on limits.
> > >
> > > Do you think it matters how big the file are?  I tend to doubt
> > it, but if
> > > anyone has better info . . .
> > >
> > >  - n
> > >
> > > > -----Original Message-----
> > > > From: mod...@li...
> > > > [mailto:mod...@li...]On Behalf Of Brian
> > > > Ghidinelli
> > > > Sent: Thursday, September 12, 2002 10:10 PM
> > > > To: mod...@li...
> > > > Subject: Re: [Modus-devs] advice on file-based persistence
> > > >
> > > >
> > > >
> > > > Nathan Dintenfass wrote:
> > > > > Again, I don't think the file-based storage needs to be very
> > > > scalable, as
> > > > > anyone looking to run a scalable app will choose a
> > different persistence
> > > > > mechanism.  The file-based persister is just for "out-of-the-box"
> > > > > implementation and simple sites that don't have access to a
> > database.
> > > >
> > > > I think for this reason alone you should go all files in one
> > directory.
> > > > Even if you had a "big" site running off of file system
> > persistence and
> > > > you had 2500 files in a directory, you would probably be just
> > fine on most
> > > > file systems.
> > > >
> > > >
> > > > Brian
> > > >
> > > >
> > > > -------------------------------------------------------
> > > > This sf.net email is sponsored by:ThinkGeek
> > > > Welcome to geek heaven.
> > > > http://thinkgeek.com/sf
> > > > _______________________________________________
> > > > Modus-devs mailing list
> > > > Mod...@li...
> > > > https://lists.sourceforge.net/lists/listinfo/modus-devs
> > > >
> > >
> > >
> > >
> > > -------------------------------------------------------
> > > This sf.net email is sponsored by:ThinkGeek
> > > Welcome to geek heaven.
> > > http://thinkgeek.com/sf
> > > _______________________________________________
> > > Modus-devs mailing list
> > > Mod...@li...
> > > https://lists.sourceforge.net/lists/listinfo/modus-devs
> > >
> > >
> >
> >
> >
> >
> >
> > -------------------------------------------------------
> > This sf.net email is sponsored by:ThinkGeek
> > Welcome to geek heaven.
> > http://thinkgeek.com/sf
> > _______________________________________________
> > Modus-devs mailing list
> > Mod...@li...
> > https://lists.sourceforge.net/lists/listinfo/modus-devs
> >
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Modus-devs mailing list
> Mod...@li...
> https://lists.sourceforge.net/lists/listinfo/modus-devs
> 
> 