Menu
▾
▴
Re: [Logicaldoc-devel] [Fwd: [WebDAV]Performance that Matters]
|
From: <m.m...@lo...> - 2008-10-21 13:10:39
|
m.m...@lo... ha scritto: > First of all, it's a pleasure to speak about issues like this, well >> Hi there, >> >> surely i could agree with you in that case that we implement this >> approach by creating a "folder-table". Reconsiderd again, i mention >> we should keep an eye(or better a bunch of eyes) on this behaviour. >> There are much more implications we dont seeing yet. If we have an >> amount of 50 Folders, of course, changing is really fast possible, >> but it depents on transaction level if other peoble are currently on >> the table. Highly frequency gets worse. Therefore we should consider >> following questions: >> >> First: How often will be a folder moved? > By now in LogicalDOC, folders cannot be moved: the user can move > documents only. >> Second: How many Folders and Subfolders exists? > this is unpredictable, i suggest to consider an amount of 2% against > the total amount of documents (2 folders each 100 documents) >> Third: How often is a folder renamed? > In my opinion the folder rename is no frequent, we could estimate one > write in ld_menu each 1000 writes into ld_document. But i think that > one the archive is created it is difficult that a folder changes name. > It is more frequent that new folders are created. >> Fourth: What measures more of the first three points? > I thing that the point is the number of folders that basically is the > number of records in table ls_menu >> >> My mention is that its more possible that a folder must be moved as >> renamed. The payload is unequally higher on the first point if we >> having a folder-table. The amount of payload is defined by the second >> point. The more subfolders exists the longer must be a transaction >> avoid changes/reads(Transaction Level) from other processes. >> Furthermore we have to lookup every WebDAV Request on ever Datarow >> for getting the right one. 250 Folders a 100 KB are 2,5 megabyte that >> have to be each call loaded from hard-drive to the db cache >> (theoretically). In real cache has got in most cases enough size. > Yes Gustav, but if we store the full path in the index(each entry has > a 'folderPath' field), we will have this 100KB multiplied by the > number of documents, let's say 60.000 docs and a total amount of more > than 5.800MB. Or do you mean we have to create and handle a new index > only for folders? But in this case what's better than the DB > implementation? >> >> In fact, using that way by using an index on foldersname can be >> usefully as long as we have enough distinguish folders. Index >> speedups, i guess. The really problem on using indices is only then, >> if we have a highly frequency rename party on the system. This >> imagination is rather funny. But if we have such case, index is >> changed and has to be rebuild for synchronizing. But its no really >> big deal for doing this in nightly hours. > Even in the case of DB table, we can maintain it by using a scheduled > job(but i think it is not necessary, we can update it when the user > changes a folder name). >> >> But you asked me for my proposal. >> >> My expierience with thousends of documents is not really god-like at >> all ;-) I agree with you as long as we doing testcases with >> auto-generated folders in an appropriating size at tje end of the >> implementation. >> >> Best Regards > Also we have to face another issue: read permissions. We have to list > only those folders the user can see, but these permissions are stored > int the DB, so the DB solution it is more simple to retrieve only > accessible paths by using SQL join. > > Tell me Gustav, explain me more details about your idea to use the > lucene index and why do you thing this solution is better than the > database one. > I think we can discuss about this issue and find the right solution in > order to start the module development. > > PS > Let me say some development lines that we have to follow, because i > think you can understand me: > > 1. First of all things must run, than they can be beautiful. If a > beautiful feature doesn't run on a large population of > documents, it is dropped > 2. Performances are the key element of this software, we would be > able to give a search respose time of 4 seconds on a polulation > of 1 million documents > 3. The program must be as simple as possible to the end user, it > can be a hell for developers(i hope not so hot) but it MUST be > simple for the end user > > As you can see, we are implementing very deep changes in the data > model in order to maximize performances. > > I appreciate your considerations about performances. My compliments > Gustav! > > --------------------------------------------------------------- > ing. Marco Meschieri > e-mail: m.m...@lo... > <mailto:m.m...@lo...> > --------------------------------------------------------------- > Logical Objects snc > Via Bonasi, 2/A 41012 Carpi (MO) Italy > Tel./Fax. 059 688969 > web: http://www.logicalobjects.it > > > -- > AVVERTENZE AI SENSI DEL DLGS 196/2003: > Le informazioni contenute in questo messaggio di posta elettronica e/o > nel/i file/s allegato/i, sono da considerarsi strettamente riservate. > Il loro utilizzo e' consentito esclusivamente al destinatario del > messaggio, per le finalita' inidcate nel messaggio stesso. Qualora > riceveste questo messaggio senza esserne il destinatario Vi preghiamo > cortesemente di darcene notizia via e-mail e di procedere alla > distruzione del messaggio stesso, cancellando dal vostro sistema; > costituisce comportamento contrario ai principi dettati dal > Dlgs196/2003 il trattenere il messaggio stesso, divulgandolo anche in > parte, distribuirlo ad altri soggetti,copiarlo, od utilizzarlo per > finalita'diverse. Titolare del trattamento e' Logical Objects SNC Via > Bonasi 2/A 41012 CARPI (MO) Tel. 059/688969 Fax 059/688969 > > This e-mail and any file transmitted with it is intended only for the > person or entity to which is addressed and may contain information > that is privileged, confidential or otherwise protected from > disclosure. Copying, dissemination or use of this e-mail or the > information herein by anyone other than the intended recipient is > prohibited. If you have received this e-mail by mistake, please notify > us immediately by telephone or fax. Proprietor of treatment is Logical > Objects SNC Via Bonasi 2/A 41012 CARPI (MO) Tel. 059/688969 Fax > 059/688969 -- --------------------------------------------------------------- ing. Marco Meschieri e-mail: m.m...@lo... <mailto:m.m...@lo...> --------------------------------------------------------------- Logical Objects snc Via Bonasi, 2/A 41012 Carpi (MO) Italy Tel./Fax. 059 688969 web: http://www.logicalobjects.it -- AVVERTENZE AI SENSI DEL DLGS 196/2003: Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i, sono da considerarsi strettamente riservate. Il loro utilizzo e' consentito esclusivamente al destinatario del messaggio, per le finalita' inidcate nel messaggio stesso. Qualora riceveste questo messaggio senza esserne il destinatario Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellando dal vostro sistema; costituisce comportamento contrario ai principi dettati dal Dlgs196/2003 il trattenere il messaggio stesso, divulgandolo anche in parte, distribuirlo ad altri soggetti,copiarlo, od utilizzarlo per finalita'diverse. Titolare del trattamento e' Logical Objects SNC Via Bonasi 2/A 41012 CARPI (MO) Tel. 059/688969 Fax 059/688969 This e-mail and any file transmitted with it is intended only for the person or entity to which is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. Copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this e-mail by mistake, please notify us immediately by telephone or fax. Proprietor of treatment is Logical Objects SNC Via Bonasi 2/A 41012 CARPI (MO) Tel. 059/688969 Fax 059/688969 |
