Tracker: Feature Requests

5 Manual non-destructive database refresh - ID: 3114841
Last Update: Comment added ( jamespharaoh )

MiniDLNA fails to pickup changes done to networked filesystems (no inotify support for these) or done to local filesystems while it's not running. I would recommend providing a way to refresh the database content that doesn't involve rebuilding it from scratch. This could be done either through a command line switch, or on a configurable timer (db_refresh=14400 would refresh the DB every 24 hours)


Eric Sauvageau ( rmerlin ) - 2010-11-21 14:12:37 PST

5

Open

None

Nobody/Anonymous

None

None

Public


Comments ( 9 )

Date: 2012-04-12 00:50:39 PDT
Sender: jamespharaoh

I also think this is a very important feature. jsanchez and cionci77 have
the right idea - it should be a light refresh which only examines the file
meta data unless it looks like something has changed.

I don't think handling deleted files is hard as jsanchez suggests. You just
compare the list of each directory you scan to the list in the database as
you go. No need to do it all in one batch at the end.

I think cionci77's idea to crc the end of the file is a bad idea. This will
slow things a lot. We can simply use the timestamp (and other meta data) to
detect probably changes.

This should probably be the default on every startup, probably at the same
time as the inotify watches are placed. I guess some people might want to
disable it for efficiency but I think the "least surprising option" is for
it to perform the scan.

I might have a go at this some time soon. It would be interesting to hear
from the project maintainers on it?


Date: 2012-01-11 20:53:35 PST
Sender: tehsu

I think this would be a good idea as for OS X I don't see anyway to use
inotify.


Date: 2011-03-29 09:19:55 PDT
Sender: frederic-b

Yes war59312 I tried with ushare and it is incredibly faster. For 700 Go of
files - Ushare some minutes, Minidlna hours - (Nas DNS 323)

But Ushare use a trick, it work only in memory not with sqlite


Date: 2011-03-22 03:37:43 PDT
Sender: lasaAccepting Donations

I really would like to see this feature as well. On a fast machine scanning
and indexing all my media files still take 20 minutes.

It would also be nice if it was documented somewhere in the files provided
in the distribution that the following setting must be larger than the
number of media files:

echo 65536 >/proc/sys/fs/inotify/max_user_watches


Date: 2011-02-21 04:11:44 PST
Sender: frederic-b

Unfortunately creating a index for a larger database can be very time
consuming, for this i think non-destructive database refresh can be a good
improvement

Thank you


Date: 2011-02-20 10:44:40 PST
Sender: war59312Accepting Donations

Sounds good to me, personally I like how ushare handles it.


Date: 2011-01-27 02:43:35 PST
Sender: jsanchez

The logic behind inotify seems to be easy to leverage for incremental
scans. It is possible to scan all dirs and call inotify_insert_file for
every file, its logic will ignore files whose timestamps have not changed
and skip them. This is efficient, but will not detect modified files with
unchanged time. Maybe add a force flag or something. Detecting deleted
files is slightly harder. Best approach is to add a "present" column to
the object table, make it default to "true". When scanning, set "present"
on all rows to "false", as every object is seen, set the value to "true".
At the end of the scan, call inotify_remove_file for each file where
"present" is "false". Add a global lock so you don't get into this routine
twice or a real mess will be the result.


Date: 2010-12-12 04:36:17 PST
Sender: cionci77

I've been looking for the same feature. At least a smart rescan option
should be provided at minidlna startup:
- take a fast fingerprint of each file (i.e.: use file size, file name and
crc32 of a bunch of data at the end of file)
- if fingerprint already in database, use metadata from previous entry
- if fingerprint not in database scan for new metadata

Thank you


Date: 2010-11-23 11:26:38 PST
Sender: fktpa813

I too would like MiniDLNA to be able to do a "refresh on demand" - some way
to simulate the "inotify" event.

For example, here's the simplistic solution that if implemented, would work
for me perfectly... If MiniDLNA gets SIGHUP signal, it can look for a
certain text file (the file name can be configurable), and then read from
it the list of directories and files that have to be added to (or
refreshed, if it already exists) or removed from the database. Each line in
the file will start from "+" or "-" sign (to indicate "add/refresh" or
"remove") followed by the directory/file name, and can be handled the same
way as corresponding inotify event.


Attached File

No Files Currently Attached

Change

No changes have been made to this artifact.