1. Summary
  2. Files
  3. Support
  4. Report Spam
  5. Create account
  6. Log in

Ticket #44 (closed ontology draft: fixed)

Opened 5 years ago

Last modified 3 years ago

Edgewall890 Software851

Reported by: phreedom_ Owned by: konttori
Priority: major Milestone: good-idea-for-later
Component: ontology-nmm Version: 1.1
Keywords: Edgewall Software Cc: mylka, trueg, rwman


This document aims to identify and correct some of flaws of the current Nepomuk approach towards metadata:

  • Significant number of semantically irrelevant properties(as explained in ticket:14), some of which also don't have proper parents from core ontologies
  • Properties putting structures into strings like nid3:trackNumber(track number and track count in a single property)
  • Some NID3 properties apply to audio files with different tag standards/formats such as OGG.
  • Some NID3 properties apply to video(such as music clips) or to the media container itself.
  • Album as a resource, album art handling. While ID3 provides an ability to store this along with audio/clip, not all standards do and in practice this information is downloaded from external databases. It makes sense to make album a first-class citizen and have it contain pictures and other album data, while songs would simply link to the album. Care must be taken to only include basic properties and not try and make NIE into a full-blown music ontology, but instead provide hooks for extension by topic ontologies.
  • Some NEXIF properties apply to video and raster images(color space, detailed sample format description), some define content creator/comments(semantically irrelevant properties, ticket:14), and only some of them are actually photography-specific.
  • Synchronized text from ID3 is also applicable to subtitles in movies: makes sense to turn them into a separate stream like nxx:syncTextStream: there can be english audio but EN, ES, FR subtitles in the movie.

This is an attempt to discuss concepts and issues only. RDF snippets would be trivial to produce and minor details are likely to only distract us, so this is better done later.

Core concepts

nfo:Image vs nmm:Photo

What is the difference between them?

An Image is a computer-generated 3d rendering, drawing, graph, icon or cursor.

While a Photo is something that's made with a camera, which is best evidenced by the presense of camera-specific tags like camera make and model.

The current Nepomuk "unofficially" draws the line by looking at presense of EXIF tags. This is better than nothing, but still not perfect since EXIF tags can be used to provide mundane things like comments(which are actually best represented using NIE and NFO).

It would be much better to keep image-specific things(like color space and pixel definitions) in Image class, creator data in NIE/NFO properties and photo-specific data in Photo class.

nfo:Audio stream vs nmm:Music

Audio is just a digitally recorded or synthesized sound(could be a sound effect, security system recording). Whether this particular Audio is also Music is usually indicated by tags such as artist, album. But music clip is Music too.

Currently for MP3 files Nepomuk draws the distinction by the presense of ID3 tags which seems to be good enough, but fails when it comes to music clips, because Music metadata is best assigned to the container and not audio stream.

Audio stream properties such as bitrate, sample rate are better off stored as properties of the stream. While Music class along with artist, album and such properties is better assigned to either audio or media stream

nmm:Music vs pimo:Music

Whether something is nmm:Music is inferred using tags and/or databases, and it's not subjective since it's automatically inferred using readily available information.

pimo:Music is a subjective interpretation of a user.

These classes are stricly related although probably there's going to be a significant corellation. Same applies to nmm:Movie vs pimo:Movie and nmm:Photo vs pimo:Photo.

nfo:Media stream vs nmm:Music or nmm:Movie

Whether a particular media stream is a Music or Movie or a Music Album depends on what tags are assigned and what databases say. Classes can be added to better represent the metadata and distinctions on as needed-basis when reliable data sources become available.

nmm:Album is a real thing

Stuff that makes sense to store in an nmm:Album:

  • album artist
  • track count
  • album length
  • album art
  • various references
  • a good place to put extra data from topic ontologies and data sources.

Why can't we just search for Album X by artist Y? There are compilation albums and guest artists, which break such queries.

ID3 provides a way to store album art. Why not use it? Album art is often not stored in ID3 tags and downloaded from external databases( all modern players do something like this already). We better not modify files and they could be read-only. Data duplication.

What if you want to use PIMO to annotate a specific album? You'd want a nice groudingOccurence like nmm:Album

Subtitles vs ID3 Synchronized text

Video subtitles are synchronized text too.

But can we use ID3 synchronized text to represent subtitles in videos?

  • Quantity and language of subtitles can differ from the quantity and language of audio tracks. So assigning synchronized text to audio tracks is impossible.
  • You can assign existing ID3 synchronized text to a random resource if you assign the ID3 class to it, but then ID3 class loses any practical use, becomes ambiguous because it can be arbitrarily assigned to music, movies or whatever else needs those properties.
  • Current synchronized text implementation fails to reuse existing NIE properties such as language, creator, license (subtitles can be eg contributed by users), description etc.

Synchronized Text is begging to become a nie:InformationElement, probably a stream and text too.

Putting things where they belong

The use cases mentioned follow the exactly same pattern:

  • Content creator and creator's description of the content
  • Technical data such as dimensions, rates, codecs
  • Classification data such as music, movie series, photos.

We already have NIE for content creators; NFO for technical data, where current audio, visual and such classes reside; and we badly miss a place to put classification data.

Unfortunately legacy standards like ID3 and EXIF tend to mix up all 3 categories of data. Thus the presense of tags in these formats doesn't necessarily provide/imply proper classification, especially if we "abuse" these standards to also describe OGG, Movies, Subtitles.

Less low-level stuff, more semantics and extensibility

As shown above, the presense of ID3 or EXIF tags is just a minor technical detail and not a replacement for proper classification.

If ID3 is "stretched" to also represent Subtitles in movies, assignment of ID3 class loses any practical meaning apart from the fact that we want to assign some properties defined in NID3: to a resource that's somewhat related to media.

Breaking free from low-level details and embracing semantic classification not only means that data extractors can better assist software in representing data to the user. Making less assumptions about the inner workings of formats means we can last longer before hitting the limitations of our model.

Embracing semantic classification means we can integrate and better support many different data sources and databases while availing us of the need to support lots of topic ontologies. All we have to do is provide a few basic "hooks" for topic ontologies.

Proposed migration/uptake path

  1. Initial coexistense with NID3 and NEXIF with NMM being the preferred way to express multimedia metadata
  2. Remapping of NID3 and NEXIF properties as children of NMM to provide smoother migration path
  3. NID3 and NEXIF become either:
    • mappings of external standards onto nepomuk OR
    • extensions over NMM to provide only format-specific(ID3 or EXIF) extensions on top of NMM, with everything else deprecated.

Change History

follow-up: ↓ 3   Changed 4 years ago by trueg

  • cc trueg added

Isn't it about time to integrate this proposal into the "official" ontologies?

  Changed 4 years ago by phreedom_

Urho Konttori:


nmm:fingerprint. There are several and more likely to come. Needs clarification, also good for video and images(digikam does this)

Evgeny Egorochkin:

There are many different kinds of fingerprints for different kinds of content types. I can easily see fingerprints for eg documents. The question is do we need an answer to "has this file a media fingerprint?" without knowing specifics of fingerprints possibly assigned to the file. My bet is unlikely.

NIE has Hash class already which could potentially be useful. The trouble with the class is that hash usually aims at identifying a specific byte sequence while fingerprint aims at identifying the same content. Anyway the fingerprint stuff should go deep into NIE if we want to make a distinction between hash and content fingerprint.

Urho Konttori:

I agree. Let's drop nmm:fingerprint. Or specify a very specific fingerprint. (like the musicCDIdentifier, which makes sense to include - and you have done so)

in reply to: ↑ 1   Changed 4 years ago by phreedom_

  • status changed from new to closed
  • resolution set to fixed

Replying to trueg:

Isn't it about time to integrate this proposal into the "official" ontologies?

Committed as a draft for now because the ontology lacks proper documentation and as such is not good to be published just yet. This doesn't preclude including it into the shared desktop ontologies package. changeset:41

  Changed 4 years ago by trueg

Does this mean we can deprecate nid3?

  Changed 4 years ago by rwman

  • cc rwman added

  Changed 3 years ago by xrumer

  • summary changed from Nepomuk MultiMedia(NMM) ontology proposal draft to Xrumer Forums

In need of Xrumer files? check out the Xrchat forums!

  Changed 3 years ago by trueg

  • summary changed from Xrumer Forums to Nepomuk MultiMedia(NMM) ontology proposal draft

  Changed 3 years ago by TarasBulba

  • keywords Edgewall Software added
  • summary changed from Nepomuk MultiMedia(NMM) ontology proposal draft to Edgewall890 Software851
Note: See TracTickets for help on using tickets.