Over the years, I've got into the habit of using CAAFMasterMob::OpenEssence() to open any media essence that I need. I've been doing this for years and it's the only strategy I've ever used. CAAFMasterMob::OpenEssence() ultimately ends up calling ImplAAFEssenceAccess::Open(). This seems logical to me and I've always assumed it to be the recommended way of opening essence.
Lately (maybe during the past year) I've been presented with an increasing number of AAF files (mostly originated by Avid and DigiDesign) whose essence descriptors do not contain a codec definition. In fact, since the beginning of this year, pretty much every AAF file from these 2 originators has contained no codec defs at all (according to InfoDumper). This is borne out when I single step through ImplAAFEssenceAccess::Open().
Line 1764 of ImplAAFEssenceAccess.cpp fails if the Essence Descriptor has no codec def present. Therefore all attempts to open media using my current strategy are doomed to failure. Can anyone suggest an alternative way to open essence since I need to abandon CAAFMasterMob::OpenEssence().
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
If you are looking for a sample based access to and encoding/decoding of the media the IAAFEssenceAccess interface is the only option. You can access raw stream via IAAFEssenceData2 or IAAFTypeDefStream3 interfaces. But I wonder if there's another way to get IAAFEssenceAccess interface, maybe repeating the steps from OpenEssence?
This issue is not limited to AAF files though. Most of MXF files out there don't have this property either. MXF support in AAF codecs is planned for one of the next SDK releases and this problem will have to be addressed there, one way or another.
As for the fate of the Codec property in Avid AAF files - I don't know why it's been removed. If I can find anything out I will let you know.
Alexey
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks for your help Alexey. I don't know if it's significant - but since my first post, I've realised that these particular AAF files have no codecs registered at all; not even in the dictionary (in other words, CAAFDictionary::CountCodecDefs() returns zero. I thought it was mandatory that the common codecs (such as JPEG, AIFC and WAVE) needed to be registered for every session. Maybe I'm getting mixed up with something else or has this requirement been changed without me realising it?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Alexey - you wrote:
" You can access raw stream via IAAFEssenceData2 or IAAFTypeDefStream3 interfaces. But I wonder if there's another way to get IAAFEssenceAccess interface, maybe repeating the steps from OpenEssence? "
Hi Alexey - I took a look at that but I can't see an obvious way to extract the code from OpenEssence and use it within the context of a master mob. Ultimately, OpenEssence relies on knowing what codec to use - but the master mob has no knowledge of codecs (indeed, a master mob may reference many different types of media, so a simple _codec member would be meaningless). IAAFEssenceData::Read() is the only other possibility - but it would be very cumbersome to utilise that, when given only a master mob ID and slot ID (it's not impossible - just cumbersome).
In any case, the bottom line here is that a suitable interface is already built into the toolkit, precisely for this purpose - namely; IAAFMasterMob::OpenEssence() - but someone (in all probability, inadvertantly) has broken it. The relevant files can be imported back into Avid & ProTools (I know that because I've tried it) and the fact that the originating systems can cope with the error probably explains why it slipped through the net. Nevertheless, this is quite a serious problem. There'll be loads of kit out there that simply won't be able to import from Avid or Pro Tools any more.
This kind of thing has important ramifications for AAF/AMW. Given that all specifications are open to interpretation, who at AMW (if anyone) is responsible for deciding when an originator has overstepped the mark (even innocuously). The goal of AAF was surely to deliver a stable, reliable interchange format - but that simply cannot happen if originators - however innocuously - are exporting files that nothing else can read.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I must confess I’m surprised that there’s only been one reply about this issue (even allowing for the generally small traffic on the lists). AFAICT, this problem will pretty much break AAF for anyone whose name isn’t Avid or Digidesign. So what’s consensus? Am I panicking unnecessarily or has the rest of the AMW community stopped bothering??
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
John, this is a serious issue and thank you for bringing this up.
The presence of AAF codecs is not mandatory. Neither is the presence of the FileDescriptor::CodecDef property. It is hard to make a case for them to be required by the main specification. There's enough metadata in an AAF file to identify, access and decode essence without use of IAAFEssenceAccess. Also, there's the MXF standard and MXF files to consider. They have the same issues and they're out of the Association's control.
Avid products do not use AAF codecs simply because they don't need to (see above.) You are probably right about this being the reason why the change to not write out CodecDef went unnoticed on the Avid's side. As far as I know the change was accidental and it's the first time I hear someone mentioning this issue.
The best course of action is to make the EssenceAccess implementation more robust. It's the right thing to do and it will probably be faster than waiting for Avid to put CodecDef back into its AAF files (I don't see any reason why it shouldn't).
Alexey
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Alexey - you wrote:-
" The presence of AAF codecs is not mandatory. Neither is the presence of the FileDescriptor::CodecDef property. It is hard to make a case for them to be required by the main specification. "
Alexey and Phil - I'll try to make a case for this a bit later but in the meantime, I'd just like to say that I do appreciate the effort and thought that you've both put into this. I tried to see if there might be a 'simple' way of fixing the problem. Given that an essence accessor already has its own codec ID, I thought that something like this might work:-
Lines 1764 thru 1767 of ImplAAFEssenceAccess.cpp currently look like this:-
In this particular case I was expecting it to return a codec ID of kAAFCodecDef_WAVE - whereas in fact, it returned kAAFCodecDef_None. So no luck there. I can't ascertain the correct codec that way either.
Here's my case for changing the spec:- The fundamental job of AAF/AMW (or indeed, any interchange specification) is to deliver a common format by which relevant data can be easily exchanged between interested participants. This simply cannot happen if the import methods require properties to be present but the export methods are free to leave them out. Regardless of what the SMPTE specification says, the fundamental requirement of an interchange format is that party A should be able to read party B's files. if each originator is free to produce files which aren't guaranteed to be readable by the other participants, doesn't that defeat the whole object of the exercise?
I’m sorry if this seems harsh but from where I stand, AAF/AMW is a technology in crisis. Like OMF before it, AAF is becoming dominated by self-interested parties all churning out incompatible files whilst claiming that their interpretation of the spec is as good as anyone else’s. Already this year I’ve reported files with duplicated mob ID’s, mob slots with negative offsets and now files with missing codec definitions. In previous years I’ve reported other show-stopping problems that nobody seemed willing to police. Is there nobody in AMW who is willing (or whose job it is) to enforce consistency?
Whilst I’ve always been one of AAF’s strongest advocates, I’m very much on the fringes. I’m just a small, one-man-band and I don’t have the resources to subscribe or attend meetings - much as I’d like to. And because I’m on the fringes it’s quite possible that I’m misreading the situation - or that there’s some great method in the apparent mayhem. It has to be said though that - to those of us on the fringes - it seems to be a simple case of too many chiefs and not enough indians.
In spite of the lofty names supporting it, out in the field AAF is perceived as being somewhat rudderless; a big pie with too many people sticking their fingers in (sorry for the mixed metaphors). I wish I had the answer but I don’t. I can only point out the irony of OMF, whose reputation went up markedly after everyone stopped tinkering with it. Maybe there’s a lesson to be learned there....
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I think you're making a case for defining a protocol for exchange of audio and video essence via AAF files.
The AAF specification does not define import and export methods. In the best case scenario those are defined some place else. In the worst case scenario the parties involved define their own import and export methods and hope they are compatible. IAAFEssenceAccess happens to implement an import mechanism that puts certain constraints on generic AAF object model, like requiring an essence descriptor to carry information about a codec. Avid implements an export mechanism that omits certain metadata elements, if allowed by the specification, including codec information. The two mechanisms are incompatible. You can blame IAAFEssenceAccess or you can blame Avid but I don't think it's the fault of the AAF specification.
Alexey
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi Alexey - so, from what you're saying (in fact, from re-reading SourceForge's 'summary' of the AAF project) it is NOT one of the goals of AAF that the format should be useable as an interchange mechanism between systems created by its participants? Or (to put it another way) if it happens to work as an interchange mechanism, that's simply a happy accident - not one of the intended design goals. Is that a reasonable summary of the situation?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
What Phil said (https://sourceforge.net/forum/message.php?msg_id=4864538). Thanks Phil.
AAF should be usable as a generic interchange mechanism as long as the participants comply with the AAF specification. As the part of the official reference implementation IAAFEssenceAccess should implement the generic mechanism. It tries to but has some issues and we will be fixing them.
My comment in the previous post about parties defining their own import/export methods probably needs more context. What I mean is that AAF is designed for generic interchange but less generic mechanisms, based on AAF, may also be defined to suit particular purposes. Already there's AMWA's Edit Protocol, a document that applies some constraints to AAF to simplify metadata interchange between non-linear editors. There's MXF, which uses a subset of the AAF object model, the minimum needed for optimized media interchange. And even MXF is then further constrained by operational patterns defined in additional SMPTE documents. But all these 'formats' are based on AAF and applications that support generic AAF interchange will also understand both MXF and EP metadata. MXF and EP applications, on the other hand, will only understand metadata that comply with their respective refinements of the AAF object model.
Alexey
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
This issue has been there since year zero - that MasterMob::EssenceAccess can only open essence if the encoder writes into the file which codec the decoder should use. FileDescriptor::CodecDef is optional so this information can be missing. We have got away with it because AAF applications typically have added a codec def which matches one of the codecs in the SDK.
I don't think the answer is just to require the encoder to indicate which codec to use, although this has been the workaround up until now. One way to go might be to enable the decoder to set which codec to use as part of the OpenEssence call, based on inspection of the EssenceDescriptor (i.e. CompressionID etc.) and knowledge of what codecs are actually available at the decoder.
As Alexey mentioned, there are some tweaks planned to the codecs for v113 and we will add this to the list to fix.
Phil
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Sorting it out at the decoder end isn't too dissimilar from what I was considering for a workaround. However, I've refrained from starting yet because I don't know whether it would still work for a file that's opened "read only".
As you observed, the basic problem is that FileDescriptor::CodecDef is an optional property - and yet EssenceAccess::Open() well and truly falls over if the codec def isn't present. What would be the detriment in making it a required property instead of an optional one? If there were file descriptors for which a codec definition isn't appropriate, it could surely be set (or even initialised by default) to kAAFCodecDef_None
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
It's the other way around. The problem is that EssenceAccess relies on the optional FileDescriptor::CodecDef to be always present.
Making CodecDef required would be against the rules. As far as I remember, we did flip status of a few properties but it was to comply with the MXF standards. Either way, it would not be a simple change because we need to maintain forward and backward compatibility. Also don't forget that CodecDef is specified as optional by SMPTE 377M-2004. I think it is easier and more appropriate to fix EssenceAccess.
Alexey
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I don't think that's an accurate summary of Alexey's posting. He is distinguishing between what the AAF specification defines for the format, and the behaviour of a particular helper method in the AAF toolkit. His point is that the interchange standard is the file format, not the implementation methods at encoder and decoder. That the helper method does not robustly work with all files compliant with the file format specification is unfortunate and a bug - but we cannot solve it by retrospectively redefining the file format to say that the element needed by the current helper function is now required.
The consensus of the AMWA is that the best approach is to mend the implementation of OpenEssence in the AAF toolkit so that it works robustly for all files compliant with the file format specification.
Regards,
Phil Tudor
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Ah - that clarifies it, Phil. I misinterpreted Alexey's remark about each party defining its own import/export methodology and 'hoping' they'd somehow be compatible.
Is it too early to say when OpenEssence might be scheduled for a fix? I appreciate that the problem was totally unforeseen but it's nevertheless an important show stopper at the moment.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Over the years, I've got into the habit of using CAAFMasterMob::OpenEssence() to open any media essence that I need. I've been doing this for years and it's the only strategy I've ever used. CAAFMasterMob::OpenEssence() ultimately ends up calling ImplAAFEssenceAccess::Open(). This seems logical to me and I've always assumed it to be the recommended way of opening essence.
Lately (maybe during the past year) I've been presented with an increasing number of AAF files (mostly originated by Avid and DigiDesign) whose essence descriptors do not contain a codec definition. In fact, since the beginning of this year, pretty much every AAF file from these 2 originators has contained no codec defs at all (according to InfoDumper). This is borne out when I single step through ImplAAFEssenceAccess::Open().
Line 1764 of ImplAAFEssenceAccess.cpp fails if the Essence Descriptor has no codec def present. Therefore all attempts to open media using my current strategy are doomed to failure. Can anyone suggest an alternative way to open essence since I need to abandon CAAFMasterMob::OpenEssence().
If you are looking for a sample based access to and encoding/decoding of the media the IAAFEssenceAccess interface is the only option. You can access raw stream via IAAFEssenceData2 or IAAFTypeDefStream3 interfaces. But I wonder if there's another way to get IAAFEssenceAccess interface, maybe repeating the steps from OpenEssence?
This issue is not limited to AAF files though. Most of MXF files out there don't have this property either. MXF support in AAF codecs is planned for one of the next SDK releases and this problem will have to be addressed there, one way or another.
As for the fate of the Codec property in Avid AAF files - I don't know why it's been removed. If I can find anything out I will let you know.
Alexey
Thanks for your help Alexey. I don't know if it's significant - but since my first post, I've realised that these particular AAF files have no codecs registered at all; not even in the dictionary (in other words, CAAFDictionary::CountCodecDefs() returns zero. I thought it was mandatory that the common codecs (such as JPEG, AIFC and WAVE) needed to be registered for every session. Maybe I'm getting mixed up with something else or has this requirement been changed without me realising it?
Alexey - you wrote:
" You can access raw stream via IAAFEssenceData2 or IAAFTypeDefStream3 interfaces. But I wonder if there's another way to get IAAFEssenceAccess interface, maybe repeating the steps from OpenEssence? "
Hi Alexey - I took a look at that but I can't see an obvious way to extract the code from OpenEssence and use it within the context of a master mob. Ultimately, OpenEssence relies on knowing what codec to use - but the master mob has no knowledge of codecs (indeed, a master mob may reference many different types of media, so a simple _codec member would be meaningless). IAAFEssenceData::Read() is the only other possibility - but it would be very cumbersome to utilise that, when given only a master mob ID and slot ID (it's not impossible - just cumbersome).
In any case, the bottom line here is that a suitable interface is already built into the toolkit, precisely for this purpose - namely; IAAFMasterMob::OpenEssence() - but someone (in all probability, inadvertantly) has broken it. The relevant files can be imported back into Avid & ProTools (I know that because I've tried it) and the fact that the originating systems can cope with the error probably explains why it slipped through the net. Nevertheless, this is quite a serious problem. There'll be loads of kit out there that simply won't be able to import from Avid or Pro Tools any more.
This kind of thing has important ramifications for AAF/AMW. Given that all specifications are open to interpretation, who at AMW (if anyone) is responsible for deciding when an originator has overstepped the mark (even innocuously). The goal of AAF was surely to deliver a stable, reliable interchange format - but that simply cannot happen if originators - however innocuously - are exporting files that nothing else can read.
I must confess I’m surprised that there’s only been one reply about this issue (even allowing for the generally small traffic on the lists). AFAICT, this problem will pretty much break AAF for anyone whose name isn’t Avid or Digidesign. So what’s consensus? Am I panicking unnecessarily or has the rest of the AMW community stopped bothering??
Pardon me, I got a bit distracted.
John, this is a serious issue and thank you for bringing this up.
The presence of AAF codecs is not mandatory. Neither is the presence of the FileDescriptor::CodecDef property. It is hard to make a case for them to be required by the main specification. There's enough metadata in an AAF file to identify, access and decode essence without use of IAAFEssenceAccess. Also, there's the MXF standard and MXF files to consider. They have the same issues and they're out of the Association's control.
Avid products do not use AAF codecs simply because they don't need to (see above.) You are probably right about this being the reason why the change to not write out CodecDef went unnoticed on the Avid's side. As far as I know the change was accidental and it's the first time I hear someone mentioning this issue.
The best course of action is to make the EssenceAccess implementation more robust. It's the right thing to do and it will probably be faster than waiting for Avid to put CodecDef back into its AAF files (I don't see any reason why it shouldn't).
Alexey
Alexey - you wrote:-
" The presence of AAF codecs is not mandatory. Neither is the presence of the FileDescriptor::CodecDef property. It is hard to make a case for them to be required by the main specification. "
Alexey and Phil - I'll try to make a case for this a bit later but in the meantime, I'd just like to say that I do appreciate the effort and thought that you've both put into this. I tried to see if there might be a 'simple' way of fixing the problem. Given that an essence accessor already has its own codec ID, I thought that something like this might work:-
Lines 1764 thru 1767 of ImplAAFEssenceAccess.cpp currently look like this:-
CHECK(access.mdes->GetCodecDef(&codecDef));
CHECK(codecDef->GetAUID(&codecID));
codecDef->ReleaseReference();
codecDef = NULL;
I changed them to look like this:-
if (FAILED(access.mdes->GetCodecDef(&codecDef)))
{
CHECK(GetCodecID(&codecID));
}
else
{
CHECK(codecDef->GetAUID(&codecID));
codecDef->ReleaseReference();
codecDef = NULL;
}
In this particular case I was expecting it to return a codec ID of kAAFCodecDef_WAVE - whereas in fact, it returned kAAFCodecDef_None. So no luck there. I can't ascertain the correct codec that way either.
Here's my case for changing the spec:- The fundamental job of AAF/AMW (or indeed, any interchange specification) is to deliver a common format by which relevant data can be easily exchanged between interested participants. This simply cannot happen if the import methods require properties to be present but the export methods are free to leave them out. Regardless of what the SMPTE specification says, the fundamental requirement of an interchange format is that party A should be able to read party B's files. if each originator is free to produce files which aren't guaranteed to be readable by the other participants, doesn't that defeat the whole object of the exercise?
I’m sorry if this seems harsh but from where I stand, AAF/AMW is a technology in crisis. Like OMF before it, AAF is becoming dominated by self-interested parties all churning out incompatible files whilst claiming that their interpretation of the spec is as good as anyone else’s. Already this year I’ve reported files with duplicated mob ID’s, mob slots with negative offsets and now files with missing codec definitions. In previous years I’ve reported other show-stopping problems that nobody seemed willing to police. Is there nobody in AMW who is willing (or whose job it is) to enforce consistency?
Whilst I’ve always been one of AAF’s strongest advocates, I’m very much on the fringes. I’m just a small, one-man-band and I don’t have the resources to subscribe or attend meetings - much as I’d like to. And because I’m on the fringes it’s quite possible that I’m misreading the situation - or that there’s some great method in the apparent mayhem. It has to be said though that - to those of us on the fringes - it seems to be a simple case of too many chiefs and not enough indians.
In spite of the lofty names supporting it, out in the field AAF is perceived as being somewhat rudderless; a big pie with too many people sticking their fingers in (sorry for the mixed metaphors). I wish I had the answer but I don’t. I can only point out the irony of OMF, whose reputation went up markedly after everyone stopped tinkering with it. Maybe there’s a lesson to be learned there....
John,
I think you're making a case for defining a protocol for exchange of audio and video essence via AAF files.
The AAF specification does not define import and export methods. In the best case scenario those are defined some place else. In the worst case scenario the parties involved define their own import and export methods and hope they are compatible. IAAFEssenceAccess happens to implement an import mechanism that puts certain constraints on generic AAF object model, like requiring an essence descriptor to carry information about a codec. Avid implements an export mechanism that omits certain metadata elements, if allowed by the specification, including codec information. The two mechanisms are incompatible. You can blame IAAFEssenceAccess or you can blame Avid but I don't think it's the fault of the AAF specification.
Alexey
Hi Alexey - so, from what you're saying (in fact, from re-reading SourceForge's 'summary' of the AAF project) it is NOT one of the goals of AAF that the format should be useable as an interchange mechanism between systems created by its participants? Or (to put it another way) if it happens to work as an interchange mechanism, that's simply a happy accident - not one of the intended design goals. Is that a reasonable summary of the situation?
What Phil said (https://sourceforge.net/forum/message.php?msg_id=4864538). Thanks Phil.
AAF should be usable as a generic interchange mechanism as long as the participants comply with the AAF specification. As the part of the official reference implementation IAAFEssenceAccess should implement the generic mechanism. It tries to but has some issues and we will be fixing them.
My comment in the previous post about parties defining their own import/export methods probably needs more context. What I mean is that AAF is designed for generic interchange but less generic mechanisms, based on AAF, may also be defined to suit particular purposes. Already there's AMWA's Edit Protocol, a document that applies some constraints to AAF to simplify metadata interchange between non-linear editors. There's MXF, which uses a subset of the AAF object model, the minimum needed for optimized media interchange. And even MXF is then further constrained by operational patterns defined in additional SMPTE documents. But all these 'formats' are based on AAF and applications that support generic AAF interchange will also understand both MXF and EP metadata. MXF and EP applications, on the other hand, will only understand metadata that comply with their respective refinements of the AAF object model.
Alexey
Hi John,
This issue has been there since year zero - that MasterMob::EssenceAccess can only open essence if the encoder writes into the file which codec the decoder should use. FileDescriptor::CodecDef is optional so this information can be missing. We have got away with it because AAF applications typically have added a codec def which matches one of the codecs in the SDK.
I don't think the answer is just to require the encoder to indicate which codec to use, although this has been the workaround up until now. One way to go might be to enable the decoder to set which codec to use as part of the OpenEssence call, based on inspection of the EssenceDescriptor (i.e. CompressionID etc.) and knowledge of what codecs are actually available at the decoder.
As Alexey mentioned, there are some tweaks planned to the codecs for v113 and we will add this to the list to fix.
Phil
Hi Phil,
Sorting it out at the decoder end isn't too dissimilar from what I was considering for a workaround. However, I've refrained from starting yet because I don't know whether it would still work for a file that's opened "read only".
As you observed, the basic problem is that FileDescriptor::CodecDef is an optional property - and yet EssenceAccess::Open() well and truly falls over if the codec def isn't present. What would be the detriment in making it a required property instead of an optional one? If there were file descriptors for which a codec definition isn't appropriate, it could surely be set (or even initialised by default) to kAAFCodecDef_None
It's the other way around. The problem is that EssenceAccess relies on the optional FileDescriptor::CodecDef to be always present.
Making CodecDef required would be against the rules. As far as I remember, we did flip status of a few properties but it was to comply with the MXF standards. Either way, it would not be a simple change because we need to maintain forward and backward compatibility. Also don't forget that CodecDef is specified as optional by SMPTE 377M-2004. I think it is easier and more appropriate to fix EssenceAccess.
Alexey
John,
I don't think that's an accurate summary of Alexey's posting. He is distinguishing between what the AAF specification defines for the format, and the behaviour of a particular helper method in the AAF toolkit. His point is that the interchange standard is the file format, not the implementation methods at encoder and decoder. That the helper method does not robustly work with all files compliant with the file format specification is unfortunate and a bug - but we cannot solve it by retrospectively redefining the file format to say that the element needed by the current helper function is now required.
The consensus of the AMWA is that the best approach is to mend the implementation of OpenEssence in the AAF toolkit so that it works robustly for all files compliant with the file format specification.
Regards,
Phil Tudor
Ah - that clarifies it, Phil. I misinterpreted Alexey's remark about each party defining its own import/export methodology and 'hoping' they'd somehow be compatible.
Is it too early to say when OpenEssence might be scheduled for a fix? I appreciate that the problem was totally unforeseen but it's nevertheless an important show stopper at the moment.