Re: [Lcms-user] profile corrupt after update

[Maria, M. <mar...@hp...>]

>because I extract the CurveSets from the original tag.
> [...]
>other data (like CurveSet) are indeed extracted from the read pipeline and inserted in the new one

Here is the issue. Two pipelines are sharing same stage. When one is set free, the other got corrupted. Just make a copy with cmsStageDup and all will be solved.

>Making the copy is very counter productive, since updating a tag would mean in that way that all data is at least copied 3 times:
>- from read tag to duplicated tag
>- from duplicated tag to modified tag
>- from modified tag to written tag (because the write will copy again)

cmsReadTag() returns a pointer to the tag object, then you make copy , modify the copy and when you write it, the original object is discarded. After writing you can free your modified copy.
This is for creating profiles, which is not a resource or time critical operation.

Please consider following scenario: You read AtoB0 and got a pointer. Then you modify the contents of this pointer and write it to AtoB1. Now you want to create AtoB2, so you read again AtoB0,,, but wait, you don't get what is in the profile but AtoB1, since you modified the pointer.... Is that what you want? Otherwise, you would need to read and parse the structure from disk each time. Wouldn't be better to read it just once and do caching? What if the profile is reused thousands of times in color transforms?

>Does this also mean that if I update the sample data in a CLUT that I need to copy the pipeline first ?

The procedure is always same. You read the pipeline with cmsReadTag(), what you get should be treated as const. If you want to change a tag, make a copy of the original, modify the copy and then write the copy over the original.

>That might indeed be a problem: the update will overwrite the link, and release the memory that was assigned to the tag,
> but the original tag still has the original data pointers, which the code might try to copy on save (or something like that).

I've tried and it works ok. When you overwrite a linked tag, it gets just "unlinked" and the original is preserved.

Let's go for an example, AtoB0 is the original and AtoB1 points to the same space that AtoB0

We  have two possibilities

a)      You write AtoB0: Then you replace the original. AtoB1 will still point to AtoB0, and you got both tags changed

b)      You write AtoB1: Then you replace the link, AtoB0 will keep its original data and a brand new AtoB1 will be created. It no longer is linked to anything.


You can use cmsIsLinkedTag to know which one is the original and which one is linked.

Regards
Marti


From: Frank Vyncke [mailto:fra...@yi...]
Sent: martes, 31 de julio de 2012 13:30
To: Maria, Marti
Cc: lcm...@li...
Subject: Re: [Lcms-user] profile corrupt after update


On 31 Jul 2012, at 13:01, Maria, Marti wrote:



Hi Frank,

After taking a look, it seems to me it is not a bug on the lcms library but in your code. If you examine the documentation, in the API reference of "cmsReadTag", is stated that:

"The memory belongs to the profile and is set free on closing the profile."

This means: anything coming from cmsReadTag should be treated as const. Otherwise, you are modifying structures that are owned by the profile, when the profile is set free, it tries to free those structures. If you have modified the internal pointers, it can get corrupted.

The obvious solution is to dup' the pipeline after reading it. This guarantees the copy of pipeline is not owned by the profile but by you, and you are free to modify anything because nobody else keeps pointers to it.

You code does something like:

Pipeline = cmsReadTag().
..Modify Pipeline..
cmsWriteTag(pipeline)
cmsPipelineFree (pipeline)

And should do:

Pipeline = cmsReadTag().
Newpipeline = cmsPipelineDup(pipeline)
..Modify newpipeline..
cmsWriteTag(newpipeline)
cmsPipelineFree(newpipeline)

Does it make sense?

Yes and No.
I do not really update the pipeline. I create a new pipeline, and insert there the modified structure, and other data (like CurveSet) are indeed extracted from the read pipeline and inserted in the new one. And then I overwrite the tag, and at that point lcms releases the original data.
So, I do not update the read tag...
but in a way I violate the lcms ownership rules because I extract the CurveSets from the original tag.

However, if that would be the problem, then nothing would work because I do this all the time for non-linked tags.

Making the copy is very counter productive, since updating a tag would mean in that way that all data is at least copied 3 times:
- from read tag to duplicated tag
- from duplicated tag to modified tag
- from modified tag to written tag (because the write will copy again)

Does this also mean that if I update the sample data in a CLUT that I need to copy the pipeline first ?


For linked tags, think the linked tag is just an alias that points to a real tag. If you modify the real tag, the alias would point to the modified tag as well, if you modify the alias, then it will be no longer an alias but a real tag. You can check whatever a tag is an alias by using the function cmsIsLinkedTag(). The syntax is cmsIsLinkedTag(<alias>) --> real tag.

That might indeed be a problem: the update will overwrite the link, and release the memory that was assigned to the tag, but the original tag still has the original data pointers, which the code might try to copy on save (or something like that).
I will check what happens if I try to update the 'real' tag in stead of the linked tag (that is not hard to change).

But note though, that for as far as I can see there is a serious difference between a linked tag that is 'created' and a linked that is loaded (because in the loaded case, the pointers etc are filled in)

Frank


I've added a test in the testbed to double check this case.

Best regards
Marti




From: Frank Vyncke [mailto:fra...@yi...]<mailto:[mailto:fra...@yi...]>
Sent: lunes, 30 de julio de 2012 16:32
To: Marti Maria
Cc: lcm...@li...<mailto:lcm...@li...>
Subject: Re: [Lcms-user] profile corrupt after update

Marti,

I extracted the code from our library, and removed nearly all calls besides the ones to lcms2
What the code does now (because most of the useful code is gone), is to
- open the profile
- load the tag
- 'resize' the CLUT in the pipeline to 3 dimensions, 33 samples per dimension, 3 output channels
- save this tag again in the profile
- save the profile

I did not really try to compile this code, but these are indeed all the calls made to lcms2

-------------- begin snippet ------------------------
                                FILE * loadFrom = gmg::fopen(_path, "rb");
                                cmsProfile profile = cmsOpenProfileFromStream(loadFrom, "r");

                                cmsPipeline * pipeline = reinterpret_cast< cmsPipeline * >(cmsReadTag(profile, cmsSigAToB1Tag));
                                if (pipeline == NULL)
                                {
                                                return false;
                                }

                               for (cmsStage* stage = cmsPipelineGetPtrToFirstStage(pipeline); stage != NULL; stage = cmsPipelineGetPtrToFirstStage(pipeline))
                                {
                                                cmsStageSignature stageSignature = cmsStageType(stage);
                                                if (stageSignature == cmsSigCLutElemType)
                                                {
                                                                // delete stage from pipe line
                                                                cmsPipelineUnlinkStage(_pipeline, cmsAT_BEGIN, NULL);

                                                                // and create the one with the size we want
                                                                std::vector< cmsUInt32Number > lcmsGranularity(3, 33);

                                                                stage = cmsStageAllocCLut16bitGranular(contextID, &lcmsGranularity[0], granularity.size(), 3, NULL);
                                                }
                                                else
                                                {
                                                                // extract stage from pipe line
                                                                cmsPipelineUnlinkStage(_pipeline, cmsAT_BEGIN, &stage);
                                                }

                                                cmsPipelineInsertStage(updatedPipeline, cmsAT_END, stage);
                                }

                                cmsWriteTag(_profile.GetLCMS2Profile(), lcms2::ProfileEditingSignatureToLCMSSignature(_signature), sizedpipeline);
                                // (over)writing the tag will also free the 'old' pipe line, so we do not have to free the old
                                // pipeline. But .... writing the tag will also store only a copy of our new pipeline, so we
                                // have to free our pipeline
                                cmsPipelineFree(sizedpipeline);

                                FILE * saveTo = gmg::fopen(savePath, "wb");
                                bool success = cmsSaveProfileToStream(cmsProfile, saveTo) == TRUE;
                                cmsCloseProfile(cmsProfile);

-------------- end snippet ------------------------

After running this kind of code on a profile with linked tags, the profile is corrupt after write

Regards,
Frank
On 30 Jul 2012, at 16:14, Marti Maria wrote:


Hi Frank,

Do you mean you are trying to edit a profile which already have linked tags,
and then you try to rewrite one of the copies?

It may be a bug... Do you have code that illustrates the issue?
Thanks,
Marti


-----Original Message-----
From: Frank Vyncke [mailto:fra...@yi...]<mailto:[mailto:fra...@yi...]>
Sent: lunes, 30 de julio de 2012 14:50
To: lcm...@li...<mailto:lcm...@li...>
Subject: [Lcms-user] profile corrupt after update

Hi,

I am using lcms2 to update profile (pipe-lines) In general this works fine,
but the profiles become corrupt when they have linked tags.

steps:
I open a profile that has linked tags for A2B1 and A2B2.
I see in the internal structures that all data is correctly read, both tags
shared the same pointer, offset etc, and A2B2 is marked as a link to A2B1 I
then create a new pipeline, and call writetag on it to store in in A2B1
(this appears the be the only way to update the CLUT data in a pipeline)
however, when I follow the code in writetag, then I do not see any change to
the A2B2 data. e.g. the offset and pointer are not reset, neither are they
linked to the new data. I am not sure what the correct behavior would be,
but something is wrong here.
I can also not 'relink' with linktag because that one complains about the
fact that the A2B2 tag already exists...

Any idea?

Frank
----------------------------------------------------------------------------
--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and threat
landscape has changed and how IT managers can respond. Discussions will
include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Lcms-user mailing list
Lcm...@li...<mailto:Lcm...@li...>
https://lists.sourceforge.net/lists/listinfo/lcms-user