Re: [robinhood-devel] Lustre UUIDs and Robinhood

[Faccini, B. <bru...@in...>]

> Le Mar 31, 2016 à 13:46, LEIBOVICI Thomas <tho...@ce...> a écrit :
> 
> 
> On 03/31/16 10:49, Faccini, Bruno wrote:
>> Well I am a bit late to answer in this thread, but yes, as Thomas has indicated, you may use almost the same set/scheme of changes that I had used to integrate archive_id, my related patch can be found at https://github.com/cea-hpc/robinhood/commit/1a3de2759ffcf980685724d41eaf98207dfe3522.
> Thank you Bruno.
> This depends on the targeted version: rbh 2.5.x or 3.0.
> This patch is somehow specific to v2.5.x. I believe this much simpler to add such a field now in v3.

Sure Thomas!
But your answer covered 3.0 when my patch may help with 2.5, BTW master branch is still for 2.5, right? ;-)

> 
> Thomas.
> 
>> HTH, bye,
>> Bruno.
>> 
>>> Le Mar 31, 2016 à 08:39, LEIBOVICI Thomas <tho...@ce...> a écrit :
>>> 
>>> On 03/29/16 20:38, Frank Zago wrote:
>>>> Hello,
>>>> 
>>>> I'm considering adding a UUID support in Robinhood to support a per file
>>>> UUID in Lustre. There's been a short discussion of a proposed format at
>>>> https://jira.hpdd.intel.com/browse/LU-7207. Cray's format is less
>>>> complex, but in essence is more or less compatible.
>>>> 
>>>> For our copytool, we are storing a UUID in the "trusted" extended
>>>> attribute namespace, since only an admin can see/change them. It's up to
>>>> the copytool to set it and retrieve it, which works fine for archiving,
>>>> restoring and importing.
>>>> 
>>>> But when the file is removed, we need to get that UUID it from somewhere
>>>> else, since the file is gone so is its UUID. I'd like to enhance
>>>> Robinhood to store it. Here's what I think is needed:
>>>> 
>>>>    - The ENTRIES table must have a UUID column.
>>>>    - The SOFTRM table must include the UUID.
>>>>    - Robinhood must look for the UUID extended attribute, if not
>>>>       present, when it gets the HSM archive event. For some reason,
>>>>       Lustre 2.5 to at least 2.9 will not issue a CL_XATTR event for the
>>>>       trusted namespace.
>>>>    - When scanning a file system to rebuild/diff a database, Robinhood
>>>>       must look for the UUID extended attribute.
>>> Hi Frank,
>>> 
>>> This sounds exactly like what is done for storing "archive_id" attribute
>>> in rbh DB.
>>> To manage this new attribute, you just need to complete the "lhsm" module:
>>> - define this attribute in lhsm specific attributes (like archive_id)
>>> - add the right bit in the SOFTRM mask of this module
>>> - do specific actions in the changelog callback about CL_XATTR or CL_HSM
>>> archive.
>>> 
>>>> What to store could be driven by some new configuration format in the
>>>> General section to make things versatile enough:
>>>> 
>>>> uuid {
>>>>    method = "xattr";
>>>>    uuid = "trusted.copytool.uuid";
>>>> }
>>>> 
>>>> or, for LU-7207
>>>> 
>>>> uuid {
>>>>    method = "json";
>>>>    uuid = "replicas[1].id"
>>>> }
>>>> 
>>>> or, also for LU-7207, storing the whole thing:
>>>> 
>>>> uuid {
>>>>    method = "bin";
>>>> }
>>> As it is lustre-HSM specific, such a custom definition should be
>>> included to the configuration of lhsm module ("lhsm_config" block).
>>> 
>>>> I'm only speculating for that LU-7207 format.
>>>> 
>>>> To retrieve the UUID depending on the filename, rbh-find can be
>>>> modified. I was thinking that I could add an option to pass an output
>>>> string formater, similar to printf or strftime, with Robinhood
>>>> specifiers. Same for rbh_undelete, rbh_recov.
>>> Do you mean a specific way of displaying the uuid? Can't it be infered
>>> from the custom uuid definition above?
>>> 
>>> Regards,
>>> Thomas
>>>> Comments?
>>>> 
>>>> Frank.
>>>> 
>>>> ------------------------------------------------------------------------------
>>>> Transform Data into Opportunity.
>>>> Accelerate data analysis in your applications with
>>>> Intel Data Analytics Acceleration Library.
>>>> Click to learn more.
>>>> http://pubads.g.doubleclick.net/gampad/clk?id=278785471&iu=/4140
>>>> _______________________________________________
>>>> robinhood-devel mailing list
>>>> rob...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/robinhood-devel
>>> 
>>> ------------------------------------------------------------------------------
>>> Transform Data into Opportunity.
>>> Accelerate data analysis in your applications with
>>> Intel Data Analytics Acceleration Library.
>>> Click to learn more.
>>> http://pubads.g.doubleclick.net/gampad/clk?id=278785471&iu=/4140
>>> _______________________________________________
>>> robinhood-devel mailing list
>>> rob...@li...
>>> https://lists.sourceforge.net/lists/listinfo/robinhood-devel
>> ---------------------------------------------------------------------
>> Intel Corporation SAS (French simplified joint stock company)
>> Registered headquarters: "Les Montalets"- 2, rue de Paris,
>> 92196 Meudon Cedex, France
>> Registration Number:  302 456 199 R.C.S. NANTERRE
>> Capital: 4,572,000 Euros
>> 
>> This e-mail and any attachments may contain confidential material for
>> the sole use of the intended recipient(s). Any review or distribution
>> by others is strictly prohibited. If you are not the intended
>> recipient, please contact the sender and delete all copies.
> 

---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris, 
92196 Meudon Cedex, France
Registration Number:  302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.