#3889 xcatd not running preprocess for multiple plugins when mgt=ipmi

2.8.4
closed
None
general
5
2014-04-01
2013-11-11
No

I worked through this issue with John Simpson and his new configfpc code, but it seems the problem may actually be a bug in xcatd.

In configfpc, there are several subreq calls to rspconfig using Utils::runxcmd. The node that is being processed needs to be handled by ipmi.pm. For other reasons, both ipmi.pm and blade.pm have a handled_commands entry for nodes with mgt=ipmi. The preprocess_request in each of these routines calls Utils::filter_nodes to determine the correct plugin to actually handle the request.

HOWEVER, this is the processing that actually occurs:
- configfpc calls rspconfig through runxcmd
- xcatd sees that ipmi.pm needs to be called. since this is a new subreq, it calls ipmi.pm preprocess_request to filter_nodes, which determines that it should handle this node, and ipmi process_request is called
- xcatd then sees that blade.pm needs to be called. Since _xcatpreprocessed was already set by ipmi.pm, xcatd goes straight to process_request in blade.pm, skipping the code in preprocess_request to call filter_nodes which would have filtered out this node, preventing blade.pm from handling it. Instead, blade.pm fails with all kinds of errors running code it should not be executing for this node.

In order to allow John to get his code checked in for the final xCAT 2.8.3 build, he is HACKING his runxcmd call to set "subreq->sequential='1'. This will force xcatd to process the plugins for rspconfig in sequential alphabetical order. This in turn will force blade.pm to run first and skip the node, then have ipmi.pm run to actually handle the function.

I think the correct fix for this is to change xcatd processing for multiple plugins. If the initial request comes in requiring preprocess to run, preprocess should run for ALL of the plugins that are called for that command. I think this change may be important for hierarchy to work correctly as well, since each plugin may have different hierarchy calls that need to be made for the given nodelist.

Er Tao - Bruce said that you put the multiple plugin processing for rspconfig into blade.pm and ipmi.pm. Did you also make the corresponding xcatd changes? If not, this defect may need to get reassigned to someone else.

Discussion

  • zhao er tao

    zhao er tao - 2013-11-12

    For a normal 'rspconfig' command, it will be passed to both ipmi.pm and blade.pm to check which plugin shall handle this command, the checking is based on 'filter_nodes' in preprocess_request subroutine.
    But for 'rspconfig' running through 'runxcmd', xcatd directly pass it to the process_request subroutine in both plugins since '_xcatpreprocessed' is set and there is no such plugin checking as in 'preprocess_request'.
    So my suggest is running 'rspconfig $node ip/netmask/gateway' directly instead of using 'runxcmd' or 'runcmd' to run rspconfig.

     
    Last edit: zhao er tao 2013-11-12
  • Linda Mellor

    Linda Mellor - 2013-11-12

    From our team discussion this morning:

    • one suggestion is, when there are multiple plugins that will be called for one execution of a command, to have xcatd create a separate copy of the request structure for each plugin call so that execution of one plugin does not modify the structure being handled by another plugin. (in this defect case, the _xcatpreprocessed setting was being changed which affected the other plugin calls)

    • need to look at all the plugins handling copycds to make sure we don't break anything there. someone mentioned that there may be a check to see if some other plugin already did a copy.

    • need to make sure if there are other commands that run multiple plugins for the same node/function that whatever solution will not break any existing processing

    • Bruce talked through a possible design of xcatd calling a new function in each plugin (called at the time the command is going to be run) that would not be dependent on _xcatpreprocessed and that would have the plugin return which list of nodes it would actually be able to work on. This could then be used for preprocess to determine hierarchy calls.

     
  • zhao er tao

    zhao er tao - 2013-11-21

    I have fixed this bug with commit hash num "b5409a" for 2.8 stream and "8bc048" for master.
    The reason is the command processed in plugin_command subroutine will be one by one among plugin modules rather than parallel when it is passed to xcat by &do_request subroutine as a subrequest.

     
  • zhao er tao

    zhao er tao - 2013-11-21
    • status: open --> pending
     
  • Linda Mellor

    Linda Mellor - 2014-04-01
    • status: pending --> closed
    • component: --> general
     

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks