[Openinteract-help] SPOPS::LDAP implementation isn't robust

[Andrew S. H. <ste...@ci...>]

I'll admit right off the bat that I haven't checked the archives or 
looked much deeper than a couple of the docs, so this suggestion might 
be answered elsewhere.  I also don't know if this is most appropriate 
for openinteract-dev or openinteract-help, but since I'm not a 
developer, help seemed a little safer. Please shoot me down and redirect 
me if I'm in the wrong place or it's a solved problem. Thanks.

I just wanted to make a feature request that a future version of SPOPS 
include more robust multi-record querying, or (at least) an easily 
overridden method to handle the queries.  The problem I'm running into 
is that ActiveDirectory limits the query size limit to something 
insanely low (under 1,000, if memory serves).  It just croaks if you try 
to grab all of them at once: "Sizelimit exceeded".

The solution, is to request the records in batches of limited size.  
Net::LDAP::Control::Paged contains an example on how to do this. Here is 
the implementation I used for queries prior to switching to SPOPS:

    # This code is mostly ripped off from the Net::LDAP::Control::Paged POD
    my $page = Net::LDAP::Control::Paged->new(size => $page_size);
    my @args = (@_, control => [ $page ]);
    my $cookie;
    while (1) {
        my $mesg = $$self{ldap}->search(@args);
        $mesg->code and die "Error occurred during search: ",$mesg->error;
        my ($resp) = $mesg->control(LDAP_CONTROL_PAGED) or last;
        $cookie = $resp->cookie or last;
        $page->cookie($cookie);
    }

    # check for abnormal exit and cleanup if we have one
    if ($cookie) {
        $page->cookie($cookie);
        $page->size(0);
        $$self{ldap}->search(@args);

        return '';
    }

After a quick look through of SPOPS::LDAP, it should be easy enough to 
adapt SPOPS::LDAP::_execute_multiple_record_query to use paged rather 
than plain queries.  In fact, this is likely what I'll do as a stop 
gap---i.e., just redefine this method in my custom extensions.  However, 
that's digging inside of your code and will most certainly break some 
day when you guys change the internals.

So, my request is either to incorporate paging in the searches, or add 
an option:

   paged_queries => 'yes',
   paged_query_size => 500,

or something to that effect. OR, provide a standard place to override 
the query mechanism to replace it with a custom implementation in a 
derived class (or whatever).

SPOPS is the niftiest POOP I've used thus far, so keep up the good work.

Regards,
Sterling