Re: [Phpwebsite-developers] help with time-out on a custom script

[Verdon V. <ve...@ve...>]

Hi Matt,

I finally had a chance to get back to this, and have made a little  
progress but am still stumbling. I appreciate your previous pointers  
and would welcome any thoughts from yourself or others to the  
comments I have added below.

Thanks and best regards,
verdon

On 5-Feb-07, at 8:45 AM, Matthew McNaney wrote:

> Verdon,
>
> One method would be to to limit the select results.
>
> For example, 1.0's conversion script pulls data in batches,  
> converts the
> batch, then loads the next page. The batch number is incremented and
> continues until out of results.
I've looked at Batch.php and it looks pretty intriguing, but if I  
understand correctly, it requires user input to click on the Continue  
link between batches? Unfortunately, I need to run this script via cron.

> A better solution my be to break from the API and use a select join:
>
> SELECT mod_users.user_id
> FROM   mod_users, mod_cstmembers_members
> WHERE  expiryDate < $today
>        AND
>        mod_cstmembers_members.owner = mod_users.user_id
I started doing something along these lines and have gained some  
improvement. Not enough yet though.

This is what I'm doing now to get my array of users to process. I'm  
still selecting a little more than I need, but it is helping me  
visualize...

$sql = 'select mem.id as mem_id, usr.user_id, usr.username,  
usr.groups from en_mod_cstmembers_members as mem, en_mod_users as usr  
where mem.expiryDate < "' . $today . '" and mem.owner = usr.username';
$expiredMembers = $GLOBALS["core"]->getAllAssoc($sql);

>
> I would use getCol for the results.
> Then I would manually delete on the results,
>
> foreach ($results as $user_id) {
> $GLOBALS['core']->query("DELETE mod_groups WHERE user_id=$user_id and
> group_id=$group_id");
> }
>
I think if this was all I had to do, I could probably manage it.  
Unfortunately, what I need to do once I've identified all the users  
in question is two things, and this I don't have the skills to do  
without using the api and loading group/user class for each result.  
This is where my bottleneck is. What I need to do to each user is...

a) check to see if they are a member of a couple groups and if they  
are, then remove those group_ids from the groups string in the user  
record
b) if they were in those groups, then also remove their user_id from  
the members string in the group record

This is the sort of thing I was trying and is timing out

foreach ($expiredMembers as $member){

     $groups = explode(':', $member['groups']);

     foreach ($groups as $group) {

         if ($group == '22' || $group == '1') {
            PHPWS_User_Groups::removeGroupFromUser($member 
['user_id'], $group);
         }

     }
PHPWS_User_Groups::updateUserGroups($member['user_id']);

}

Now, there are a couple problems with this. First, updateUserGroups()  
didn't quite do what I expected (didn't seem to do anything at all),  
so I may have to write something for that. removeGroupFromUser() did  
behave as expected. The biggest problem though is that I have a  
result set of about 140 users* and the script is timing out after  
processing about 8 or so users.

I've done some googling and looking and php.net. I found a class at  
phpclasses.org that looks like it might do the job, but requires php  
5. I've also considered using ini_set() to extend the max_execution  
value, but I think I need to find a better solution than that.

Ultimately, I think I need to break out of using the  
PHPWS_User_Groups class and write my own functions for removing a  
group from a user's groups and a users from a group's members. It's  
just a little over my head ;-)


* even though I need to run this cron nightly and there are appx 140  
users that match at this time, most nights there will be none or only  
a few, but once a year, there will be a large batch of users  
(300-400), so the script needs to be throttled/batched in some way.


> I haven't tried it personally, but I believe mysql 4.0 allows delete
> joins so you could skip the select part.
>
> This is all from the top of my head so testing might be necessary :)
>
> Good luck,
> Matt
>
> On Sun, 2007-02-04 at 16:31 -0500, Verdon Vaillancourt wrote:
>> Hi,
>>
>> I'm trying to cobble together something kind of customized to do some
>> maintenance on a phpws 0.10.2 site and am running into timeout
>> errors. I'm hoping someone might be able to suggest an alternative
>> method for me.
>>
>> The error is
>> PHP Fatal error:  Maximum execution time of 30 seconds exceeded in /
>> home/user/public_html/dev/en/lib/pear/DB/mysql.php on line 316
>>
>> The goal of the script is that I have a table with member profiles
>> that include an expiration date. I need a script that can look at
>> that table, find the expired users, then look up the corresponding
>> user and remove them from a couple groups. Eventually, this script
>> will get run via cron. I am just getting started...
>>
>> <?php
>>
>> require_once("/home/user/public_html/dev/en/conf/config.php");
>> define('PHPWS_SOURCE_DIR', $source_dir);
>> require_once PHPWS_SOURCE_DIR . 'security.php';
>> require_once PHPWS_SOURCE_DIR . 'core/Core.php';
>> require_once PHPWS_SOURCE_DIR . 'mod/users/class/Users.php';
>> require_once PHPWS_SOURCE_DIR . 'mod/users/class/Groups.php';
>>
>> $GLOBALS['core'] =& new PHPWS_Core(NULL, NULL);
>>
>> $today = date("Y-m-d H:i:s", mktime(0, 0, 0, date("m"), date("d"),
>> date("Y")));
>>
>> /* first get the array of expired members */
>> $expiredMembers = $GLOBALS["core"]->sqlSelect
>> ("mod_cstmembers_members", "expiryDate", $today, NULL, "<", NULL,
>> NULL, NULL, NULL, TRUE);
>
>> /* then loop through them */
>> foreach ($expiredMembers as $member){
>>      /* get the user_id */
>>      $user_id = PHPWS_User::getUserId($member['owner']);
>>      if ($user_id) {
>>          /* remove them from the groups */
>>          $group_id = 1;
>>          PHPWS_User_Groups::removeGroupFromUser($user_id, $group_id);
>>      }
>> }
>>
>> ?>
>>
>> Thanks,
>> verdon
>
> -- 
> Matthew McNaney
> Electronic Student Services
> Appalachian State University
> http://phpwebsite.appstate.edu
>
>
> ---------------------------------------------------------------------- 
> ---
> Using Tomcat but need to do more? Need to support web services,  
> security?
> Get stuff done quickly with pre-integrated technology to make your  
> job easier.
> Download IBM WebSphere Application Server v.1.0.1 based on Apache  
> Geronimo
> http://sel.as-us.falkag.net/sel? 
> cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Phpwebsite-developers mailing list
> Php...@li...
> https://lists.sourceforge.net/lists/listinfo/phpwebsite-developers