#91 Trouble with getpwent()

None
closed-fixed
Jan Kara
None
5
2016-02-19
2012-10-25
No

Running repquota -a on a system using sssd, users not in /etc/passwd will not be resolved by default. This is because sssd will not enumerate a full list of users by default (as with very large user databases this may be a very expensive operation) and getpwent() will not list them. Better would be to lookup user ids with getpwuid(). Same issue with groups.

Discussion

  • Jan Kara

    Jan Kara - 2012-11-06

    Yes, I'm aware of this. The problem with getpwuid() is that you have to get the uid from somewhere. And doing for (i = 0; i < 0xffffffff; i++) { user = getpwuid(i); ... } won't quite cut it either... Really the right solution for this is to provide a kernel interface to dump all known quota structures to userspace.

    And yes, I'm guilty for not implementing such an interface for quite a long time :(.

     
  • Orion Poplawski

    Orion Poplawski - 2012-11-06

    Perhaps I'm not reading things correctly, but in repquota.c dump_cached_dquots(), it looks like you already have all of the quota entries in dquot_cache[]. Couldn't you just loop through that (as done at the end to pick up unknown users/groups) and do a getpwuid(dquot_cache[i].dq_id) for each entry? It changes the sort order of the output from /etc/passwd order to dquot_cache order, but I imagine you could even sort that array on uid if desired.

     
  • Jan Kara

    Jan Kara - 2012-11-08

    Now I see what you are pointing at. OK, so that particular code could be changed to use getpwuid(). It's just that with plain old /etc/passwd what we do is faster (that's why I ended up implementing cached uid->name translation after all). Also this code is not executed for every filesystem. XFS, ocfs2, or ext4 (just recently) do not have quota files available to userspace applications and thus repquota has to be implemented by iterating over all users and querying kernel for quota information. So for those filesystems we have to use getpwent().

     
    • Brian Hoy

      Brian Hoy - 2015-12-21

      I've encountered this problem too as we are using winbind (Samba AD integration) and getpwent and getgrent do not play nicely with winbind. This is well documented due to the incompatible APIs so it is best to avoid these calls if possible.

      I've implemented this little patch in the repquota code that solves the problem for us. If you think is it ok, we'd appreciate it being added to the code. Many thanks!

      [root@zzsv01:~] diff -u quota-4.03/repquota.c quota-4.03-PATCHED/repquota.c
      --- quota-4.03/repquota.c   2014-12-02 22:39:52.000000000 +1300
      +++ quota-4.03-PATCHED/repquota.c   2015-12-21 15:24:48.390344195 +1300
      @@ -280,28 +280,22 @@
          if (type == USRQUOTA) {
              struct passwd *pwent;
      
      -       setpwent();
      -       while ((pwent = getpwent())) {
      -           for (i = 0; i < cached_dquots && pwent->pw_uid != dquot_cache[i].dq_id; i++);
      -           if (i < cached_dquots && !(dquot_cache[i].dq_flags & DQ_PRINTED)) {
      +       for (i = 0; i < cached_dquots; i++) {
      +           if ((pwent = getpwuid(dquot_cache[i].dq_id)) != NULL) {
                      print(dquot_cache+i, pwent->pw_name);
                      dquot_cache[i].dq_flags |= DQ_PRINTED;
                  }
              }
      -       endpwent();
          }
          else {
              struct group *grent;
      
      -       setgrent();
      -       while ((grent = getgrent())) {
      -           for (i = 0; i < cached_dquots && grent->gr_gid != dquot_cache[i].dq_id; i++);
      -           if (i < cached_dquots && !(dquot_cache[i].dq_flags & DQ_PRINTED)) {
      +       for (i = 0; i < cached_dquots; i++) {
      +           if ((grent = getgrgid(dquot_cache[i].dq_id)) != NULL) {
                      print(dquot_cache+i, grent->gr_name);
                      dquot_cache[i].dq_flags |= DQ_PRINTED;
                  }
              }
      -       endgrent();
          }
          for (i = 0; i < cached_dquots; i++)
              if (!(dquot_cache[i].dq_flags & DQ_PRINTED)) {
      
       
      • Jan Kara

        Jan Kara - 2015-12-23

        Thanks for the patch. This is actually a different problem than in the original report - that was about issues with enumerating all users when doing repquota.

        You can achieve the same behavior as in your patch using -C (--no-cache) option. Also recent quota-tools have autodetection of used name resolver by parsing /etc/nsswitch.conf and they switch cache off if they find resolver where using setpwent() is inefficient. If you attach here your /etc/nsswitch.conf, I can have a look whether the heuristic there doesn't need updating...

         
  • Nobody/Anonymous

    Simply want to say your article is as astonishing. The clarity in your post is simply cool and i can assume youre an expert on this subject. Well with your permission allow me to grab your RSS feed to keep up to date with forthcoming post. Thanks a million and please continue the enjoyable work.
    <a href="http://high-quality-katy-perry-dresses86.onsugar.com/" title="For the greatest">For the greatest</a>

     
  • Nobody/Anonymous

    How have you ever help make your blog look this cool. Email me in case you possibly can and reveal your intelligence. Id end up being appreciative!
    <a href="http://games.darbhanga.net/index.php?params=profile/view/9132/" title="Beverage">Beverage</a>

     
  • Nobody/Anonymous

    Thank you for provide good information about this, this content must be write by expert
    Wedding Planner http://bharatjanani.com/activity-2/p/9975/

     
  • Nobody/Anonymous

    Hey there! Im in the office browsing your website from my own new iphone! Just desired to say I really like reading your website and look forward to all your posts! Carry about the outstanding perform!
    Unlimited Options http://facebooks.lv/index.php?do=/blog/353606/dresses-unlimited-options/

     
  • Nobody/Anonymous

    I am impressed with this web site, really I am a fan.
    Mobile devices For http://www.pktalent.com/activity/p/23419/

     
  • JM

    JM - 2013-01-26

    I'm running into the same problem, repquota with sssd shows only the user ids instead of the usernames, is there a chance to fix this problem?

     
  • Petr Písař

    Petr Písař - 2016-02-10

    This looks like a duplicate of https://sourceforge.net/p/linuxquota/bugs/43/ that was addresed with two upstream commits (Scan dquots using Q_GETNEXTQUOTA, Add support for scanning using Q_XGETNEXTQUOTA) recently.

     
  • Jan Kara

    Jan Kara - 2016-02-19

    Correct. Also it should be noted that an appropriate kernel support is necessary - should be present in kernel 4.6.

     
  • Jan Kara

    Jan Kara - 2016-02-19
    • status: open --> closed-fixed
    • assigned_to: Jan Kara
    • Group: -->
     


Anonymous

Cancel  Add attachments