#91 Trouble with getpwent()

[Orion Poplawski]

Running repquota -a on a system using sssd, users not in /etc/passwd will not be resolved by default. This is because sssd will not enumerate a full list of users by default (as with very large user databases this may be a very expensive operation) and getpwent() will not list them. Better would be to lookup user ids with getpwuid(). Same issue with groups.

[Jan Kara]

Yes, I'm aware of this. The problem with getpwuid() is that you have to get the uid from somewhere. And doing for (i = 0; i < 0xffffffff; i++) { user = getpwuid(i); ... } won't quite cut it either... Really the right solution for this is to provide a kernel interface to dump all known quota structures to userspace.

And yes, I'm guilty for not implementing such an interface for quite a long time :(.

[Orion Poplawski]

Perhaps I'm not reading things correctly, but in repquota.c dump_cached_dquots(), it looks like you already have all of the quota entries in dquot_cache[]. Couldn't you just loop through that (as done at the end to pick up unknown users/groups) and do a getpwuid(dquot_cache[i].dq_id) for each entry? It changes the sort order of the output from /etc/passwd order to dquot_cache order, but I imagine you could even sort that array on uid if desired.

[Jan Kara]

Now I see what you are pointing at. OK, so that particular code could be changed to use getpwuid(). It's just that with plain old /etc/passwd what we do is faster (that's why I ended up implementing cached uid->name translation after all). Also this code is not executed for every filesystem. XFS, ocfs2, or ext4 (just recently) do not have quota files available to userspace applications and thus repquota has to be implemented by iterating over all users and querying kernel for quota information. So for those filesystems we have to use getpwent().

[Brian Hoy]

I've encountered this problem too as we are using winbind (Samba AD integration) and getpwent and getgrent do not play nicely with winbind. This is well documented due to the incompatible APIs so it is best to avoid these calls if possible.

I've implemented this little patch in the repquota code that solves the problem for us. If you think is it ok, we'd appreciate it being added to the code. Many thanks!

[root@zzsv01:~] diff -u quota-4.03/repquota.c quota-4.03-PATCHED/repquota.c
--- quota-4.03/repquota.c   2014-12-02 22:39:52.000000000 +1300
+++ quota-4.03-PATCHED/repquota.c   2015-12-21 15:24:48.390344195 +1300
@@ -280,28 +280,22 @@
    if (type == USRQUOTA) {
        struct passwd *pwent;

-       setpwent();
-       while ((pwent = getpwent())) {
-           for (i = 0; i < cached_dquots && pwent->pw_uid != dquot_cache[i].dq_id; i++);
-           if (i < cached_dquots && !(dquot_cache[i].dq_flags & DQ_PRINTED)) {
+       for (i = 0; i < cached_dquots; i++) {
+           if ((pwent = getpwuid(dquot_cache[i].dq_id)) != NULL) {
                print(dquot_cache+i, pwent->pw_name);
                dquot_cache[i].dq_flags |= DQ_PRINTED;
            }
        }
-       endpwent();
    }
    else {
        struct group *grent;

-       setgrent();
-       while ((grent = getgrent())) {
-           for (i = 0; i < cached_dquots && grent->gr_gid != dquot_cache[i].dq_id; i++);
-           if (i < cached_dquots && !(dquot_cache[i].dq_flags & DQ_PRINTED)) {
+       for (i = 0; i < cached_dquots; i++) {
+           if ((grent = getgrgid(dquot_cache[i].dq_id)) != NULL) {
                print(dquot_cache+i, grent->gr_name);
                dquot_cache[i].dq_flags |= DQ_PRINTED;
            }
        }
-       endgrent();
    }
    for (i = 0; i < cached_dquots; i++)
        if (!(dquot_cache[i].dq_flags & DQ_PRINTED)) {

[Jan Kara]

Thanks for the patch. This is actually a different problem than in the original report - that was about issues with enumerating all users when doing repquota.

You can achieve the same behavior as in your patch using -C (--no-cache) option. Also recent quota-tools have autodetection of used name resolver by parsing /etc/nsswitch.conf and they switch cache off if they find resolver where using setpwent() is inefficient. If you attach here your /etc/nsswitch.conf, I can have a look whether the heuristic there doesn't need updating...

[Nobody/Anonymous]

Simply want to say your article is as astonishing. The clarity in your post is simply cool and i can assume youre an expert on this subject. Well with your permission allow me to grab your RSS feed to keep up to date with forthcoming post. Thanks a million and please continue the enjoyable work.
<a href="http://high-quality-katy-perry-dresses86.onsugar.com/" title="For the greatest">For the greatest</a>

[Nobody/Anonymous]

How have you ever help make your blog look this cool. Email me in case you possibly can and reveal your intelligence. Id end up being appreciative!
<a href="http://games.darbhanga.net/index.php?params=profile/view/9132/" title="Beverage">Beverage</a>

[Nobody/Anonymous]

Thank you for provide good information about this, this content must be write by expert
Wedding Planner http://bharatjanani.com/activity-2/p/9975/

[Nobody/Anonymous]

Hey there! Im in the office browsing your website from my own new iphone! Just desired to say I really like reading your website and look forward to all your posts! Carry about the outstanding perform!
Unlimited Options http://facebooks.lv/index.php?do=/blog/353606/dresses-unlimited-options/

[Nobody/Anonymous]

Your use of such emphatic words in your article shows an intense excitement in what you are trying to convey. Keep up the good work!
Old-fashioned http://pagkakaibigan.com/index.php?do=/blog/80538/towle-beaded-vintage-flatware-duvet-bedding/

[Nobody/Anonymous]

Please let me know if youre looking for a article writer for your blog. You have some really great articles and I feel I would be a good asset. If you ever want to take some of the load off, Id love to write some content for your blog in exchange for a link back to mine. Please shoot me an e-mail if interested. Thanks!
Protecting against http://thangbinhcharity.com/index.php?do=/blog/7732/methods-for-protecting-against-accidental-injury-to-university-student-note/

[Nobody/Anonymous]

I am impressed with this web site, really I am a fan.
Mobile devices For http://www.pktalent.com/activity/p/23419/

[JM]

I'm running into the same problem, repquota with sssd shows only the user ids instead of the usernames, is there a chance to fix this problem?

[Petr Písař]

This looks like a duplicate of https://sourceforge.net/p/linuxquota/bugs/43/ that was addresed with two upstream commits (Scan dquots using Q_GETNEXTQUOTA, Add support for scanning using Q_XGETNEXTQUOTA) recently.

[Jan Kara]

Correct. Also it should be noted that an appropriate kernel support is necessary - should be present in kernel 4.6.