On 2010.06.28 02:37, Peter Stuge wrote:
>> Is there a rule that forbids Windows application running in a POSIX
>> environment to use native Windows threading?
> I don't understand the use case that you see here?
The use case is that we are not using cygwin's (POSIX?) poll, so that
all Windows platforms behave in the same fashion, by using our poll
emulation. I think it should be the same with regards to threading.
The main goal of cygwin is to easily port Unix/Posix application to
Windows without having to do stuff like poll or thread emulation, by
providing some of the UNIX calls (through cygwin1.dll). But when you
have a native emulation you can use, and your goal is to produce
binaries that are the best fit for the target platform, then the more
native calls, the better. Yes, our emulation code is not as tried as
tested as the cygwin one. Is that justification enough not to do what is
right and use as many Windows native calls as possible? I don't think so.
> If someone is using Cygwin then it is because they want a POSIX API.
Nope, it's because they don't want to be bothered having to emulate the
a *foreign* API with native calls. Whole different reason.
If someone really wants the POSIX API, they'll run on a native POSIX system.
Given the choice, how many people developing software for Windows do you
know will think something like "I want to develop a *brand new* Windows
application, but I'd rather use POSIX instead of the native Windows
APIs. Eventhough it's an application I'm developing from scratch, I'm
gonna use cygwin"?
> Cross-platform is never about allowing all possible permutations,
> it's about finding as few permutations as possible that cover as
> much ground as possible.
Which would be fine if we didn't already have our own emulation code.
But we have it, ready to be used.
>> but I haven't seen anybody complaining about the Windows backend
>> not using cygwin's poll when buidling for cygwin.
> Isn't that what poll_posix.h is for? I actually thought this was how
> it worked?
Not at all. poll_posix is not used for cygwin, precisely because we want
to see some uniform default behaviour for all Windows libusb users.
>> In fact, if we really want to avoid surprises between compilation
> That is not my motivation. I think it's important to have as few
> awkward solutions as possible in every environment. To be like the
> majority of other programs, so as to fit in, and to not throw
> unfamiliar errors.
Yes, you are very narrow minded in your approach to software. If it
doesn't look good to you in the code or in the version control tree,
even if it's the best for the end users and will avoid problems down the
line, it should be thrown away. That's also why you were reluctant to
introducing the calling convention in the first instance, despite the
fact that it was obvious it would be needed eventually, because it meant
our precious source wouldn't look as good.
I don't think that's a healthy approach at all. We want software that
works, that behaves in a predictable fashion, and that is efficient. If
we have to throw that away simply because it will make our code look bad
in the eyes of a minority of people, I'd say that's a wrong motivation.
Using native threading on cygwin, if we can do it, has nothing awkward.
If anything, because the use of native threading is tailored to our
needs, whereas the pthread emulation is meant to be as generic as
possible, we should see some performance gain.
>> we should attempt to make native threading default on cygwin,
> No. Cygwin has better pthread-on-win32 code than we do, so the best
> we can do for now is to use that when available.
As I pointed out, not a valid reason. "Minix/HURD has better kernel code
than we do right now, so what's the point of continuing the development
of this new 'Linux' kernel?"
And if I were to follow your hypocritical reasoning to its logical end,
since cygwin uses pthread-win32, and you just stated that pthread-win32
is better code, why did we move away from pthread-win32 in the first
place? There's actually *no* logical reason for us not to use POSIX
threading for all Windows platforms for all the reasons you exposed above.
As far as I am concerned, you were the one pressuring to move away from
pthread-win32 to use native threading (and delayed integration effort as
a result of wanting to have native threading first), while I was
perfectly fine with keeping pthread-win32. But now that I am advocating
the use of native threading, this is wrong?
> I would like for that code (native threading) to be used only when neccessary.
That's the first reasonable statement I have seen from you so far.
That's what you should have started with instead of the above, because
then I know where you actually come from. I'm disagreeing with that of
course, because I believe we should instead concentrate on making sure
that the default Windows libusb binaries work in the same fashion
wherever possible, but now I understand your position.
> I think that in the future, libusb-1 (maybe -2 by then) will be
> designed to fit Windows better, threads and otherwise. The current
> solution is a bit of a bolt-on, but it works.
Agreed. We have had to make *compromises* to get a very POSIX library to
work on a platform that is quite remote from POSIX, and there's no
escaping this until we review the whole library.
>>> 1. have one target in the build files for each variant of pthreads-win32
>> Which means we would be implying that, since we provide build files for
>> each variant, we have actually tested each variant.
> I disagree strongly that there would be any such implication.
> Instead, what it would communicate to me is something like this:
> "We know that pthread-win32 comes in many flavors, so we created
> these options for you in the hope that they will be useful. We can
> not test everything and would appreciate your help if you encounter
> some problems."
> Please read COPYING, which states basically the same thing. Any and
> all responsibility when using libusb will always lie with the user.
Yeah, and everybody reads the docs in full rather than, in the case of
different project files for each pthread-win32 DLL, simply assume that
if the files are provided, there is a reasonable expectation that they
> I'm sure you agree that all open source software is "at own risk".
Yup. But that doesn't mean open source software should throw explicit
options that haven't been properly tested.
There's two interpretation to "at your won risk":
1. We've done stuff, but we have absolutely no clue whether it works =>
at our own risk
2. We've done stuff and we've checked it to the best of our abilities,
but there's only so much we can do => any scenario or bug we failed to
identify is at your own risk
> That said, of course it's nice if we indicate somewhere that one
> configuration is prefered over others. The wiki or README.WINDOWS
> (maybe link to file from wiki?) might be good for that?
Perhaps you read an open source software wiki diligently every time you
compile a new OSS library. I don't. The less documentation your users
need to read before they can start using your product, the better.
Instead I'm gonna try to compile the library, read about the API calls
and use them in the way I want to use them, and only if I have an issue,
revert back to the docs. Sadly, I don't believe I am the only one
following that approach. If there's something people using feature X
should know, better make it obvious so that they can't use feature X
unless they've been made aware of it.
If I were to push for an approach, I'd say have a warning on the
--withe-pthread-win32 from configure stating that it was only ever
tested with GC2 (not GCE2), and I'd follow Michael's advice of not
having any working default set in MS.
>>> I keep thinking that maybe we can provide a tool (.cmd or .exe) for
>>> automatically configuring the sources before MS toolchain builds.
>> Yes, that would be nice. But there are plenty of things we can look
>> at *once* the integration process is finally over.
> Since integration will continue to require development it might make
> sense to deal with it sooner rather than later.
Sooner meaning during the integration process?
All I'm advocating right now is, let's not do unnecessary stuff that
will stall the integration process further.
> But anyway, I don't see the MS build methods as high priority for release
Which is what I was advocating when I started the Windows backend,
before finding that other people had quite different views about that.
> I could definately
> imagine a release also before all build file changes have been
> included in libusb.git.
Which doesn't mean we should delay the second part of the backend source
integration with optional features, unless we have a very good reason to
do so (or some ulterior motive to want to delay that integration for as
long as possible).
>> Do you see us providing binaries with and without pthread-win32 in
>> that case?
> The thought certainly occured to me. It's not going to be a lot of
> extra work since multiple source builds should be included in each
> binary pack anyway. (32/64, with/out pthread-win32 so far)
Yeah, it's never extra work, when someone else has to do it ;)
>> unless we do MS later, but if we do it later, the we only have a
>> solution for MinGW at the end of this exercise, which doesn't seem
>> quite right to me.
> I think it's right since MinGW uses the existing build system, so
> requires fewer changes, and since it's also sufficient to
> cross-compile binaries for binary release.
And there you go compromising again on your "do it right or don't
bother" statement. Not doing MS along with MinGW is *not* "doing it
> So I'm again of the opposite opinion. I'd be fine with a MinGW-only
> release when that is working well, and then work on getting the other
> MS build methods to work just as well.
> I think I explained this reasoning already a good while ago.
So why aren't you fine with a "no optional pthread-win32" release that
is working well, and then work on getting optional pthread-win32 in? Or
does that only work when you want it to?
>> In all, this is what I saw for the optional use of pthread-win32:
>> - patches needed to be redone: 3 (25, 28, 33)
> What commit ids in your repo would that be? Those numbers aren't
> very precise.
The repo I pushed goes up to 30 (no 33 in yet), and most of the commit
messages are actually just the patch numbers (the others being easily
If you actually look at my repo, you will know. Otherwise, you can just
look at the e-mail threads and find the most recent patch, since each
thread has the patch number.
Since none of these patches have been officially applied, giving commit
IDs would be pointless when the commit messages will do just fine.
>>> smaller pieces are easier to do right.
>> Then the smallest piece is really the temporary removal of
> Agree, but the smallest possible scope is not always right either.
> There's no simple rule for that scope consideration, it's rather
Ah, subjective. The refuge of people who can no longer validate their
position logically. Sorry, but I am indeed calling bullshit on that.
Otherwise, I'm going to have a very hard time trying to figure out what
you "subjectively" want from the integration.
> It's a fact that there have not been enough resources in the libusb
> project to support your work with feedback, and it's also a fact that
> you did not seem prepared and willing to wait for feedback at the
> rate by which it would come naturally.
Well, yeah, because all of the other OSS projects I'm seeing have a very
active integration process for patches.
When somebody submit patches (which I repeatedly did), they usually get
feedback on them within a week and then the project maintainers tell
them if the patches are approved or rejected. So my bad, I should have
known that libusb didn't have anything close to an active patch
> Your very first patches needed heavy rework, I tried to do that
> quickly and provide detailed feedback.
I think that was the case, but I did process your input and applied your
recommendation. So where exactly is your input on the second set of
patches I submitted on 2010.01.31? I've checked the e-mail threads, and
I haven't seen any input from you on any of these 7 patches I submitted.
Surely, if you're talking about the first set of patches being bad, then
the second time I submit patches, I would have expected some explicit
follow up. And as far as I recall, the issues you mentioned in the first
set of patch had been taken into account for the second batch.
> The next patches didn't seem
> to take into account that feedback to the degree that I had hoped for
> and expected, and at that point I felt I needed to make a choice.
> 1. halt the Windows effort there and then, to get everything closer
> to perfect the first time around, or
> 2. set up a repo where work could continue independently, knowing
> that this would require much fixup before commit to libusb.git
Well, obviously, if you provide neither feedback, nor explain your
actions, we're going to be facing a headache down the line. What was
wrong really about pointing out the problems in the second set of
patches? Or was your plan to keep people going doing some hard work
knowing that you could just exert your executive authority down the line
and pick whatever you liked ("You spent time on HID, DDK builds,
autoclaim, etc? I don't think we'll need that - Please throw it away")
Poor management choice if I ever saw one.
> As frustrating as the current situation is, at least we have working
> code that we can touch up, and get into libusb.git. I knew how
> frustrated I would have been with lack of feedback on my work, so I
> wanted to make sure that you at least had the option to work
> independently if you wanted to.
I'll only be working independently if I think there is good chance that
what I am working on will make it into official. I'm not exploring stuff
for the sake of producing experimental code. If I don't get feedback,
but I see other people actually starting to use that code because
integration is slow and people seem to be satisfied with that code, I
can only assume that it will integrated.
>> Integrating things gradually is usually a good way to identify and
>> sort problems as they occur. It would have been so nice if we had
>> had integration when we started dealing with MinGW and cygwin, 6
>> months ago, because then I would have known that not everybody was
>> happy about removing -Wshadow, and I would have addressed that
> On the other hand there wasn't a lot of discussion before the facts.
Yeah, because that's what timely patch reviews are for in projects that
follow the "Release early, release often" motto. Avoids having to send
e-mails about every single feature you introduce, which is a good thing
as people here have already expressed their annoyance about the volume
of e-mails related to the Windows backend.
>> So, taking the line that getting everything right at once is better
>> than the alternative is not something I can see myself agreeing
> Let's do some of the things right enough. Nice and vague, huh? :)
You're voluntarily distorting what I am saying into the exact opposite
(as well as clearly exposing what you would like me to be saying to
prove a point that you cannot make).
There's nothing vague about removing the LIBUSB_USE_PTHREAD variable for
the time being, and if you want to pretend that I ever advocated doing
things "nice and vague", then please point to a specific example.
You know, I'm getting a bit tired of some of the baseless accusations
you have been making against my work (xusb not being designed for cross
platform, supposedly stubborn refusal to patch dpfp, my way of
addressing issues being awkward or thinly veiled allegations that the
Windows backend code is not up to good standards ("There's enough bad
code in the world as it is" or something)), which, whenever I ask you to
point to a specific example where I am actually doing something that you
think is wrong, seem to fall flat.
If you don't like my approach, fine. But if you want to prove that it's
wrong, better be prepared to back it up with hard facts.
Believe it or not, I too am fussy and uncompromising when it comes to
producing quality software. I want my software to work, be the best and
the most intuitive (read, not requiring to read more docs than necessary
before usage), and I usually think long and hard about the "right"
course of action to achieve these goals. That's why sometimes I'm
pushing for libtool patches to be integrated. That's why sometimes I'm
going to add experimental #ifdef features, when I can't resolve what
will be the best option for our users on my own. And that's why
sometimes I will also either remove or be reluctant to introduce
features that I see as liabilities, like optional pthread-win32 support,
until I have had a chance to look at them thoroughly and ensure they
meet my quality standards.
Unfortunately, this means being quite consistent with your approach, and
not just doing things when you feel like doing them (let's introduce
optional pthread-win32 for MinGW but not for MS), and if that sounds
like lacking flexibility, so be it, but there's nothing "nice and vague"
about that. I believe it's still better than showing clear
inconsistencies in only picking the things you like, without being able
to back them up logically.