dcplusplus-devel — Developer's list

Re: [dcplusplus-devel] LinuxDC++: Fixes a nick encoding issue that could cause transfers to fail

[Jacek S. <arn...@gm...>]

Steven Sheehy wrote:
snip

> The attached patch solves this in the following way. First, the peer
> nick is
> stored on the list of expected connections in ACP, not in uft-8. This is
> not
> a problem, because the stored nick is just a key and is not used for any
> other purpose. Next, when UserConnection::on() receives $MyNick it is
> passed
> further untranslated. Peer's nick gets translated to utf-8 later inside
> ConnectionManager::on(MyNick) when it becomes possible to determine and set
> the correct encoding.
> 
> This all can be seen directly from the patch, of course. Perhaps, a better
> way of doing things could be to separate incomming connection checking from
> ConnectionManager::on(MyNick) and put it in UserConnection::on() or do
> something similar.
> 
I guess this is ok...at some point maybe it would make sense to store the non-utf8 nick somewhere
when it first arrives from the hub and use that always without encoding back and forth to utf-8
since some round-trip conversions may be lossy (or rather, might not yield the same byte
sequence)...expectedConn would probably have to keep a UserPtr reference at that point along with
the expected byte sequence...Then encoding and possibly huburl could go away from userconnection I
think and it would be correct from an encoding-point-of-view, and the cid wouldn't have to be
synthesized in connectionmanager since we'd already have a userptr...

/J

Re: [dcplusplus-devel] ADC typeing notification

[Jacek S. <arn...@gm...>]

I've added the extension to the adc wiki
(http://adc.sourceforge.net/wiki/index.php/Extensions#TYPE_-_Typing_notification) - I'll have a few
more comments to add there...

/J

Jan Trofimov wrote:
>> I generally agree that this should go as a separate message - MSG is for sending text to the user,
>> no use sending this kind of stuff as text (it would get logged, it couldn't be detected
>> automagically etc etc)...
>>
>> Intuitively I'm not convinced about the code set of what the user is doing - I'd say that would need
>> some review and maybe a reference implementation to see it in action...
> Well i got this types not from the ceiling.
> http://www.xmpp.org/extensions/xep-0085.html#bizrules
> 
> This works in jabber for years and IMHO it's nice as it already is.
> 
> Maybe code shouldnt be a number, maybe it should be a string like "gone".
> 
>> Also, this should clearly go as an extension, not part of BASE. It wouldn't even have to be present
>> in the SUP since it doesn't require anything as far as hub-client comms go...
> Well... may be if it (well be) put in SUP than client may make feature broadcast. This will generate less traffic.
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2005.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> dcplusplus-devel mailing list
> dcp...@li...
> https://lists.sourceforge.net/lists/listinfo/dcplusplus-devel
> 

Re: [dcplusplus-devel] i18n

[Jacek S. <arn...@gm...>]

0.17, yes...I'm compiling on linux though so it's slightly easier for me =)...it's supposed to be
compileable on win32 as well so if you make some binaries, maybe it would make sense to put them up
somewhere to make it easier for others...

/J

David Grundberg wrote:
> Slightly OT, what version of the gettext tools are required? Building 
> with scons, I get a 'xgettext: unrecognized flag "--package-name=dcpp"'. 
> I'm using the latest precompiled binaries I could find (0.13.1), but 
> they generate this error message. Have you compiled them yourself from 
> the gettext 0.17 source? Naturally, I'm on win32.
> 
> David
> 
> 
> 
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2005.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> dcplusplus-devel mailing list
> dcp...@li...
> https://lists.sourceforge.net/lists/listinfo/dcplusplus-devel
> 

Re: [dcplusplus-devel] i18n

[David G. <c0...@cs...>]

Slightly OT, what version of the gettext tools are required? Building 
with scons, I get a 'xgettext: unrecognized flag "--package-name=dcpp"'. 
I'm using the latest precompiled binaries I could find (0.13.1), but 
they generate this error message. Have you compiled them yourself from 
the gettext 0.17 source? Naturally, I'm on win32.

David

Re: [dcplusplus-devel] i18n

[David G. <c0...@cs...>]

When you have decided what formatting function to use, I will help out 
with 2, 3 and 4.

David (individ)

Jacek Sieka wrote:
> 1) Decide on formatting function (with positional args, preferably with automatic memory managment).
> Boost seems the best so far (positional params, supported by xgettext, no memory hassles) but I'm
> open to (msvcrt-friendly) alternatives... Steven? If boost, I'm open to using more of the boost
> library (pointers, threads etc), evaluating each sublibrary on its own...
> 
> 2) Convert all strings to use above mentioned formatting function
> 
> 3) Make a script that converts the old language files
> 
> 4) Preferably create the automatic merge script for the two project guis...
> 
> Anyone interested?

[dcplusplus-devel] i18n

[Jacek S. <arn...@gm...>]

Ok,

I've now committed the basic i18n support using gettext...I'm no gettext expert so there are
probably better ways to go about, but it seems to be working more or less...let me know what I got
wrong...

What's left is the following:

1) Decide on formatting function (with positional args, preferably with automatic memory managment).
Boost seems the best so far (positional params, supported by xgettext, no memory hassles) but I'm
open to (msvcrt-friendly) alternatives... Steven? If boost, I'm open to using more of the boost
library (pointers, threads etc), evaluating each sublibrary on its own...

2) Convert all strings to use above mentioned formatting function

3) Make a script that converts the old language files

4) Preferably create the automatic merge script for the two project guis...

Anyone interested?

/J

Re: [dcplusplus-devel] Internationalization

[David G. <c0...@cs...>]

Jacek Sieka wrote:
>> The gnutext manual is geared towards the use case where two packages are 
>> merged. I would need to read further, but I think it's possible to 
>> "sync" translations between win32/linux using msgcomm and msgcat 
>> intelligently.
>>     
> Automatically is better than intelligently...
>   
Writing the script that runs those utilities will require thought, while 
executing the script itself will be as automatic as it can be.

The boost format library IMHO looks nice, I remember a similar design 
(or "named parameters", as the pattern is called) in QString from Qt. 
The only thing printf has going for it is the gettext "support".

David

Re: [dcplusplus-devel] Internationalization

[Jacek S. <arn...@gm...>]

> Did you mean that win32 doesn't have positional parameters? I'm not a 
> win32 developer, but what about printf_p? Doesn't seem that standard 
> though...
> 
> printf_p: http://msdn2.microsoft.com/en-us/library/bt7tawza(VS.80).aspx
> 
We're limited to using msvcrt.dll (essentially msvc 6.0) which is the only widely available c
library on windows (the others are restricted for redistribution and not supported by mingw), so no.

> Why just not use the syntax of printf's positional parameters? Since 
> gettext recognizes them. Also, translators are more likely to recognize 
> something that is commonly used.
Because then we'd have to reimplement the rest of printf (all other formatting flags) or limit the
support to %m$s (strings)...it's feasible but not optimal. At this point we might as well ship a
complete free implementation of printf (gettext includes one with a c++ wrapper which solves some of
the issues of normal printf), but IMHO if we're to ship a printf, we might as well use boost which
is widely supported and fits well with c++ in general (unlike the gettext wrapper which is not
typesafe and still requires that types be specified in the formatting string)...

> The gnutext manual is geared towards the use case where two packages are 
> merged. I would need to read further, but I think it's possible to 
> "sync" translations between win32/linux using msgcomm and msgcat 
> intelligently.
Automatically is better than intelligently...

> I'm glad to hear this talk about i18n... :)
talk in this case isn't enough though =)

/J

Re: [dcplusplus-devel] Internationalization

[David G. <c0...@cs...>]

Jacek Sieka skrev:
> - I don't know of any other reasonable formatting solutions (steven suggested printf which on most
> unices supports positional arguments by using %m$ but that's not supported on unix and still
> requires manual memory managment...)
>   
Did you mean that win32 doesn't have positional parameters? I'm not a 
win32 developer, but what about printf_p? Doesn't seem that standard 
though...

printf_p: http://msdn2.microsoft.com/en-us/library/bt7tawza(VS.80).aspx

> - One possible solution would be to use our own string replacement routines ("%[xxx]")...this is
> slightly ugly since we have to create a map each time and convert the arguments to strings manually
> (Util::toString), and we lose some of the tooling support that gettext offers for standard format
> strings.
>   
Why just not use the syntax of printf's positional parameters? Since 
gettext recognizes them. Also, translators are more likely to recognize 
something that is commonly used.

> * dcpp/ and win32/ would have separate translation files - but since the strings for both the win32
> and linux gui are similar, it would be nice to have a way to synchronize those strings every now and
> then. I'm not sure if there are some tools to merge all equal strings from two po's and leave the
> rest alone, but if there aren't a script would be nice here too...
>   
The gnutext manual is geared towards the use case where two packages are 
merged. I would need to read further, but I think it's possible to 
"sync" translations between win32/linux using msgcomm and msgcat 
intelligently.

I'm glad to hear this talk about i18n... :)

David (individ)

[dcplusplus-devel] LinuxDC++: Fixes a nick encoding issue that could cause transfers to fail

[Steven S. <ste...@gm...>]

Credit goes to Stanislav Maslovski. Here's his original argument for it:

Consider a client operating in active mode. In order to get a filelist
NmdcHub::connectToMe() first places the triplet (( peerNick, myNick, hubUrl ))
on the list of expected connections. Internally, nicks are stored in utf-8.
Note that peerNick in this triplet gets stored also in utf-8.

Next, NmdcHub::connectToMe() sends
"$ConnectToMe " + toAcp(peerNick) + ....

So far so good. Now the peer connects. ConnectionManager::accept() creates
new UserConnection, sets flags, etc. but because it does not know anything
about peer's encoding (the new connection can be from an arbitrary user from
arbitrary hub) it leaves it with the default (which is Text::systemCharset).

Now peer sends
$MyNick peerNick_in_acp

UserConnection::on() receives this, blindly translates peerNick_in_acp to
utf-8 from the encoding associated with the connection (which is
Text::systemCharset at the moment). If peer's encoding was different we get
an obvious result: the connection is rejected in ConnectionManager::on(MyNick)
because the translation fails and the peer is not found in the list of
expected connections.

Here goes an example session (text below contains russian chars):

NmdcHub::connect Трельник
NmdcHub::connectToMe Трельник
BufferedSocket::accept() 0x86d2a00
BufferedSocket::run() start 0x86d2a00
ConnectionManager::onMyNick 0x86974e0, ________
Unknown incoming connection from ________
BufferedSocket::run() end 0x86d2a00

In this example, the system locale was ru_RU.UTF-8, but the hub charset was CP1251.
If someone thinks that setting locale to CP1251 will solve the problem, then it
is not so, because there always will be hubs with different encodings and we
want linuxdcpp operate _simultaneously_ on all of them.

The attached patch solves this in the following way. First, the peer nick is
stored on the list of expected connections in ACP, not in uft-8. This is not
a problem, because the stored nick is just a key and is not used for any
other purpose. Next, when UserConnection::on() receives $MyNick it is passed
further untranslated. Peer's nick gets translated to utf-8 later inside
ConnectionManager::on(MyNick) when it becomes possible to determine and set
the correct encoding.

This all can be seen directly from the patch, of course. Perhaps, a better
way of doing things could be to separate incomming connection checking from
ConnectionManager::on(MyNick) and put it in UserConnection::on() or do
something similar.

[dcplusplus-devel] Internationalization

[Jacek S. <arn...@gm...>]

I've been discussing with Steven Sheehy (linuxdc++) the possible replacement of stringdefs.h/cpp
with gnu gettext. I18n in dc++ currently suffers from a few issues such as parameter ordering (some
translation require reordering of %-arguments), memory managment issues (manual buffer allocation et
al), poor distribution model (user needs to know english to install language file).

By using gettext we solve the distribution issue (and add another - a new release is needed to
update a language) - language files are delivered together with dc++ in the installer. We also get
to use existing translation tools, of which a few even work with windows (www.poedit.net). Steven
pointed out another advantage - with gettext the actual string can be seen directly without having
to look it up in the enum table.

There are a few issues though:
* We still have to deal with formatting. Currently we either do STRING(XXX) + Util::toString(yyy) or
sprintf(buf, CSTRING(XXX), yyy), both of which need to go away.
- boost::format solves both these things but introduces boost as dependency for the dcpp/ core as
well. steven doesn't like that, unless boost is used for more than just formatting. Some useful
boost libraries unfortunately require compiling which is slightly messy on windows - obviously using
the header-only libraries is a good thing in terms of code correctness and platform independence so
those could obviously be used.
- I don't know of any other reasonable formatting solutions (steven suggested printf which on most
unices supports positional arguments by using %m$ but that's not supported on unix and still
requires manual memory managment...)
- One possible solution would be to use our own string replacement routines ("%[xxx]")...this is
slightly ugly since we have to create a map each time and convert the arguments to strings manually
(Util::toString), and we lose some of the tooling support that gettext offers for standard format
strings.

* Windows support in the gettext library is scarce, so a way to convince it to use a particular
language and load it from the correct place (since gettext has its own specific opinion about where
language files should be) needs to be found. This shouldn't be too hard though...

* Windows and unix use different names for locales, and gettext uses the locale name to figure out
where to load the file...

* All of the existing STRING(xxx) need to be replaced by their respective texts from stringdefs,
while paying attention to arguments etc (so a simple script won't be able to do it). This is quite a
lot of work...anyone interested?

* A script should be made to convert the current xml translations - I think there are some 30
translations already so losing them wouldn't be nice...again, anyone interested?

* dcpp/ and win32/ would have separate translation files - but since the strings for both the win32
and linux gui are similar, it would be nice to have a way to synchronize those strings every now and
then. I'm not sure if there are some tools to merge all equal strings from two po's and leave the
rest alone, but if there aren't a script would be nice here too...

/J

Re: [dcplusplus-devel] patch: about SharedFile

[Jacek S. <arn...@gm...>]

poy wrote:
> makes SharedFile inherit from OutputStream instead of both InputStream
> and OutputStream; removes the need to have a read() function which was
> updating the written bytes value...
> 
> also, what about calling ::CreateFile() with FILE_SHARE_WRITE in the
> File constructor, so that Windows handles multiple writers by itself?
Bah. because that would be too easy =)

> 
> poy
/J

[dcplusplus-devel] patch: about SharedFile

[poy <po...@12...>]

makes SharedFile inherit from OutputStream instead of both InputStream and 
OutputStream; removes the need to have a read() function which was updating 
the written bytes value...

also, what about calling ::CreateFile() with FILE_SHARE_WRITE in the File 
constructor, so that Windows handles multiple writers by itself?

poy 

Re: [dcplusplus-devel] shell links

[poy <po...@12...>]

you'll need the IShellLink and IPersistFile interfaces, which need the win32 
headers to be included. having DC++ follow shortcuts automatically, so that 
one can add a shortcut to a file in one's share instead of copying the whole 
file, would be awesome. :)

poy

----- Original Message ----- 
From: "Jacek Sieka" <arn...@gm...>
To: "Patches & development discussion" 
<dcp...@li...>
Sent: Friday, December 14, 2007 8:45 PM
Subject: Re: [dcplusplus-devel] LinuxDC++: Convert u_int*_t to uint*_t C99 
types
>> Adds an option to not follow symbolic links on *nix systems. Sometimes
>> following links can be problematic so this option is necessary for these
>> situations.
> Ok. At some point I really have to make the settings stuff a little less 
> centralized...technically
> this could be implemented for win32 as well I think, but I don't remember 
> the exact semantics of
> shell links with regards to directories... 

Re: [dcplusplus-devel] LinuxDC++: Convert u_int*_t to uint*_t C99 types

[Stanislav M. <sta...@gm...>]

On Fri, Dec 14, 2007 at 02:10:25PM -0600, Steven Sheehy wrote:

[ skipped ]

>>> Description from original patch writer:
>>>
>>> My computer is connected to 2 huge LANs and the internet.
>>> Respectively, the outside world knows it by 3 different IPs.
>>
>> While the patch itself seems fine, I'll be needing agreement from the original patch author before I
>> add this...I'd also like to know who it was that wrote it in order to give due recognition (if the
>> author wants)...
>
>
> The author is Stanislav Maslovski. I cc'd him so he can provide his input.

Thanks! I am, of course, not against that inclusion. Both for the patch and
for the name if you feel that it is appropriate.

-- 
Stanislav

Re: [dcplusplus-devel] LinuxDC++: Convert u_int*_t to uint*_t C99 types

[Razzloss <Raz...@gm...>]

On Friday 14 December 2007 22:10:25 Steven Sheehy wrote:
> Jacek Sieka wrote:
> >> Fixes the disconnected string showing in the transfers pane before the
> >> no slots available status.
> >
> > I discussed this patch with razzloss and we both came to the conclusion
> > that its the responsability of the gui to tell which error messages to
> > display from where...besides, I'm anticipating some changes to the
> > transfer view now that we have segmented downloading in the core...
>
> Great, razzloss failed to mention this...
>

In my defense it was only few (three at most) weeks ago... And I've been 
pretty busy with school & work lately and so I haven't got any time to do  
anything related to (Linux)DC++. I've been meaning to talk to you about it, 
but... yeah.

IMHO the state wether to show the disconnect should be added to GUI-part as 
there are few other things that would benefit from it (setting the 
sensitivity of some context menu options to false/true atleast, though 
clearly it is not the killer feature everyone is waiting for, but gives a 
more finished look for the app). And also I'm assuming that the TTH/rollback 
failures lead to similar disconnect? So it would be easier just to add one 
hidden column to transferview and use that to determine if disconnect should 
be shown.

--RZ

Re: [dcplusplus-devel] LinuxDC++: Convert u_int*_t to uint*_t C99 types

[Steven S. <ste...@gm...>]

Jacek Sieka wrote:
> Steven Sheehy wrote:
>> Use the standardized C99 types. This is our fault for not checking it
>> when we added the *nix fast hash. :D
> ok. ever considered mmap?

fasthash() already does use mmap...


>> Fixes the disconnected string showing in the transfers pane before the
>> no slots available status.
> I discussed this patch with razzloss and we both came to the conclusion that its the responsability
> of the gui to tell which error messages to display from where...besides, I'm anticipating some
> changes to the transfer view now that we have segmented downloading in the core...

Great, razzloss failed to mention this...

>> Description from original patch writer:
>>
>> My computer is connected to 2 huge LANs and the internet.
>> Respectively, the outside world knows it by 3 different IPs.
> 
> While the patch itself seems fine, I'll be needing agreement from the original patch author before I
> add this...I'd also like to know who it was that wrote it in order to give due recognition (if the
> author wants)...


The author is Stanislav Maslovski. I cc'd him so he can provide his input.

-Steven

Re: [dcplusplus-devel] LinuxDC++: Convert u_int*_t to uint*_t C99 types

[Jacek S. <arn...@gm...>]

Steven Sheehy wrote:
> Use the standardized C99 types. This is our fault for not checking it
> when we added the *nix fast hash. :D
ok. ever considered mmap?

> Use strtoll instead of atoll to enable build on HPUX
Ok.

> Fixes the disconnected string showing in the transfers pane before the
> no slots available status.
I discussed this patch with razzloss and we both came to the conclusion that its the responsability
of the gui to tell which error messages to display from where...besides, I'm anticipating some
changes to the transfer view now that we have segmented downloading in the core...

> Adds an option to not follow symbolic links on *nix systems. Sometimes
> following links can be problematic so this option is necessary for these
> situations.
Ok. At some point I really have to make the settings stuff a little less centralized...technically
this could be implemented for win32 as well I think, but I don't remember the exact semantics of
shell links with regards to directories...

> Currently, if the Text::convert() encounters a conversion where the from
> and to encodings are the same, it returns the initial string with no
> conversion taking place. As I found out, this causes problems where the
> text that says it is utf-8 isn't actually valid utf-8. In this case,
> chat messages would not appear in the GUI since GTK+ won't display or
> convert invalid utf-8. To remedy this, the text conversion functions
> need to validate the text even if their encoding implies that no
> conversion is necessary. I moved these checks into the WIN32 blocks.
> Feel free to delete them altogether if you think you need to validate on
> windows as well.
I left them as is for now, although I guess at some point the win32 version should make the
validation as well...apart from validation it should also make normalization to be adc compliant but
that's an issue for a later date...

> Description from original patch writer:
>
> My computer is connected to 2 huge LANs and the internet.
> Respectively, the outside world knows it by 3 different IPs.

While the patch itself seems fine, I'll be needing agreement from the original patch author before I
add this...I'd also like to know who it was that wrote it in order to give due recognition (if the
author wants)...

/J

[dcplusplus-devel] LinuxDC++: Improve active searches when using multiple IPs

[Steven S. <ste...@gm...>]

Description from original patch writer:

My computer is connected to 2 huge LANs and the internet.
Respectively, the outside world knows it by 3 different IPs.

There are dc hubs on the two LANs and, of course, on the internet.
I noticed that when I connect to hubs which are on networks different
from the network of the hub I have first connected to, the active
searches never work on these hubs. After some investigation I found that
the IP my client sends to these hubs in search requests was wrong.
In the same time the IP sent in ConnectToMe requests was correct,
so that I could download files, but could not do searches.

The attached patch solves this problem by removing bogus cachedIp
code used in search requests, and makes the behaviour of search consistent
with the behavior of ConnectToMe. Please notice that the functionality of
Client::getLocalIp() is similar to that of the removed
ClientManager::updateCachedIp(), except that the former does not make
any assumptions about hosts having just a single IP.

Moreover, despite the fact that the caching approach was simply incorrect,
it was also useless because the IP was already stored locally
(see the code of Client::getLocalIp()).

[dcplusplus-devel] LinuxDC++: Validate UTF-8 text

[Steven S. <ste...@gm...>]

Currently, if the Text::convert() encounters a conversion where the from and to 
encodings are the same, it returns the initial string with no conversion taking 
place. As I found out, this causes problems where the text that says it is utf-8 
isn't actually valid utf-8. In this case, chat messages would not appear in the 
GUI since GTK+ won't display or convert invalid utf-8. To remedy this, the text 
conversion functions need to validate the text even if their encoding implies 
that no conversion is necessary. I moved these checks into the WIN32 blocks. 
Feel free to delete them altogether if you think you need to validate on windows 
as well.

[dcplusplus-devel] LinuxDC++: Don't follow symbolic links

[Steven S. <ste...@gm...>]

Adds an option to not follow symbolic links on *nix systems. Sometimes following 
links can be problematic so this option is necessary for these situations.

[dcplusplus-devel] LinuxDC++: Disconnected showing before no slots available

[Steven S. <ste...@gm...>]

Fixes the disconnected string showing in the transfers pane before the no slots 
available status.

[dcplusplus-devel] LinuxDC++: Use strtoll instead of atoll to enable build on HPUX

[Steven S. <ste...@gm...>]

Use strtoll instead of atoll to enable build on HPUX

[dcplusplus-devel] LinuxDC++: Convert u_int*_t to uint*_t C99 types

[Steven S. <ste...@gm...>]

Use the standardized C99 types. This is our fault for not checking it when we 
added the *nix fast hash. :D

Re: [dcplusplus-devel] adchpp patch: unicode for lua

[Jacek S. <arn...@gm...>]

> i am not the author but i changed the name of some functions in the libary because i got
> "redefined" compiler error in all.c.  these functions where already defined in the string libary of lua.
> i used replace of an editor and replaced also the name of the lua functions "match" and "gmatch" in the table  "unicode" which is not necessary, that was the mistake.

Well, the correct solution would be to compile it separately from all.c without any modifications to
the files...it would require playing some with the sconstruct to get it to build correctly, but the
end result will be neater...besides, changing the library would surprise other lua coders if  the
functions weren't named the same...


/J