sqlgrey-users

[Sqlgrey-users] greylisting and UTF8

[Nicolas S. <Ni...@st...>]

Hi,

I have encountered this error while using sqlgrey:

Nov 25 08:43:20 ovh sqlgrey: mail: failed to send: Cannot allocate memory
Nov 25 08:43:20 ovh sqlgrey: dbaccess: error: couldn't access from_awl
table:
Nov 25 08:43:20 ovh sqlgrey: dbaccess: warning: couldn't do query:
UPDATE from_awl SET last_seen = NOW(), first_seen = first_seen WHERE
sender_name = 'robertandr�2' AND sender_domain = 'ohiohills.com' AND src
= '222.123.27.243': ERREUR:  Séquence d'octets invalide pour le codage
«UTF8» : 0xe93227 HINT:  Cette erreur peut aussi survenir si la séquence
d'octets ne correspond au jeu de caractères attendu par le serveur et
qui est contrôlé par « client_encoding ». , reconnecting to DB
Nov 25 08:43:30 ovh sqlgrey: mail: failed to send: Cannot allocate memory

The important thing is that by using UTF8 characters in the FROM adress,
any machine is able to bypass sqlgrey because it fails to deal with UTF8
characters... Is there any workaround for this ? I'm using sqlgrey 1.7.3
with postgresql.

Thanks,
-- 
Nico
La science ne se soucie ni de plaire, ni de déplaire, elle est
inhumaine.
	-+- Jacques Anatole François Thibault dit Anatole France
	    (1844-1924) -+-

Re: [Sqlgrey-users] greylisting and UTF8

[Lionel B. <lio...@bo...>]

Nicolas STRANSKY wrote the following on 25.11.2007 11:53 :
> Hi,
>
> I have encountered this error while using sqlgrey:
>
> Nov 25 08:43:20 ovh sqlgrey: mail: failed to send: Cannot allocate memory
> Nov 25 08:43:20 ovh sqlgrey: dbaccess: error: couldn't access from_awl
> table:
> Nov 25 08:43:20 ovh sqlgrey: dbaccess: warning: couldn't do query:
> UPDATE from_awl SET last_seen = NOW(), first_seen = first_seen WHERE
> sender_name = 'robertandr�2' AND sender_domain = 'ohiohills.com' AND src
> = '222.123.27.243': ERREUR:  Séquence d'octets invalide pour le codage
> «UTF8» : 0xe93227 HINT:  Cette erreur peut aussi survenir si la séquence
> d'octets ne correspond au jeu de caractères attendu par le serveur et
> qui est contrôlé par « client_encoding ». , reconnecting to DB
>   

Your database is configured to use the UTF-8 encoding, use ASCII instead.

> Nov 25 08:43:30 ovh sqlgrey: mail: failed to send: Cannot allocate memory
>   

It seems you have other problems...

> The important thing is that by using UTF8 characters in the FROM adress,
> any machine is able to bypass sqlgrey because it fails to deal with UTF8
> characters...

This isn't the problem. Mail addresses use ASCII. In ASCII some
characters aren't valid UTF-8 sequences. So you should't use UTF-8 to
store email adresses...

Lionel

Re: [Sqlgrey-users] greylisting and UTF8

[Karl O. P. <ko...@me...>]

On 11/25/2007 05:10:23 AM, Lionel Bouton wrote:
> Nicolas STRANSKY wrote the following on 25.11.2007 11:53 :
>=20
> This isn't the problem. Mail addresses use ASCII. In ASCII some
> characters aren't valid UTF-8 sequences. So you should't use UTF-8 to
> store email adresses...

There are (now) unicode domain name representations.
Does sqlgrey deal with internationalized domain names (RFC3490)
punycode (RFC3492) and friends (RFC3454, RFC3491) or is that already
taken care of by the MTAs?

What does domain name internationalization mean for the content
of sqlgrey's db?

(See also: http://en.wikipedia.org/wiki/Internationalized_domain_name)


Karl <ko...@me...>
Free Software:  "You don't pay back, you pay forward."
                  -- Robert A. Heinlein

Re: [Sqlgrey-users] greylisting and UTF8

[Lionel B. <lio...@bo...>]

Karl O. Pinc wrote:
> On 11/25/2007 05:10:23 AM, Lionel Bouton wrote:
>   
>> Nicolas STRANSKY wrote the following on 25.11.2007 11:53 :
>>
>> This isn't the problem. Mail addresses use ASCII. In ASCII some
>> characters aren't valid UTF-8 sequences. So you should't use UTF-8 to
>> store email adresses...
>>     
>
> There are (now) unicode domain name representations.
>   

They are quite old. The mechanism used by IDN predates the first SQLgrey
release :-)

> Does sqlgrey deal with internationalized domain names (RFC3490)
> punycode (RFC3492) and friends (RFC3454, RFC3491) or is that already
> taken care of by the MTAs?
>   

SQLgrey doesn't care *one bit*. IDN uses an Unicode encoding which is
meant to work with tools understanding ASCII only (which SQLgrey is).
IDN wouldn't be possible if you'd have to change each and every name
server, mail server, tool used for mail handling out there...

The only greylisting problem with SQLgrey could be the heuristics used
to guess if a name is more likely to be a mail server or a personal
(dialup/ADSL/cable/...) connection. Even then, the heuristics only care
about the first part of the DNS name, not the DNS zone it is attached
to. So they should be OK even with IDN.
> What does domain name internationalization mean for the content
> of sqlgrey's db?
>   

Nothing much, if you have tools designed to show the DB content, you'll
have to make them IDN-aware (which means they'll output unicode by the
way) in order to decode the domain name.

The only real practical problem with SQLgrey and IDN is the log. You'll
have the ASCII-encoded names in them and that could make things
difficult to diagnose. Though I suspect that everybody logs
ASCII-encoded names, so matching SQLgrey log entries with Postfix' or
other mail-related tool's ones should be OK.

Lionel