Menu
▾
▴
sqlgrey-users
|
From: Klaus A. S. <kse...@gm...> - 2004-12-19 19:37:59
|
Hi, Just wondering . . . My primary MX, mx.szn.dk, accepts connections on both IPv4 and IPv6. How does SQLgrey handle IPv6 addresses? Would anyone with a IPv6 capable MTA care to test mail me at kl...@se... so I can see how SQLgrey reacts? Thanks. Cheers, --=20 Klaus Alexander Seistrup SubZeroNet =B7 Copenhagen =B7 Denmark |
|
From: Lionel B. <lio...@bo...> - 2004-12-19 21:04:43
|
Klaus Alexander Seistrup wrote the following on 12/19/04 20:37 : >Hi, > >Just wondering . . . My primary MX, mx.szn.dk, accepts connections on >both IPv4 and IPv6. How does SQLgrey handle IPv6 addresses? > > I believe badly... I must admit I never thought of the IPv6 case. I've not yet thought much about this but the main problem is that the current address fields aren't large enough to store every IPv6 addresses. The simplest thing for SQLgrey is take the input it gets from Postfix and put it directly in the address field. The second problem is in the smart and classc greylisting algorithms, they'll have to be IPv6 aware (what's a rough equivalent of a class C network in v6, can it be made of all IPv6 with only the last 2 bytes modified or are there more subtle things to consider, think of the representation of IPv4 addresses encapsulated in IPv6 for example). Unfortunately, I've zero access to IPv6 networks so I can't test any of these. What I can do is changing the table layouts in 1.5.x to handle the maximum IPv6 address representation size (39 bytes). You should then switch to the 'full' greylisting method and send me examples of IPv6 entries (I'm not sure of the layout used by Postfix, in fact I'm not sure Postfix decides the layout, it might be the libc). I'll ask on the Postfix mailing-list... Lionel. |
|
From: Klaus A. S. <kse...@gm...> - 2004-12-19 21:33:19
|
Lionel Bouton wrote: >> How does SQLgrey handle IPv6 addresses? >=20 > I believe badly... > I must admit I never thought of the IPv6 case. Heheh... :-) > I've not yet thought much about this but the main problem is that the > current address fields aren't large enough to store every IPv6 addresses. I'm using SQLite (v2) db. I think SQLite supports "unlimited" storage for strings if using TEXT instead of VARCHAR. Well, even the size of the VARCHAR is not respected by SQLite, so I would expect an SQLite db to survive IPv6 addresses. However, ... > The second problem is in the smart and classc greylisting algorithms, ... Yes. > Unfortunately, I've zero access to IPv6 networks so I can't test any of > these. Neiter have I, execept my own server (and all connections appear to come from ::1). However, I hope that other list members do, and mail me... > What I can do is changing the table layouts in 1.5.x to handle the > maximum IPv6 address representation size (39 bytes). You should then > switch to the 'full' greylisting method and send me examples of IPv6 > entries (I'm not sure of the layout used by Postfix, in fact I'm not > sure Postfix decides the layout, it might be the libc). Hm, yes, I think "full" greylisting will have to be the answer in that case, I didn't think of that before. > I'll ask on the Postfix mailing-list... Meanwhile I'll look for IPv6 connections to my MX. Cheers, --=20 Klaus Alexander Seistrup Copenhagen =B7 Denmark |
|
From: Klaus A. S. <kse...@gm...> - 2004-12-19 22:04:31
|
Lionel Bouton wrote:
> (I'm not sure of the layout used by Postfix, in fact I'm not
> sure Postfix decides the layout, it might be the libc).
No matter what IPv6 address I use for telnet'ing to my MX (currently
also my workstation), it appears as ::1, so I'm afraid I cannot help
much. Here's how it looks in the Received header:
#v+
Received: from ip6-localhost (unknown [IPv6:::1])
by mx.szn.dk (Postfix) with ESMTP id C56448F7CF
for <kl...@se...>; Sun, 19 Dec 2004 22:56:51 +0100 (CET)
#v-
And since this is localhost, it doesn't go through SQLgrey.
Cheers,
--=20
Klaus Alexander Seistrup
Copenhagen =B7 Denmark
|
|
From: Klaus A. S. <kse...@gm...> - 2005-03-06 10:22:27
|
On Sun 19 Dec 2004, Lionel Bouton <lio...@bo...> wrote: > What I can do is changing the table layouts in 1.5.x to handle the > maximum IPv6 address representation size (39 bytes). Any news about the IPv6 issue? I'm running SQLgrey v1.5.1, and the table layout still uses 15 charcters for storing IP addresses. Some time ago I switched from SQLite to PostgreSQL. Any chance to have SQLgrey use the INET storage type if database is PostgreSQL? Would it be worth the trouble to put database specific rutines in separate perl scripts, each having a similar API? E.g. mysql-api.pm, postgres-api.pm, sqlite-api.pm, ..., that each have a create_tables() routine to create necessary tables. Cheers, --=20 Klaus Alexander Seistrup SubZeroNet =B7 Copenhagen =B7 Denmark |
|
From: Lionel B. <lio...@bo...> - 2005-03-06 11:41:34
|
Klaus Alexander Seistrup wrote the following on 06.03.2005 11:22 :
>On Sun 19 Dec 2004, Lionel Bouton <lio...@bo...> wrote:
>
>
>
>>What I can do is changing the table layouts in 1.5.x to handle the
>>maximum IPv6 address representation size (39 bytes).
>>
>>
>
>Any news about the IPv6 issue? I'm running SQLgrey v1.5.1, and the
>table layout still uses 15 charcters for storing IP addresses.
>
>
>
Ooops, I forgot about IPv6. It seems as Postfix 2.2 is coming shortly
with official IPv6 support we can now support it too.
It still will be tricky to handle. I need to learn a few things about
IPv6 beforehand:
- what's the longest representation of an IPv6 address again?
- for 'smart' and 'classc', how do we compute the IPv6 network similar
to the IPv4 "x.x.x.0/24" network? For the common case, is removing the
last 32 bit hexadecimal component a good approach? Or are the IPv6
addresses addresses distributed to end-users in a way that makes it
possible to have wildly different systems in the same "/120" network?
- can we blindly remove the last :hexa component or are there
compression tricks in the representation that will prevent this? i.e.:
. common case "2001:4f8:3:ba:2e0:81ff:fe22:d1f1" ->
"2001:4f8:3:ba:2e0:81ff:fe22",
. imaginary problematic case: "2001:4f8:3:ba:2e0::" ->
"2001:4f8:3:ba:2e0:" (can it happen?).
From my reading of the Postfix documentation, we can at least rule out
the programmer's nightmare of IPv4 in IPv6 addresses representations
("::101.45.23.74" is a valid IPv6 address).
>Some time ago I switched from SQLite to PostgreSQL. Any chance to
>have SQLgrey use the INET storage type if database is PostgreSQL?
>
>
In fact the original SQLgrey code used a PostgreSQL specific whitelist
table using INET or CIDR. I removed this. I don't like database specific
code. This make for more complex and less maintainable code.
If INET proves to be useful it will probably be by allowing to handle
some computations inside the database, it won't happen without a lot of
code being reengineered for PostgreSQL. SQLgrey will probably have to
fork into SQLgrey-generic and SQLgrey-PostgreSQL to make it easier to
maintain.
VARCHAR is probably less efficient than INET, but probably not enough to
have PostgreSQL specific code.
I'm not even sure that the performance difference will be measurable.
INET is a complex storage type compared to VARCHAR, it must probably
make provisions to store the type (IPv4/IPv6) and does store the network
too -> it will use at least 17 bytes, probably more. On the other hand a
VARCHAR IPv6 representation of an IPv4 address - the typical case -
should be less than 20 bytes.
>Would it be worth the trouble to put database specific rutines in
>separate perl scripts, each having a similar API? E.g. mysql-api.pm,
>postgres-api.pm, sqlite-api.pm, ..., that each have a create_tables()
>routine to create necessary tables.
>
>
>
Probably not at this point. This could be used to avoid forking SQLgrey
to support database specific code but you'll probably end with 90% of
SQLgrey's code in the postgres-api.pm for example.
IPv6 in TODO.
Lionel.
|
|
From: Klaus A. S. <kse...@gm...> - 2005-03-06 12:12:24
|
Lionel Bouton wrote: > Ooops, I forgot about IPv6. It seems as Postfix 2.2 is coming shortly > with official IPv6 support we can now support it too. :-) > - what's the longest representation of an IPv6 address again? 1234:1234:1234:1234:1234:1234:1234:1234 =3D 39 characters > - for 'smart' and 'classc', how do we compute the IPv6 network similar > to the IPv4 "x.x.x.0/24" network? For the common case, is removing the > last 32 bit hexadecimal component a good approach? Or are the IPv6 > addresses addresses distributed to end-users in a way that makes it > possible to have wildly different systems in the same "/120" network? I'm not sure. Many tunnel brokers hand out /64 or /48 nets to end users. E.g., I have [2001:1448:89::]/48 from ngdc.dk. My impression is that IPv6/64 is somewhat similar to IPv4/32 (I'm not sure if IPv6/48 should be compared to IPv4/24, though). but that is highly subjective. > - can we blindly remove the last :hexa component or are there > compression tricks in the representation that will prevent this? i.e.: > . common case "2001:4f8:3:ba:2e0:81ff:fe22:d1f1" -> > "2001:4f8:3:ba:2e0:81ff:fe22", > . imaginary problematic case: "2001:4f8:3:ba:2e0::" -> > "2001:4f8:3:ba:2e0:" (can it happen?). As far as I know, compression always involves "::", so "2001:4f8:3:ba:2e0:" would never occur as shorthand for "2001:4f8:3:ba:2e0::". =20 >> Some time ago I switched from SQLite to PostgreSQL. Any chance to >> have SQLgrey use the INET storage type if database is PostgreSQL? >=20 > In fact the original SQLgrey code used a PostgreSQL specific whitelist > table using INET or CIDR. I removed this. I don't like database specific > code. This make for more complex and less maintainable code. Sure. =20 > VARCHAR is probably less efficient than INET, but probably not enough to > have PostgreSQL specific code. I'm a lazy type of person, that's why I would like to use INET (or CIDR) i SQLgrey. The storage type handles IPv4 and IPv6 addresses transparently, and it is very easy to test if a given IP address belongs to a network ("SELECT foo FROM bar WHERE ip << '2001:1:2::/48'"). >> Would it be worth the trouble to put database specific rutines in >> separate perl scripts, each having a similar API? E.g. mysql-api.pm, >> postgres-api.pm, sqlite-api.pm, ..., that each have a create_tables() >> routine to create necessary tables. >=20 > Probably not at this point. This could be used to avoid forking SQLgrey > to support database specific code but you'll probably end with 90% of > SQLgrey's code in the postgres-api.pm for example. I see. > IPv6 in TODO. :-) Cheers, --=20 Klaus Alexander Seistrup SubZeroNet =B7 Copenhagen =B7 Denmark |
|
From: Lionel B. <lio...@bo...> - 2005-03-06 16:40:03
|
Klaus Alexander Seistrup wrote the following on 06.03.2005 13:12 : >>- for 'smart' and 'classc', how do we compute the IPv6 network similar >>to the IPv4 "x.x.x.0/24" network? For the common case, is removing the >>last 32 bit hexadecimal component a good approach? Or are the IPv6 >>addresses addresses distributed to end-users in a way that makes it >>possible to have wildly different systems in the same "/120" network? >> >> > >I'm not sure. Many tunnel brokers hand out /64 or /48 nets to end >users. E.g., I have [2001:1448:89::]/48 from ngdc.dk. My impression >is that IPv6/64 is somewhat similar to IPv4/32 (I'm not sure if >IPv6/48 should be compared to IPv4/24, though). but that is highly >subjective. > > I'm not sure IPv6/64 is the same as IPv4/32, I'll ask around as I know some ADSL users with IPv6 connectivity. My ideas on the subject should be clear before I step into IPv6 territory especially since I've no access to IPv6 for testing purposes myself. One more thing: I need to know who will be able to test IPv6 support (I'm not able to). For testing, you'll need at least a Postfix 2.2 pre-release (not a Postfix with an unofficial IPv6 patch...) or 2.2.x when it's out and some IPv6 SMTP trafic. > > >>- can we blindly remove the last :hexa component or are there >>compression tricks in the representation that will prevent this? i.e.: >> . common case "2001:4f8:3:ba:2e0:81ff:fe22:d1f1" -> >>"2001:4f8:3:ba:2e0:81ff:fe22", >> . imaginary problematic case: "2001:4f8:3:ba:2e0::" -> >>"2001:4f8:3:ba:2e0:" (can it happen?). >> >> > >As far as I know, compression always involves "::", so >"2001:4f8:3:ba:2e0:" would never occur as shorthand for >"2001:4f8:3:ba:2e0::". > > In the example "2001:4f8:3:ba:2e0:" wasn't meant to be an IPv6 address but the result of SQLgrey "client identifier" computation. For IPv4 addresses it currently removes everything after the last ".", including the "." character itself. What I wanted to say is that if it simply removes everything after the last ":" in the IPv6 case and "2001:4f8:3:ba:2e0::" is a valid IPv6 address I'm expecting problems. On second thought the solution for this is obvious, just remove whatever comes after the ":" but don't remove the ":" character itself. I'm still wondering if there are some nasty things around I'm not expecting though. Lionel. |
|
From: Klaus A. S. <kse...@gm...> - 2005-03-10 07:14:29
|
On Sun, 06 Mar 2005 17:39:58 +0100, Lionel Bouton <lio...@bo...> wrote: >> I'm not sure. Many tunnel brokers hand out /64 or /48 nets to end >> users. E.g., I have [2001:1448:89::]/48 from ngdc.dk. My impression >> is that IPv6/64 is somewhat similar to IPv4/32 (I'm not sure if >> IPv6/48 should be compared to IPv4/24, though). but that is highly >> subjective. >=20 > I'm not sure IPv6/64 is the same as IPv4/32, I'll ask around as I know > some ADSL users with IPv6 connectivity. I didn't mean functionally equal to. What I meant was that an IPv6/64 seems to have the same "value" ("value" as in "exchange rate") as an IPv4/32. > One more thing: I need to know who will be able to test IPv6 support > (I'm not able to). For testing, you'll need at least a Postfix 2.2 > pre-release (not a Postfix with an unofficial IPv6 patch...) or 2.2.x > when it's out and some IPv6 SMTP trafic. I see that Wietse has just announced Postfix 2.2, so it might not take long before I'm able to do testing with a genuine Postfix 2.2 and IPv6. However, Wietse writes in his announcement that the IPv6 support is from third party patches, so chances are the tests will look the same with my Postfix 2.1.5 with Debian patches. Cheers, --=20 Klaus Alexander Seistrup SubZeroNet =B7 Copenhagen =B7 Denmark |
|
From: Klaus A. S. <kse...@gm...> - 2005-03-11 13:38:28
|
Well, SQLgrey dies on IPv6 connections (I have put a blank line after
each log line):
<snip>
new: 2001:878:100:12::136: und...@st... -> undisclosed@koldfront.=
dk
Warning: couldn't do query:
INSERT INTO connect (sender_name, sender_domain, src, rcpt,
first_seen) VALUES('undisclosed','stud.ku.dk','2001:878:100:12::136','undis=
cl...@ko...',
NOW()):
ERROR: value too long for type character varying(15), reconnecting to DB
</snip>
--=20
Klaus Alexander Seistrup
SubZeroNet =B7 Copenhagen =B7 Denmark
|
|
From: Klaus A. S. <kse...@gm...> - 2005-03-11 13:42:39
|
I wrote: > SQLgrey dies on IPv6 connections Will it break anything if I re-create the tables manually and have PostgreSQL use INET (or CIDR) for the IP address fields? --=20 Klaus Alexander Seistrup SubZeroNet =B7 Copenhagen =B7 Denmark |
|
From: Lionel B. <lio...@bo...> - 2005-03-11 15:48:32
|
Klaus Alexander Seistrup wrote the following on 11.03.2005 14:42 : >I wrote: > > > >>SQLgrey dies on IPv6 connections >> >> > >Will it break anything if I re-create the tables manually and have >PostgreSQL use INET (or CIDR) for the IP address fields? > > Probably not. But I'd advise to use VARCHAR(39) instead. I won't support smart and classc type algorithms with anything other than VARCHAR. Future automatic updates to the database might not apply cleanly on your database too... Lionel (burried under work). |
|
From: Klaus A. S. <kse...@gm...> - 2005-03-12 17:50:43
|
On Fri, 11 Mar 2005 16:48:30 +0100, Lionel Bouton <lio...@bo...> wrote: >> Will it break anything if I re-create the tables manually and have >> PostgreSQL use INET (or CIDR) for the IP address fields? >=20 > Probably not. > But I'd advise to use VARCHAR(39) instead. I won't support smart and > classc type algorithms with anything other than VARCHAR. I've had quite a few IPv6 connections in the past few days, so I decided to change the original VARCHAR(15) to VARCHAR(39). > Future automatic updates to the database might not apply cleanly on your > database too... Poor me... Thanks for your help. --=20 Klaus Alexander Seistrup SubZeroNet =B7 Copenhagen =B7 Denmark |
|
From: Klaus A. S. <kse...@gm...> - 2005-03-25 12:12:03
|
IPv6 issue:
The subroutine (at appr. line 711 in sqlgrey v1.5.4):
sub class_c {
return join('.', (split(/\./, shift))[0..2]);
}
causes a
"Use of uninitialized value in join or string at
/usr/local/sbin/sqlgrey line 711."
error message upon IPv6 connects.
--=20
Klaus Alexander Seistrup
SubZeroNet =B7 Copenhagen =B7 Denmark
|
|
From: Lionel B. <lio...@bo...> - 2005-03-25 13:26:31
|
Klaus Alexander Seistrup wrote the following on 25.03.2005 13:12 :
>IPv6 issue:
>
>The subroutine (at appr. line 711 in sqlgrey v1.5.4):
>
>sub class_c {
> return join('.', (split(/\./, shift))[0..2]);
>}
>
>causes a
>
>"Use of uninitialized value in join or string at
>/usr/local/sbin/sqlgrey line 711."
>
>error message upon IPv6 connects.
>
>
>
If I'm not mistaken it should be harmless (annoying but harmless). It
doesn't quit does it ? The corresponding class_c output should be
identical to its input, meaning the smart/classc algo will behave in the
same ways than 'full'.
You should use 'greymethod=full' with IPv6 until I figure out the best
strategy to implement smart and classc for IPv6.
Lionel.
|
|
From: Allan J. <al...@no...> - 2005-04-19 08:50:41
|
Lionel Bouton wrote, On 25-03-2005 14:26: > You should use 'greymethod=full' with IPv6 until I figure out the best > strategy to implement smart and classc for IPv6. What is the status of IPv6? -- Allan Joergensen - http://nowhere.dk/ "No matter what they're talking about, they're talking about money." |
|
From: Lionel B. <lio...@bo...> - 2005-04-19 10:52:38
|
Allan Joergensen wrote the following on 19.04.2005 10:50 : > Lionel Bouton wrote, On 25-03-2005 14:26: > >> You should use 'greymethod=full' with IPv6 until I figure out the best >> strategy to implement smart and classc for IPv6. > > > What is the status of IPv6? > In the pipe. You can make it work now with a simple VARCHAR column length modification and by using greymethod=full but it won't be optimal. Lionel. |
|
From: Lionel B. <lio...@bo...> - 2004-12-19 21:46:21
|
Lionel Bouton wrote the following on 12/19/04 22:04 : > > I'll ask on the Postfix mailing-list... > BTW, which Postfix version are you using ? It seems IPv6 is only available through patches. I've no answer yet from the Postfix mailing-list (just posted my questions) but it could mean that IPv6 isn't supported yet when calling a policy dameon. Until IPv6 is officialy supported, not advertising any IPv6 MX shouldn't cause any harm (all public IPv6 hosts I'm aware of do speak IPv4 as well...). Lionel. |
|
From: Klaus A. S. <kse...@gm...> - 2004-12-19 21:54:26
|
Lionel Bouton wrote: > BTW, which Postfix version are you using ? I'm using Postfix 2.1.5 from Debian/unstable (Ubuntu/hoary). It supports IPv6 out of the box (and I'm pretty sure that at least some earlier versions did as well). > It seems IPv6 is only available through patches. I've no answer yet from > the Postfix mailing-list (just posted my questions) but it could mean tha= t > IPv6 isn't supported yet when calling a policy dameon. I quite sure I've seen an IPv6 address in an earlier greylisting d=E6mon (I'm afraid it was PostGrey, though, but I haven't got any log files to support it). > Until IPv6 is officialy supported, not advertising any IPv6 MX shouldn't > cause any harm (all public IPv6 hosts I'm aware of do speak IPv4 as > well...). Either that, or just praying that spammers or viruses won't connect over IPv6. ;-) Cheers, --=20 Klaus Alexander Seistrup Copenhagen =B7 Denmark |
|
From: Lionel B. <lio...@bo...> - 2005-03-11 15:42:35
|
Klaus Alexander Seistrup wrote the following on 11.03.2005 14:38 : >Well, SQLgrey dies on IPv6 connections (I have put a blank line after >each log line): > > Not exactly, it reconnects to the database which puts unneeded load on both the DB and the SQLgrey host (if they are two different systems). In the end the resulting behavior is that greylisting is deactivated for IPv6. |
|
From: Klaus A. S. <kse...@gm...> - 2005-03-12 16:04:17
|
On Fri, 11 Mar 2005 16:42:30 +0100, Lionel Bouton <lio...@bo...> wrote: >> SQLgrey dies on IPv6 connections=20 >=20 > Not exactly, it reconnects to the database which puts unneeded load on > both the DB and the SQLgrey host (if they are two different systems). In > the end the resulting behavior is that greylisting is deactivated for IPv= 6. You are right, of course. It doesn't die. Cheers, --=20 Klaus Alexander Seistrup SubZeroNet =B7 Copenhagen =B7 Denmark |
|
From: Klaus A. S. <kse...@gm...> - 2005-03-25 13:53:29
|
Lionel Bouton wrote: >> "Use of uninitialized value in join or string at >> /usr/local/sbin/sqlgrey line 711." >=20 > If I'm not mistaken it should be harmless (annoying but harmless). It > doesn't quit does it ? You're right, it's harmless and SQLgrey doesn't quit. > You should use 'greymethod=3Dfull' with IPv6 until I figure out the best > strategy to implement smart and classc for IPv6. /me edits sqlgrey.conf Thanks! --=20 Klaus Alexander Seistrup SubZeroNet =B7 Copenhagen =B7 Denmark |
|
From: Klaus A. S. <kse...@gm...> - 2005-03-26 10:07:08
|
I wrote:
>>> "Use of uninitialized value in join or string at
>>> /usr/local/sbin/sqlgrey line 711."
>>
>> If I'm not mistaken it should be harmless (annoying but harmless). It
>> doesn't quit does it ?
>=20
> You're right, it's harmless and SQLgrey doesn't quit.
But the error messages are annoying in the logfile, so I've added a
line to the subroutine:
sub class_c {
return $_[0] unless $_[0] =3D~ /\./;
return join('.', (split(/\./, shift))[0..2]);
}
I don't speak perl, so please don't flame me for the un-prettiness. ;-)
--=20
Klaus Alexander Seistrup
SubZeroNet =B7 Copenhagen =B7 Denmark
|
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X