postfixadmin-devel — Postfix Admin development talk - users are also welcome

Re: [Postfixadmin-devel] Backupmx and redundant servers

[Christian B. <pos...@cb...>]

Hello,

Am Mittwoch, 24. November 2010 schrieb Oliver Schinagl:
> I am a happy user of postfixadmin, but have a question/comment.
> 
> I've gathered from the postfix mailing list that backupmx is
> practically dead. Dead in the sense that you can't just be a
> backupmx, you do have to actually know about the other hosts users
> or bounces, spam etc can become issues for your hosts.

Yes, indeed. 

OTOH, I could even say that I have a world-wide network of backup
mailservers - the sending servers cache the mail until they can deliver
it to my server *g*

Oh, in fact I have configured a backup MX for my domain. It's running
on a second IP on the same server, and only gives out temporary errors
to (in 99% of all connections) spammers. This takes away a nice amount 
of spam mails from the main IP that runs more checks.

The message my backup MX gives out is
    I'm a "Faulpelz", please use the primary MX
;-) (for those who don't understand german: Faulpelz means lazybones)

> With that in mind, i've changed my mailsetup so that both my
> mailhosts have identical users and alias tables. Domain tables
> already matched, with the difference in backupmx flag.
> 
> Also note, that one server isn't the main and the other the backup,
> but rather they each have their own domains, and are the backup for
> the others.
> 
> Ideally, I'd like to do all my postfixadministrating on one host, and
> synchronize the db to the other host (pg_dump -> psql import for
> example, whilst flipping the backupmx boolean). 

If you have only two servers, you could modify the queries in postfix 
("where backupmx = false" instead of "...= true") so that you don't have 
to modify the table.

That's a bit hacky, but easier to maintain ;-)

> However the GUI at
> the moment does not work for domains that have the backupmx flag.
> 
> I am unaware of a postfixadmin config setting that allows me to edit
> backupmx'ed hosts anyway. If postfixadmin would (optionally) allow
> editing of backupmx'ed hosts, then just copying the database will be
> ok and easy.

No, there isn't a config option for this.
It's a known problem. Unfortunately nobody has had the time to change it 
yet.

I just had a quick look at the code - maybe it's easy.

open functions.inc.php, search for
    function list_domains_for_admin ($username)

Near the beginning of the function, you'll find the following code:

    $backupmx_sql = db_get_boolean(False);
    $query = "SELECT $table_domain.domain, $table_domain_admins.username FROM $table_domain 
        LEFT JOIN $table_domain_admins ON $table_domain.domain=$table_domain_admins.domain 
        WHERE $table_domain_admins.username='$username' 
        AND $table_domain.active='$active_sql'
        AND $table_domain.backupmx='$backupmx_sql'    <------- remove this line                                                                                                                                           
        ORDER BY $table_domain_admins.domain";

Please remove the line I marked above and test if it works - or if it 
breaks something.
And please report back so that I know if this is something I can change
for future releases ;-)


Regards,

Christian Boltz
-- 
> 8.1 ist nicht 9.0
Also in der Quersumme schon ;-)
[> Henne Vogelsang und Manfred Tremmel in suse-linux]

Re: [Postfixadmin-devel] Need help packinging postfixadmin from SVN sources

[Christian B. <pos...@cb...>]

Hello,

Am Mittwoch, 24. November 2010 schrieb Juan Carlos Gutiérrez Deliot:
> I'm willing, for personal porpouse, to develop some new feature in
> postfixadmin, it's about exposing mailbox creation and password
> changing functionalities through web services in order to integrate
> a mail server with a portal.

Changing the password, the alias of a mailbox and switching vacaton 
on/off is already available via XMLRPC (but disabled by default in the 
config).

Mailbox creation over XMLRPC sounds like a good feature.

You should base your work on the files in the postfixadmin-cli directory 
(that's a quite new command-line interface for postfixadmin), especially 
on the model/* files. (Well, they should be merged with the "main" 
model/* files, but that's another story ;-)

postfixadmin-cli/model/UserHandler.php already contains functions to 
create a mailbox. Your task will be to
a) implement admin login over XMLRPC (using a mailbox login is pointless 
   when it comes to creating mailboxes ;-)
b) integrate the mailbox creation in the XMLRPC interface
c) make sure that you don't create any security holes ;-)

If you need some inspiration how to use the XMLRPC interface, check the 
squirrelmail plugin in the ADDITIONS directory.

> I downloaded the SVN version and got it ready to start developing in
> an IDE, eclipse for PHP developers (not connected to the official
> repository but my own one), but i don't know how to package the
> project in tarball (tar.gz) format in order to make a clean
> installation. 

Just take the SVN checkout (or your edited version of it) and make a 
tarball from it with tar czf.
In theory you could exclude the .svn directories from the tarball, but 
in practise they won't hurt except taking some disk space ;-)

> Also I wanted to ask if it's better to install a
> stable postfixadmin release and the upgrade it with the new features
> added by me (I don't even know how to do this). 

No, please take the SVN trunk as base. There have been lots of changes 
(most important: Smarty templates) so that it would mean lots of work if 
you base your changes on the 2.3.2 release and later want to integrate 
them in the SVN version.

BTW: Please keep the .svn directories in your development directory - 
they give you several advantages:
- you can run "svn up" to integrate changes done in the Postfixadmin SVN
  in your codebase. This will work automatically as long as you don't 
  edit the same line of code as we do.
- you can run "svn diff" to create a patch against "our" code. This will
  make integration in the official code easier.

Both are things that you don't want to do without the help of SVN ;-)

> Finally, if you think eclipse is not the right tool to work with
> postfixadmin, I hope you can introduce me to "the official"
> alternatives to work with.

David and I use vi, but I wouldn't go as far as calling it the "Official 
PostfixAdmin source code editor" ;-)
Seriously: take any editor or IDE you want and like.

We don't have something like coding guidelines, and for historical 
reasons we even have some files with 3 and some with 4 spaces as tab 
replacement. (New files should use 4 spaces.)

However there is a very important thing: If you need to copy more than 5 
lines of code from an existing file, please consider to put it into 
functions.inc.php as a function, and replace it with a function call at 
the original place. If you copy more than 20 lines, this is no longer 
optional ;-)
This might sound a bit strict, but I have some experience from merging 
copies of similar code. I started this merging when David and I took 
over the postfixadmin project - for some functions like "edit an alias" 
there were 4 copies of the code for the same job, with different bugs in 
each copy :-/  In the meantime I have removed/merged lots of the 
duplicates, but it's an ongoing job and still not fully done. For 
example "create $whatever" and "edit $whatever" have many things in 
common, the most visible difference is INSERT vs. UPDATE.
In short: I don't want to have even more code to compare and merge ;-))

> I hope you people can guide me through this, maybe with tutorials or
> even by personal contact. 

Personal contact as in "face to face meeting" might become difficult due 
to long distances - your name doesn't sound german ;-)

> I didn't even know to ask for this in the
> postfixadmin-devel mailing list or the svn mailing list, I just
> decided to start here and wait for some enlightment from you.

This mailinglist is the correct place.
The -svn list is read-only to distribute SVN commit messages and diffs.

Additionally you can join the #postfixadmin channel on the freenode IRC 
network. I'm usually available there in the evening (as seen from CET 
timezone), and David (nick "GingerDog") is there all the day. Be 
"warned" that you won't get an answer within one or two minutes - there 
are not too many people in the channel, and some of them (including 
David) are there 24/7, but obviously not always awake ;-)

> I hope I can share this feature with you when it's done and I'm open
> for any suggestions you might have.

Yes, this sounds like a good feature, and I'm quite sure we'll integrate 
it in postfixadmin when you have it working.


Regards,

Christian Boltz
-- 
(1) ZACKBOING: Geräusch von Stirn auf Monitor-Glasscheibe(2), direkte
Folge von Biosupgrades, selbst kompilierten Kerneln oder dem versehent-
lichen Löschen von 30 Gigabyte Pornofilmen bei einem MySQL-Upgrade.
(2) Deswegen kauft man heute oft Flachbildschirme. Die sind weicher.
[Ratti in suse-linux]

Re: [Postfixadmin-devel] Need help packinging postfixadmin for installation

[Juan C. G. D. <jua...@pe...>]

  On 24/11/2010 10:51 a.m., Juan Carlos Gutiérrez Deliot wrote:
>    Hello,
>
> I'm willing, for personal porpouse, to develop some new features in
> postfixadmin, it's about exposing mailbox creation and password changing
> functionalities through web services in order to integrate a mail server
> with a portal.
>
> I downloaded the SVN version and got it ready to start developing in an
> IDE, eclipse for PHP developers (not connected to the official
> repository but my own one), but i don't know how to package the project
> in tarball (tar.gz) format in order to make a clean installation. Also I
> wanted to ask if it's better to install a stable postfixadmin release
> and the upgrade it with the new features added by me (I don't even know
> how to do this). By the way, destination server for this development is
> Ubuntu 9.10.
> Finally, if you think eclipse is not the right tool to work with
> roundcube, I hope you can introduce me to "the official" alternatives to
> work with.
>
> I hope you people can guide me through this, maybe with tutorials or
> even by personal contact. I didn't even know to ask for this in the
> postfixadmin-devel mailing list or the svn mailing list, I just decided
> to start here and wait for some enlightment from you.
>
> I hope I can share this feature with you when it's done and I'm open for
> any suggestions you might have.
>
> Thank you very much.
>
> ------------------------------------------------------------------------------
> Increase Visibility of Your 3D Game App&  Earn a Chance To Win $500!
> Tap into the largest installed PC base&  get more eyes on your game by
> optimizing for Intel(R) Graphics Technology. Get started today with the
> Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
> http://p.sf.net/sfu/intelisp-dev2dev
> _______________________________________________
> Postfixadmin-devel mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postfixadmin-devel
Hello again,

I'm deeply sorry about the roundcube mistake, I'm also working with it 
and sometimes my tasks get mixed messing with my ideas.
Please ignore this post and let's focus in the other one.
Also if this thread can be erased, feel free to do it without hesitation.

Thanks.

[Postfixadmin-devel] Need help packinging postfixadmin for installation

[Juan C. G. D. <jua...@pe...>]

  Hello,

I'm willing, for personal porpouse, to develop some new features in 
postfixadmin, it's about exposing mailbox creation and password changing 
functionalities through web services in order to integrate a mail server 
with a portal.

I downloaded the SVN version and got it ready to start developing in an 
IDE, eclipse for PHP developers (not connected to the official 
repository but my own one), but i don't know how to package the project 
in tarball (tar.gz) format in order to make a clean installation. Also I 
wanted to ask if it's better to install a stable postfixadmin release 
and the upgrade it with the new features added by me (I don't even know 
how to do this). By the way, destination server for this development is 
Ubuntu 9.10.
Finally, if you think eclipse is not the right tool to work with 
roundcube, I hope you can introduce me to "the official" alternatives to 
work with.

I hope you people can guide me through this, maybe with tutorials or 
even by personal contact. I didn't even know to ask for this in the 
postfixadmin-devel mailing list or the svn mailing list, I just decided 
to start here and wait for some enlightment from you.

I hope I can share this feature with you when it's done and I'm open for 
any suggestions you might have.

Thank you very much.

[Postfixadmin-devel] Need help packinging postfixadmin from SVN sources

[Juan C. G. D. <jua...@pe...>]

Hello,

I'm willing, for personal porpouse, to develop some new feature in 
postfixadmin, it's about exposing mailbox creation and password changing 
functionalities through web services in order to integrate a mail server 
with a portal.

I downloaded the SVN version and got it ready to start developing in an 
IDE, eclipse for PHP developers (not connected to the official 
repository but my own one), but i don't know how to package the project 
in tarball (tar.gz) format in order to make a clean installation. Also I 
wanted to ask if it's better to install a stable postfixadmin release 
and the upgrade it with the new features added by me (I don't even know 
how to do this). By the way, destination server for this development is 
Ubuntu 9.10.
Finally, if you think eclipse is not the right tool to work with 
postfixadmin, I hope you can introduce me to "the official" alternatives 
to work with.

I hope you people can guide me through this, maybe with tutorials or 
even by personal contact. I didn't even know to ask for this in the 
postfixadmin-devel mailing list or the svn mailing list, I just decided 
to start here and wait for some enlightment from you.

I hope I can share this feature with you when it's done and I'm open for 
any suggestions you might have.

Thank you very much.

[Postfixadmin-devel] Backupmx and redundant servers

[Oliver S. <ol...@sc...>]

 Hey all,

I am a happy user of postfixadmin, but have a question/comment.

I've gathered from the postfix mailing list that backupmx is practically
dead. Dead in the sense that you can't just be a backupmx, you do have
to actually know about the other hosts users or bounces, spam etc can
become issues for your hosts.

With that in mind, i've changed my mailsetup so that both my mailhosts
have identical users and alias tables. Domain tables already matched,
with the difference in backupmx flag.

Also note, that one server isn't the main and the other the backup, but
rather they each have their own domains, and are the backup for the others.

Ideally, I'd like to do all my postfixadministrating on one host, and
synchronize the db to the other host (pg_dump -> psql import for
example, whilst flipping the backupmx boolean). However the GUI at the
moment does not work for domains that have the backupmx flag.

I am unaware of a postfixadmin config setting that allows me to edit
backupmx'ed hosts anyway. If postfixadmin would (optionally) allow
editing of backupmx'ed hosts, then just copying the database will be ok
and easy.

Oliver

Re: [Postfixadmin-devel] warning by create user over domain quota /solved

[Robert S. <ro...@sc...>]

Am 19.10.2010 15:31, schrieb Robert Schetterer:
> Hi @ll
> i am wondering, that i am still able to create additional users
> as admin when there is no place from domain quota left
> should nt there be a warning , and a user create denied ?
> version  2.3.2 on ubuntu lucid server using dovecot 2.0.5
> 
sorry my fault , i missunderstood config entries

-- 
Best Regards

MfG Robert Schetterer

Germany/Munich/Bavaria

[Postfixadmin-devel] warning by create user over domain quota

[Robert S. <ro...@sc...>]

Hi @ll
i am wondering, that i am still able to create additional users
as admin when there is no place from domain quota left
should nt there be a warning , and a user create denied ?
version  2.3.2 on ubuntu lucid server using dovecot 2.0.5

-- 
Best Regards

MfG Robert Schetterer

Germany/Munich/Bavaria

Re: [Postfixadmin-devel] Website: Create /screenshots/index.html

[David G. <da...@co...>]

On Fri, 2010-09-10 at 17:45 +0200, Jakob Unterwurzacher wrote:
> I thought the screenshots section of the website
> ( http://postfixadmin.sourceforge.net/screenshots/ ) could use some
> polishing :)
> 


Done; thank you...

David.

[Postfixadmin-devel] Website: Create /screenshots/index.html

[Jakob U. <jak...@gm...>]

I thought the screenshots section of the website
( http://postfixadmin.sourceforge.net/screenshots/ ) could use some
polishing :)

Thanks,
Jakob

Re: [Postfixadmin-devel] PostfixAdmin 2.3.2 ready for packaging

[Christian B. <pos...@cb...>]

Hello,

Am Montag, 23. August 2010 schrieb Tanstaafl:
> Christian Boltz wrote:
> > I've just checked in a changelog update and a new version number in
> > the 2.3.2 branch. This means the 2.3.2 release (aka SVN r858) is
> > ready from my POV - no critical bugs left :-)

As you can see in my other mail: never say a release is ready before it 
was uploaded ;-)

SVN r860 is the new 2.3.2 - and I'll hopefully do the RPM packaging fast 
enough before someone finds another bug *g*

> What about the bug about Vacation removal not clearing out
> notifications? I consider that a critical bug (maybe not ultra
>  critical, but important nonetheless)...

The problem about this bug is that it needs some time for investigation, 
which I don't have at the moment [1]. Therefore I prefer to do a release 
with one missing bugfix over delaying the release for some weeks.

(As always: hints why the existing DELETE query doesn't work are 
welcome, patches even more ;-)


Regards,

Christian Boltz

[1] We have a "Weinfest" (wine festival) here in Insheim from friday 
    till monday, and I'm involved in planning, preparations etc. ...
-- 
Hochleistungswebspace
Das sind public-html-Verzeichnisse, die jeden Morgen zwanzig Liegestütze
machen, und mit Testosteron vollgepumpt sind.  [Markus Schaber]

Re: [Postfixadmin-devel] PostfixAdmin 2.3.2 ready for packaging

[Christian B. <pos...@cb...>]

Hallo,

You might have seen the following forum post:

http://sourceforge.net/projects/postfixadmin/forums/forum/676076/topic/3822301:

SQL injection:
https://psfx.domain.com/list-domain.php?username=ma...@ma...'

This was posted early enough - I did not upload the 2.3.2 release 
yet ;-)

I have fixed it in the 2.3 branch - please re-create the 2.3.2 tarball 
and .deb based on SVN r860.

Am Montag, 23. August 2010 schrieb David Goodwin:
> Not overly sure Evolution will gpg sign this correctly...

Looks good, however a *.asc file with a signature for each file (or for 
just the tarball if the .deb already includes a GPG signature) would 
probably be more useful IMHO.


Regards,

Christian Boltz
-- 
Swapping horses in the middle of a river... ah no the wrong citate...
changing package managers in the middle of a beta was a very bad
decision, specifically as it was not read for production use when
being introduced. [Ulrich Windl in opensuse]

Re: [Postfixadmin-devel] PostfixAdmin 2.3.2 ready for packaging

[Tanstaafl <tan...@li...>]

Christian Boltz wrote:
> Hello,
> 
> I've just checked in a changelog update and a new version number in the 
> 2.3.2 branch. This means the 2.3.2 release (aka SVN r858) is ready from 
> my POV - no critical bugs left :-)

What about the bug about Vacation removal not clearing out
notifications? I consider that a critical bug (maybe not ultra critical,
but important nonetheless)...

Otherwise... thanks to Christian and David (and everyone else who
contribs) for postfixadmin!

Re: [Postfixadmin-devel] Sicherheitshinweis für Postfixadmin

[Sebastian S. <seb...@vi...>]

Hi David,

On Aug 23, 2010, at 12:52 PM, David Goodwin wrote:
> I understand it's technically a vulnerability, but it's not new -
> similar vulnerabilities have been encountered before in other web
> applications - I'd argue that e.g. Facebook's recent bug whereby it's
> possible to discover if an email address is valid (well - known to
> facebook as a login) and get someone's full name out of it is more
> concerning. 

Indeed, "side channel" vulnerabilities are well known, but I have developed
a new method that detects "hidden" side channels.
The paper is really more about the formal implication for the detection and
prevention of side channels, and less about the vulnerabilities itself.

Drop me a line off-list if anyone is interested in a copy of the paper as soon
as it is published.

> Anyway, thank you for reporting it - Christian has fixed the problem it
> seems - and release 2.3.2 will be finding it's way onto the internet
> shortly ... I've packaged up the .deb / .tar.gz and Christian will be
> pushing them and an .rpm to sourceforge shortly.
> 
> Please feel free to continue prodding Postfixadmin, and we welcome all
> feedback and bug reports...  and patches even more so! 

I will. Keep up the good work!

Cheers,
Sebastian

Re: [Postfixadmin-devel] Sicherheitshinweis für Postfixadmin

[David G. <da...@co...>]

> 
> On Aug 22, 2010, at 2:37 PM, Christian Boltz wrote:
> > Hallo Herr Schinzel,
> > 
> > (@David: please use google translate de -> en ;-)
> > 
> 
> 
> Sorry for writing in German. Christian's name is clearly german
> and it did not come to me that non-germans could be involved.
> 


Don't worry; I have a Spanish friend who keeps me using
http://translate.google.com :-) it's no great problem. Anyway, Christian
is more active than I in the project at the moment, so you did at least
choose correctly :)

(As a side note, Christian is a common English name, yet there do indeed
appear to be a lack of Boltz's in the telephone book - e.g.
http://www.thephonebook.bt.com/publisha.content/en/search/residential/search.publisha?Surname=boltz&x=0&y=0&Location=LONDON&OriginalLocation=london&Range=xloc )
Christian - why don't your family emigrate and spread around the world a
bit more? :-) )


Anyway, back on track :

My only input is that a similar bug existed before - namely if you had
an incorrect password postfixadmin displayed a slightly different
message (e.g. invalid password) rather than the more secure "Invalid
username and/or password". I think this was on the user login page. We
did fix this, but it can be argued to be useful to tell the end user
that their username is correct but password isn't. 

Ignorance would make me think that most admin's are going to be in the
system as 'admin@$domain_name' or 'support@$domain_name' - so it may not
be hard to guess correctly anyway. I tend to always install postfixadmin
behind an Apache password prompt thing anyway - just for additional
protection.


I understand it's technically a vulnerability, but it's not new -
similar vulnerabilities have been encountered before in other web
applications - I'd argue that e.g. Facebook's recent bug whereby it's
possible to discover if an email address is valid (well - known to
facebook as a login) and get someone's full name out of it is more
concerning. 

What would worry me far more would be if we somehow echo'ed out the
hashed password in the HTML returned, or had an SQL / XSS injection or
arbitrary code execution security hole :)

Anyway, thank you for reporting it - Christian has fixed the problem it
seems - and release 2.3.2 will be finding it's way onto the internet
shortly ... I've packaged up the .deb / .tar.gz and Christian will be
pushing them and an .rpm to sourceforge shortly.

Please feel free to continue prodding Postfixadmin, and we welcome all
feedback and bug reports...  and patches even more so! 


thanks,

David.

[Postfixadmin-devel] PostfixAdmin 2.3.2 ready for packaging

[Christian B. <pos...@cb...>]

Hello,

I've just checked in a changelog update and a new version number in the 
2.3.2 branch. This means the 2.3.2 release (aka SVN r858) is ready from 
my POV - no critical bugs left :-)

@all: please test the 2.3 branch if everything works.

@David: please create a tarball and the deb file and send it to me. I'll 
then create the RPM package and upload everything to SF.


Regards,

Christian Boltz
-- 
Das hätte man auch kürzer sagen können:
| "Please don't use evolution anymore. It's not intended as a
|  mailingprogramm, we're just riding around a little bit on our
|  C-Compilers to find out how to break the rules."
Yes, sir. Nice outlooks.      [Ratti in suse-linux]

Re: [Postfixadmin-devel] Debian package should depend on alternative "httpd"

[Christian B. <pos...@cb...>]

Hello,

Am Donnerstag, 29. Juli 2010 schrieb GW:
> While trying to use Postfixadmin 2.3.1 (and older) Debian packages I
> noticed that it depends only on "apache2 | lighttpd" and this would
> install a bounch on unnecessary programs if I hadn't changed the
> dependency to "apache2 | lighttpd | httpd". The virtual package
>  "httpd" is used by nginx and similar and should be added as an
>  alternative.

OK, done in the 2.3 branch (maybe you've seen my commit yesterday). 
PostfixAdmin 2.3.2 will accept httpd ;-)  (and I'll of course also do 
this change in trunk after the 2.3.2 release).


Regards,

Christian Boltz
-- 
Zeichenverarbeitung in der Shell ist _evil_. Verwende eine P-Sprache
(Perl, PHP, Python) für Dein Problem. Das ist _sehr_ viel effektiver.
Deine Frage zeigt an, daß Du gerade das Programmieräquivalent zu einem
Hammer für Schrauben verwendest. [Kristian Köhntopp in suse-linux]

[Postfixadmin-devel] Debian package should depend on alternative "httpd"

[GW <gw...@tn...>]

Hi,

While trying to use Postfixadmin 2.3.1 (and older) Debian packages I
noticed that it depends only on "apache2 | lighttpd" and this would
install a bounch on unnecessary programs if I hadn't changed the
dependency to "apache2 | lighttpd | httpd". The virtual package "httpd"
is used by nginx and similar and should be added as an alternative.

Greetings,
  gw

Re: [Postfixadmin-devel] SF.net SVN: postfixadmin:[824] branches/postfixadmin-2.3

[Christian B. <pos...@cb...>]

Hello,

Am Sonntag, 27. Juni 2010 schrieb reg9009:
> Well, basically it does the same. Of course, frameworks always carry
>  a  lot of functionality which isn't fully used across the projects
>  using the framework. 

>  Nowadays, the overhead generated by frameworks
>  is compensated by the computing power growing continously.

Well, that were two arguments from you against using a framework *g*

BTW: I prefer to use the growing computing power to make things faster 
instead of just keeping the speed ;-)

> I really like schema update stuff. The update mechanism of the
>  database  structure gets transparent regardless of the rdbms using
>  it. Adding/removing/changing fields, indexes, etc. seems a lot
>  easier for me (update.php).

OK, that _might_ be an argument. But IIRC there's also something like a 
standalone "database scheme updater" part in Doctrine, so this isn't a 
argument to use Doctrine in the everyday code *g*

OTOH, upgrade.php already has some abstraction included - it's just that 
it isn't always used ;-)  (But yes, there are really things that are 
different on PostgreSQL and MySQL.)

> It's always a question of what to use and if some kind of framework 
> would ease the use in general. Especially regarding data management
>  I  tend to go for frameworks nowadays. This also frees up some kind
>  of ressources later to implement theme styles :) I would suggest we
>  start and open a new branch and have a look...

I have another proposal - do it in multiple steps directly in trunk.


* Step 1: convert INSERT, UPDATE and DELETE queries to db_* calls

Switching to a database framework requires lots of work on changing how 
database calls are done - we have to change all INSERT, UPDATE and 
DELETE queries to use db_insert(), db_update() and db_delete.

This is something that should be done in any case, and it would also be 
the base for switching to a framework (aka "no time wasted").

Target: The code should be free of UPDATE, INSERT and DELETE queries 
(except inside the db_* functions of course)

(Sidenote: Ideally do this only for the edit*.php scripts and merge them 
with create*.php afterwards. See fetchmail.php for an example how to do 
this. The advantage is that we have 50% less code to maintain in the 
future ;-)

BTW: regarding the timestamp parameters: Maybe db_update and db_insert 
should know the "usual" field names ("created" and "modified" are in 
nearly every table) so that the calling code does not even need to 
specify them...


* Step 2: remove usage of $CONF['database_type']

After step 1, we'll only have some SELECT queries left that are database 
dependent. 

We should wrap those parts in functions that create (parts of) a SELECT 
query - for example the LIMIT clause.
Writing a general db_select function isn't that easy because SELECT 
queries can become quite interesting[tm]. However such a function could 
still be useful for the 90% "simple" queries.

My first look at the Doctrine documentation didn't show me functions to 
build select queries, so this also isn't wasted time.

Target: The code (except the db_* functions) should not contain any 
condition based on $CONF['database_type'].


* Step 3: check if we really need the framework

We should now (after step 1 and 2) already have 
- lots of db_* calls in the code instead of "hardcoded" queries
- nearly no direct db_query calls (except some complex SELECT queries)
- all database specific stuff inside the db_* functions

That's already lots of abstraction and code cleanup.

We'll then see if this is enough or if we really want/need to introduce 
a framework like Doctrine.


* final notes

IMHO even with the framework we have to keep the db_* functions to do 
the postfixadmin-specific work (mapping table names, timestamps etc. - 
see my previous mail for details).

Within the db_* functions, we can call the framework if we decide to go 
this way.


The good thing about this 3-step-proposal is (IMHO) that it can be done 
directly in trunk without breaking anything. No need to fiddle with 
branch handling ;-)  (I can tell you some stories about that regarding 
the 2.3 branch...)

> Oralce was a bad example, but e.g. sqlite.

;-)


Regards,

Christian Boltz
-- 
Error: File not found -- search behind couch? (Y/N)

Re: [Postfixadmin-devel] SF.net SVN: postfixadmin:[824] branches/postfixadmin-2.3

[reg9009 <re...@ya...>]

  Am 26.06.2010 23:29, schrieb Christian Boltz:
> Hello,
>
> Am Samstag, 26. Juni 2010 schrieb reg9009:
>> we are planning to introduce some kind of Object Relational Mapper to
>> manage the database stuff. I think there were some discussions about
>>   it earlier. Anyway, we wanted to give Doctrine Project a final try.
>>   This would theoretically extend the RDBMSs support without any
>>   further intervention (e.g. Oracle, etc.). Any thoughts on this?
> Call me an enemy of frameworks, but I don't think Doctrine or any other
> database abstraction framework would be useful for postfixadmin.
>
> We have some code in functions.inc.php to compose queries and parts of
> queries (db_insert, db_update, db_delete, db_get_boolean, ... - grep for
> "function db_" to get the full list) and also do the abstraction
> regarding PostgreSQL vs. MySQL where needed.
>
> I'll take db_insert as an example - you just give it the (default) table
> name, an array(
>      'field_1' =>  'value_1',
>      'field_2' =>  'value_2',
> )
> and optionally an array with timestamp fields that should be set to
> NOW().
> The function then converts the table name[1], generates an INSERT query
> from the data array (including escaping the values) and the timestamp
> fields and executes the query.
>
> Add a condition that can be used for WHERE (address = 'fo...@do...d')
> and you have the parameters for db_update.
>
> As you can see, postfixadmin already has its own database abstraction.
> The main problem is that the "old" code doesn't use the db_* functions,
> but at least the newer code does. Needless to say that it would be a
> good idea to make the "old" code using these functions also. [5]
>
>
> If I get the Doctrine documentation right [2], it does basically the
> same ($conn->insert) with more code - I didn't read the framework code,
> but I'm quite sure db_insert with its 10 or 15 lines of code is shorter
> than the code in the framework ;-)
>
> And (of course) Doctrine doesn't handle some things that are specific to
> postfixadmin:
> - the timestamp fields that are set to NOW()
> - the table name conversion [1]
> - escaping of all fields (except the timestamps)
>
> That means we would need to have a wrapper to handle those things.
> "Just" the framework would be a step back.
>
> A wrapper around a framework sounds interesting[tm] - KISS anyone? ;-)
>
>
> Some questions my short look at the Doctrine documentation didn't
> answer:
> - how are boolean values handled? (PostgreSQL uses t/f, MySQL uses 1/0)
> - how is LIMIT handled? PostgreSQL has a different syntax... [3]
>
>
> My vote clearly goes to using the db_* functions everywhere instead of
> introducing a framework with 90% code that we never use.
> Of course you can change my opinion if you have some good arguments why
> the framework is better than the db_* functions. However "$framework is
> cool" or "Oracle support" [4] aren't real arguments for me ;-)
>
>
> Regards,
>
> Christian Boltz
>
> [1] $CONF['database_prefix'] and $CONF['database_tables'], conversion
>      is done in the table_by_key($table) function
>
> [2] http://www.doctrine-project.org/projects/dbal/2.0/docs/
>      reference/data-retrieval-and-manipulation/en
>      (as one line)
>
> [3] MySQL: LIMIT $fDisplay, $page_size
>      PostgreSQL: LIMIT $page_size OFFSET $fDisplay
>
>      I have to admit that this is hardcoded in some places - I'll
>      probably have to write a small db_limit function or a function to
>      compose SELECT queries.
>      In short: Every code section that checks for $CONF['database_type']
>      outside the db_* functions should be called a bug IMHO.
>
>      And I just found in the MySQL documentation:
>          "For compatibility with PostgreSQL, MySQL also supports the
>           LIMIT row_count OFFSET offset syntax."
>      Good to know ;-)
>
> [4] I doubt people will run Oracle for their mailserver, and AFAIK
>      postfix doesn't support Oracle at all ;-)
>
> [5] This should also result in merging the "create" and "edit" pages -
>      the only differences are the database call (insert or update) and
>      that one field (typically the address) has to be read-only on edit.
>      fetchmail.php can serve as an example how it should like.
>
Well, basically it does the same. Of course, frameworks always carry a 
lot of functionality which isn't fully used across the projects using 
the framework. Nowadays, the overhead generated by frameworks is 
compensated by the computing power growing continously.

I really like schema update stuff. The update mechanism of the database 
structure gets transparent regardless of the rdbms using it. 
Adding/removing/changing fields, indexes, etc. seems a lot easier for me 
(update.php).

It's always a question of what to use and if some kind of framework 
would ease the use in general. Especially regarding data management I 
tend to go for frameworks nowadays. This also frees up some kind of 
ressources later to implement theme styles :) I would suggest we start 
and open a new branch and have a look...

Oralce was a bad example, but e.g. sqlite.

Regards,
Sebastian

Re: [Postfixadmin-devel] SF.net SVN: postfixadmin:[824] branches/postfixadmin-2.3

[Valkum <va...@go...>]

Am 26.06.2010 23:29, schrieb Christian Boltz:
> Hello,
>
> Am Samstag, 26. Juni 2010 schrieb reg9009:
>   
>> we are planning to introduce some kind of Object Relational Mapper to
>> manage the database stuff. I think there were some discussions about
>>  it earlier. Anyway, we wanted to give Doctrine Project a final try.
>>  This would theoretically extend the RDBMSs support without any
>>  further intervention (e.g. Oracle, etc.). Any thoughts on this?
>>     
> Call me an enemy of frameworks, but I don't think Doctrine or any other 
> database abstraction framework would be useful for postfixadmin.
>
> We have some code in functions.inc.php to compose queries and parts of 
> queries (db_insert, db_update, db_delete, db_get_boolean, ... - grep for 
> "function db_" to get the full list) and also do the abstraction 
> regarding PostgreSQL vs. MySQL where needed.
>
> I'll take db_insert as an example - you just give it the (default) table 
> name, an array(
>     'field_1' => 'value_1',
>     'field_2' => 'value_2',
> )
> and optionally an array with timestamp fields that should be set to 
> NOW().
> The function then converts the table name[1], generates an INSERT query 
> from the data array (including escaping the values) and the timestamp 
> fields and executes the query.
>
> Add a condition that can be used for WHERE (address = 'fo...@do...d') 
> and you have the parameters for db_update.
>
> As you can see, postfixadmin already has its own database abstraction. 
> The main problem is that the "old" code doesn't use the db_* functions, 
> but at least the newer code does. Needless to say that it would be a 
> good idea to make the "old" code using these functions also. [5]
>
>
> If I get the Doctrine documentation right [2], it does basically the 
> same ($conn->insert) with more code - I didn't read the framework code, 
> but I'm quite sure db_insert with its 10 or 15 lines of code is shorter 
> than the code in the framework ;-)
>
> And (of course) Doctrine doesn't handle some things that are specific to 
> postfixadmin:
> - the timestamp fields that are set to NOW()
> - the table name conversion [1]
> - escaping of all fields (except the timestamps)
>
> That means we would need to have a wrapper to handle those things. 
> "Just" the framework would be a step back.
>
> A wrapper around a framework sounds interesting[tm] - KISS anyone? ;-)
>
>
> Some questions my short look at the Doctrine documentation didn't 
> answer:
> - how are boolean values handled? (PostgreSQL uses t/f, MySQL uses 1/0)
> - how is LIMIT handled? PostgreSQL has a different syntax... [3]
>
>
> My vote clearly goes to using the db_* functions everywhere instead of 
> introducing a framework with 90% code that we never use.
> Of course you can change my opinion if you have some good arguments why 
> the framework is better than the db_* functions. However "$framework is 
> cool" or "Oracle support" [4] aren't real arguments for me ;-)
>
>
> Regards,
>
> Christian Boltz
>
> [1] $CONF['database_prefix'] and $CONF['database_tables'], conversion 
>     is done in the table_by_key($table) function
>
> [2] http://www.doctrine-project.org/projects/dbal/2.0/docs/
>     reference/data-retrieval-and-manipulation/en
>     (as one line)
>
> [3] MySQL: LIMIT $fDisplay, $page_size
>     PostgreSQL: LIMIT $page_size OFFSET $fDisplay
>
>     I have to admit that this is hardcoded in some places - I'll 
>     probably have to write a small db_limit function or a function to 
>     compose SELECT queries.
>     In short: Every code section that checks for $CONF['database_type']
>     outside the db_* functions should be called a bug IMHO.
>
>     And I just found in the MySQL documentation: 
>         "For compatibility with PostgreSQL, MySQL also supports the  
>          LIMIT row_count OFFSET offset syntax."
>     Good to know ;-)
>
> [4] I doubt people will run Oracle for their mailserver, and AFAIK 
>     postfix doesn't support Oracle at all ;-)
>
> [5] This should also result in merging the "create" and "edit" pages -
>     the only differences are the database call (insert or update) and
>     that one field (typically the address) has to be read-only on edit.
>     fetchmail.php can serve as an example how it should like.
>
>   
As i know Doctrine add NOW() to all cols which are called updated or
timestamp or so on.
If Doctrine is too unspeacial for PFA the way to fo to OOP is really
useful in my eys. Writing our own DB Abstr.  layer in OOP makes the Code
cleaner and Development easier.
The Way to seperate our DB Layer from UI is so quiet simple and required
to add new technics..

Re: [Postfixadmin-devel] SF.net SVN: postfixadmin:[824] branches/postfixadmin-2.3

[Christian B. <pos...@cb...>]

Hello,

Am Samstag, 26. Juni 2010 schrieb reg9009:
> we are planning to introduce some kind of Object Relational Mapper to
> manage the database stuff. I think there were some discussions about
>  it earlier. Anyway, we wanted to give Doctrine Project a final try.
>  This would theoretically extend the RDBMSs support without any
>  further intervention (e.g. Oracle, etc.). Any thoughts on this?

Call me an enemy of frameworks, but I don't think Doctrine or any other 
database abstraction framework would be useful for postfixadmin.

We have some code in functions.inc.php to compose queries and parts of 
queries (db_insert, db_update, db_delete, db_get_boolean, ... - grep for 
"function db_" to get the full list) and also do the abstraction 
regarding PostgreSQL vs. MySQL where needed.

I'll take db_insert as an example - you just give it the (default) table 
name, an array(
    'field_1' => 'value_1',
    'field_2' => 'value_2',
)
and optionally an array with timestamp fields that should be set to 
NOW().
The function then converts the table name[1], generates an INSERT query 
from the data array (including escaping the values) and the timestamp 
fields and executes the query.

Add a condition that can be used for WHERE (address = 'fo...@do...d') 
and you have the parameters for db_update.

As you can see, postfixadmin already has its own database abstraction. 
The main problem is that the "old" code doesn't use the db_* functions, 
but at least the newer code does. Needless to say that it would be a 
good idea to make the "old" code using these functions also. [5]


If I get the Doctrine documentation right [2], it does basically the 
same ($conn->insert) with more code - I didn't read the framework code, 
but I'm quite sure db_insert with its 10 or 15 lines of code is shorter 
than the code in the framework ;-)

And (of course) Doctrine doesn't handle some things that are specific to 
postfixadmin:
- the timestamp fields that are set to NOW()
- the table name conversion [1]
- escaping of all fields (except the timestamps)

That means we would need to have a wrapper to handle those things. 
"Just" the framework would be a step back.

A wrapper around a framework sounds interesting[tm] - KISS anyone? ;-)


Some questions my short look at the Doctrine documentation didn't 
answer:
- how are boolean values handled? (PostgreSQL uses t/f, MySQL uses 1/0)
- how is LIMIT handled? PostgreSQL has a different syntax... [3]


My vote clearly goes to using the db_* functions everywhere instead of 
introducing a framework with 90% code that we never use.
Of course you can change my opinion if you have some good arguments why 
the framework is better than the db_* functions. However "$framework is 
cool" or "Oracle support" [4] aren't real arguments for me ;-)


Regards,

Christian Boltz

[1] $CONF['database_prefix'] and $CONF['database_tables'], conversion 
    is done in the table_by_key($table) function

[2] http://www.doctrine-project.org/projects/dbal/2.0/docs/
    reference/data-retrieval-and-manipulation/en
    (as one line)

[3] MySQL: LIMIT $fDisplay, $page_size
    PostgreSQL: LIMIT $page_size OFFSET $fDisplay

    I have to admit that this is hardcoded in some places - I'll 
    probably have to write a small db_limit function or a function to 
    compose SELECT queries.
    In short: Every code section that checks for $CONF['database_type']
    outside the db_* functions should be called a bug IMHO.

    And I just found in the MySQL documentation: 
        "For compatibility with PostgreSQL, MySQL also supports the  
         LIMIT row_count OFFSET offset syntax."
    Good to know ;-)

[4] I doubt people will run Oracle for their mailserver, and AFAIK 
    postfix doesn't support Oracle at all ;-)

[5] This should also result in merging the "create" and "edit" pages -
    the only differences are the database call (insert or update) and
    that one field (typically the address) has to be read-only on edit.
    fetchmail.php can serve as an example how it should like.

-- 
Und der erste, der mich darauf hinweist, ich könne das ja selbst
umkonfigurieren, ich müsse nur die 200KB-Doku lesen, den besuche
ich zu Hause und *singe*(!) ihm den Sourcecode vor,  und DAS ist
WIRKLICH eine STRAFE. [Ratti in suse-linux]

Re: [Postfixadmin-devel] SF.net SVN: postfixadmin:[824] branches/postfixadmin-2.3

[David G. <da...@co...>]

>> 
> Hi all,
> 
> we are planning to introduce some kind of Object Relational Mapper to 
> manage the database stuff. I think there were some discussions about it 
> earlier. Anyway, we wanted to give Doctrine Project a final try. This 
> would theoretically extend the RDBMSs support without any further 
> intervention (e.g. Oracle, etc.). Any thoughts on this?
> 


Sure. I sort of wanted to do this with the model directory. But I don't have time / motivation. 

David

Re: [Postfixadmin-devel] SF.net SVN: postfixadmin:[824] branches/postfixadmin-2.3

[Valkum <va...@go...>]

Am 26.06.2010 11:10, schrieb reg9009:
>   Am 23.06.2010 00:49, schrieb Christian Boltz:
>   
>> Hello,
>>
>> good news: we are near the 2.3.1 release [1] - at least that's what I
>> think ;-)
>>
>> The code is not bug-free (probably it will never be), but all major bugs
>> should be fixed. (And of course there will always be someone saying that
>> his "favorite" bug is still unfixed ;-)
>>
>> Missing parts:
>> - merge debian/* changes from r805 ->  normes? hello? ;-)
>> - testing - I did not even test the code after the merge I commited some
>>    minutes ago ;-)
>>
>> Still very important:
>>     
>>> @everybody: Please checkout the 2.3 branch from SVN and check if
>>> everything works. Especially check if everything listed in the
>>> CHANGELOG.txt works ;-)
>>>       
>> @GingerDog:
>> Notes regarding the release:
>> a) version an revision numbers in changelog and functions.inc.php need
>>     to be updated
>> b) Please send me the tarball in advance so I can create a RPM package
>>     _before uploading to SF_.
>>     The reason for this request is the "latest files" section on SF,
>>     which makes uploading the RPM later a no-go :-/ - we have to upload
>>     everything at once.
>>
>>
>> Regards,
>>
>> Christian Boltz
>>
>> [1] if we manage to do the release before someone finds another major
>>      bug *g*
>>
>>     
> Hi all,
>
> we are planning to introduce some kind of Object Relational Mapper to 
> manage the database stuff. I think there were some discussions about it 
> earlier. Anyway, we wanted to give Doctrine Project a final try. This 
> would theoretically extend the RDBMSs support without any further 
> intervention (e.g. Oracle, etc.). Any thoughts on this?
>
> Regards,
> Sebastian
>   

I think this is a good idea. In this way we can build up a manage
librarie and develop some frontends like the web front end and the
console frontemd.
So the web FE and the Lib should be fully standalone or so.

greetz
Rudi

Re: [Postfixadmin-devel] SF.net SVN: postfixadmin:[824] branches/postfixadmin-2.3

[reg9009 <re...@ya...>]

  Am 23.06.2010 00:49, schrieb Christian Boltz:
> Hello,
>
> good news: we are near the 2.3.1 release [1] - at least that's what I
> think ;-)
>
> The code is not bug-free (probably it will never be), but all major bugs
> should be fixed. (And of course there will always be someone saying that
> his "favorite" bug is still unfixed ;-)
>
> Missing parts:
> - merge debian/* changes from r805 ->  normes? hello? ;-)
> - testing - I did not even test the code after the merge I commited some
>    minutes ago ;-)
>
> Still very important:
>> @everybody: Please checkout the 2.3 branch from SVN and check if
>> everything works. Especially check if everything listed in the
>> CHANGELOG.txt works ;-)
>
> @GingerDog:
> Notes regarding the release:
> a) version an revision numbers in changelog and functions.inc.php need
>     to be updated
> b) Please send me the tarball in advance so I can create a RPM package
>     _before uploading to SF_.
>     The reason for this request is the "latest files" section on SF,
>     which makes uploading the RPM later a no-go :-/ - we have to upload
>     everything at once.
>
>
> Regards,
>
> Christian Boltz
>
> [1] if we manage to do the release before someone finds another major
>      bug *g*
>
Hi all,

we are planning to introduce some kind of Object Relational Mapper to 
manage the database stuff. I think there were some discussions about it 
earlier. Anyway, we wanted to give Doctrine Project a final try. This 
would theoretically extend the RDBMSs support without any further 
intervention (e.g. Oracle, etc.). Any thoughts on this?

Regards,
Sebastian