slashcode-development — This list is for development of slashcode, no install questions please.

[Slashcode-development] RFC: SQL Placeholders

[Stephen C. <the...@wa...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

As of late I have been working on getting Slash2 to work with Oracle (going
ahead on my own since there wasn't much interest (read: none) a while back when
I asked).  So far it's been quite successful -- the initial database creates and
loads without error.  Now it's down to messing with the DB interface 
itself.  The biggest show-stopper so far is while loading the templates in:

DBD::Oracle::db do failed: ORA-01704: string literal too long

because it's trying to embed the entire string into the SQL statement, which you
can't do in Oracle if it's over 4000 bytes (the boundary between VARCHAR and
LOB).  The variable has to be explicitly typed as a LOB when preparing the
statement, and that requires the use of DBI placeholders for bind 
values.  However, there isn't a single placeholder in the current code.  I found
this rather surprising.

Aside from the obvious technical issue above, there's a performance one as
well.  Oracle isn't able to cache any of the SQL with the variables embedded
inside, so performance will suffer when I finally do manage to get it
working.  This is actually an issue with MySQL as well, since Slash2 currently
makes quite a bit of use of prepare_cached, a use that's practically worthless
(and a large waste of memory) without using placeholders.

My proposal is to get the SQL internals using placeholders and bind values to
facilitate current and future porting, as well as making the current MySQL core
substantially cleaner and more memory-efficient.  I've spent all day looking
over the Slash::DB::Utility interface and the sql* methods and I've devised an
extension to the current interface that would make the changes to the current
code minimal (an additional argument to sql* calls with embedded variables, and
some fairly minor recoding where such calls are).  The conversion would almost
be painless.

Even so, this is a pretty radical change to the code, and I wouldn't go ahead
with it without some kind of "blessing" from the core developers and ensuring
that this is a direction they want to head.  However, my main goal is seeing 
this thing run on Oracle, and I don't see any way of getting it there without
A) these changes to allow Oracle more seamless plugging into the existing code,
or B) a substantial rewrite Slash::DB::* for Oracle, which would almost be a
code fork since the interface would have to be radically different.  That's the
last place I want to head.

There are other porting issues I've come across, but they're pathetic compared
to this.  Comments are welcome.

- -- 
Stephen Clouse <the...@wa...>
warpcore.org Founder, Chief Megalomaniac, and Evil Overlord

-----BEGIN PGP SIGNATURE-----
Version: PGP 6.5.8

iQA+AwUBOs/Lt91EXk7JbKbMEQKX8gCYsdgaJOPlKxDvE+XB3g7EBLielACeLYzY
vleJuQHrdCaBJn4gtYttfow=
=y90I
-----END PGP SIGNATURE-----

RE: [Slashcode-development] (no subject)

[Blake D. <bl...@ar...>]

For the newest solution, how are you keeping both tables replicated?  Will
slashd do this?

Thanks
Blake Day
CTO, SpunkNetwork
bl...@sp... <mailto:bl...@sp...>
home: (770) 338-1589
mobile: (678) 485-0519


-----Original Message-----
From: sla...@li...
[mailto:sla...@li...]On Behalf Of
Brian Aker
Sent: Friday, April 06, 2001 1:32 AM
To: sla...@li...
Subject: Re: [Slashcode-development] (no subject)


shane wrote:
> >Why does "newstories" even exist?
> I think it's to speed up the sql. If you've 2000 stories in the
> 'stories' table, and you've got the 10 latest stories in the
> 'newstories' table, then the db (for the majoriy of the hits
> to the system) only has to waid through those few in the
> newstories table, rather then polling the huge stories table.
That is exactly why it exists. We have a better solution
for 2.2 though for this. There will be three tables where
there was one. There will be a meta table holding all
non blob information for the site and a blob table
with the sid referenced to a blob.
Finally there will be a heap table that is a copy
of the meta table. Selects will be made against
table. This is incredibly fast and uses very little
memory.
Neat aye?
	-Brian
--
_______________________________________________________
Brian Aker, br...@ta...
Slashdot Senior Developer
Seattle, Washington
http://tangent.org/~brian/
http://slashdot.org/
_______________________________________________________
You can't grep a dead tree.

_______________________________________________
Slashcode-development mailing list
Sla...@li...
http://lists.sourceforge.net/lists/listinfo/slashcode-development

[Slashcode-development] can't upload a file with multipart/form-data submissions

[James E. M. <jm...@tr...>]

Hi,

Has anybody had any luck with uploading files to a Bender-based site
using form enctype="multipart/form-data"?  I've been hitting a brick 
wall.  I tried lots of things, and read lots of perldoc Apache
and perldoc Apache::Request, but I can't seem to figure it out.

So far I've found that simply relying on the getCurrentForm()
routine in Bender returns a blank form, which is to be expected 
(I think) because Apache->request only works for 
enctype="application/x-www-form-urlencoded".  So I tried 
my own version of that by calling Apache::Request->new($r),
but although I get closer, still no data.  I am not sure 
what to pass to the constructor for the $r, which maybe is the 
problem?  I am currently doing:

my $r = Apache->request;  ## as in Utility.pm
my $apr = Apache::Request->new($r); 
my $status = $apr->parse;  ## explicit call of parse unneccessary, but
       			   ## $status is always zero == no data
print $r->as_string; ## a debugging dump, can see no data is getting up.

Any hints would be appreciated.

thanks in advance,
james

ps,
As to why I am doing this, well, I would like to set up a plugin to 
accept academic papers in place of the articles.  My colleagues are,
um, complete html idiots.  But they can dump Adode PDF files, so 
I figured an easy interface would be to allow uploading pdf files.
This would also be useful for writing html articles and then uploading
them, rather than futzing with editing within a browser text box (of
course, lynx lets me spring open emacs, but people who use windows
are not so blessed).

sorry for the lengthy submission.
james

Re: [Slashcode-development] (no subject)

[Brian A. <br...@ta...>]

shane wrote:
> >Why does "newstories" even exist?
> I think it's to speed up the sql. If you've 2000 stories in the
> 'stories' table, and you've got the 10 latest stories in the
> 'newstories' table, then the db (for the majoriy of the hits
> to the system) only has to waid through those few in the
> newstories table, rather then polling the huge stories table.
That is exactly why it exists. We have a better solution
for 2.2 though for this. There will be three tables where
there was one. There will be a meta table holding all
non blob information for the site and a blob table
with the sid referenced to a blob. 
Finally there will be a heap table that is a copy
of the meta table. Selects will be made against
table. This is incredibly fast and uses very little
memory.
Neat aye?
	-Brian
-- 
_______________________________________________________
Brian Aker, br...@ta... 
Slashdot Senior Developer
Seattle, Washington
http://tangent.org/~brian/
http://slashdot.org/
_______________________________________________________
You can't grep a dead tree.

Re: [Slashcode-development] (no subject)

[Alessio B. <al...@al...>]

shane wrote:

> Take a look at slashd. I think it does the 'moving' from
> newstories to stories.

The other way round, from stories to newstories when stories.writestatus
= 1.

-- 
Alessio F. Bragadini		al...@al...
APL Financial Services		http://village.albourne.com
Nicosia, Cyprus		 	phone: +357-2-755750

"It is more complicated than you think"
		-- The Eighth Networking Truth from RFC 1925

Re: [Slashcode-development] (no subject)

[shane <sh...@lo...>]

At 01:04 AM 4/6/2001 -0400, you wrote:
>Hi,
>
>I need to know what determines the length of time a story remains in the
>table "newstories".  Is it a variable in the "vars" table?  Is it different
>for each section?  Does slashd move the stories out of "newstories" and into
>"stories"?

Take a look at slashd. I think it does the 'moving' from
newstories to stories.


>Why does "newstories" even exist?

I think it's to speed up the sql. If you've 2000 stories in the
'stories' table, and you've got the 10 latest stories in the
'newstories' table, then the db (for the majoriy of the hits
to the system) only has to waid through those few in the
newstories table, rather then polling the huge stories table.

[Slashcode-development] (no subject)

[Blake D. <bl...@ar...>]

Hi,

I need to know what determines the length of time a story remains in the
table "newstories".  Is it a variable in the "vars" table?  Is it different
for each section?  Does slashd move the stories out of "newstories" and into
"stories"?

Why does "newstories" even exist?


Thanks much,
Blake Day
CTO, SpunkNetwork
bl...@sp...
home: (770) 338-1589
mobile: (678) 485-0519

[Slashcode-development] [Fwd: Re: About "Migrating from MySQL to PostgreSQL"]

[Alessio B. <al...@al...>]

FYI

-------- Original Message --------
Subject: Re: About "Migrating from MySQL to PostgreSQL"
Date: Wed, 4 Apr 2001 11:44:27 -0400
From: Zachary Beane <xa...@xa...>
To: Alessio Bragadini <al...@al...>
References: <3AC...@al...>

On Wed, Apr 04, 2001 at 10:21:39AM +0300, Alessio Bragadini wrote:
> Zach,
> I've found your page on "Migrating from MySQL to PostgreSQL" very
> interesting and useful, especially the small library of conversion
> function. 
> 
> I would like to use it for porting Slash (<http://slashcode.com>) from
> MySql to PostgreSQL. Would you mind? Is there any license restriction on
> that file? (Slash is GPL)

Sure, I consider that file to be in the public domain. Warning, the
implementation may not be very efficient.

Zach
-- 
xa...@xa...     Zachary Beane     http://www.xach.com/

Re: [Slashcode-development] sub Slash::DB::PostgreSQL::getSubmissionForUser

[Alessio B. <al...@al...>]

Brian Aker wrote:

> If you get a chance go to sourceforge and enter this in
> as a bug. That will work as a note for me to fix it.

It's bug #413919

Thanks

-- 
Alessio F. Bragadini		al...@al...
APL Financial Services		http://village.albourne.com
Nicosia, Cyprus		 	phone: +357-2-755750

"It is more complicated than you think"
		-- The Eighth Networking Truth from RFC 1925

Re: [Slashcode-development] Bender Conf Problems

[Eric D. <eri...@ja...>]

Yeah, that is EXACTLY what I get..........

shane wrote:

> At 10:31 PM 4/3/2001 -0700, Eric Dannewitz wrote:
> >I'm wondering if this line in the conf files for Bender is working....
> >
> ># Enable the line below if you are worried about performance
> >     PerlTransHandler Slash::Apache::IndexHandler
> >
> >I've tried it, and if I'm a registered user, the only page I get is ones
> >with only the left blocks on it. An anonymous person seems to get a real
> >page with stories and everything.
>
> I get the same thing - no stories.
>
> Checkout
> http://sourceforge.net/tracker/index.php?func=detail&aid=408759&group_id=442
> 1&atid=104421
>
> and see if my description matches what you are seeing (note: the URLS in the
> bug description won't necessarily work now because that was mid-march).
>
> _______________________________________________
> Slashcode-development mailing list
> Sla...@li...
> http://lists.sourceforge.net/lists/listinfo/slashcode-development

Re: [Slashcode-development] sub Slash::DB::PostgreSQL::getSubmissionForUser

[Brian A. <br...@ta...>]

Alessio Bragadini wrote:
> 
> Brian Aker wrote:
> 
> > At this point, all date formatting is occuring
> > outside fo the database. This is probably a hold over from when
> > that was not the case.
> 
> Ok. There are still two occurences of date_format inside the SQL.
> They are in Slash::DB::MySQL::autoUrl and getStoryList.
If you get a chance go to sourceforge and enter this in
as a bug. That will work as a note for me to fix it.
	-Brian
-- 
_______________________________________________________
Brian Aker, br...@ta... 
Slashdot Senior Developer
Seattle, Washington
http://tangent.org/~brian/
http://slashdot.org/
_______________________________________________________
You can't grep a dead tree.

Re: [Slashcode-development] Bender Conf Problems

[shane <sh...@lo...>]

At 10:31 PM 4/3/2001 -0700, Eric Dannewitz wrote:
>I'm wondering if this line in the conf files for Bender is working....
>
># Enable the line below if you are worried about performance
>     PerlTransHandler Slash::Apache::IndexHandler
>
>I've tried it, and if I'm a registered user, the only page I get is ones
>with only the left blocks on it. An anonymous person seems to get a real
>page with stories and everything.


I get the same thing - no stories.

Checkout 
http://sourceforge.net/tracker/index.php?func=detail&aid=408759&group_id=442 
1&atid=104421

and see if my description matches what you are seeing (note: the URLS in the
bug description won't necessarily work now because that was mid-march).

Re: [Slashcode-development] Section page tweaking.

[Ask S. G. <cl...@sl...>]

On Tue, 3 Apr 2001 23:57:11 -0700, CertIndex.com Webmaster said:

> greetings,
>  
>  is it possible to make section pages 'paged' like the comment page is? like
>  if you have 30 stories and you're displaying 10 per page, show the 1, 2, 3
>  at the top and bottom of the page like would happen with the comment page.
>  
>  hopefully not only this is possible, but also limiting stories per date with
>  some url string var so that if you have 50 stories in the db, 30 of the
>  stories were posted in the past 1 month, and the 'date filter' is set to 30
>  days, only 30 would be dealt with. combined with the paging stuff from
>  above, you'd have 30 dealt with, and 10 per page. giving you 3 pages.
>  
>  hope you guys can understand me, very, very tired right now.
>  
>  thanks!

Not quite sure if I do. 

IF I do, I don't think this is possible with the story display without some
code changes in Slash.pm. I'll look into this if I have the time, this week,
and get back to you.

- Cliff

[Slashcode-development] [ slashcode-Patches-413645 ] Adding a wrapper around LOCK TABLES

[<no...@so...>]

Patches item #413645, was updated on 2001-04-03 23:56
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=304421&aid=413645&group_id=4421

Category: MAIN branch
Group: None
Status: Open
Priority: 5
Submitted By: Alessio Bragadini (alessio)
Assigned to: Nobody/Anonymous (nobody)
Summary: Adding a wrapper around LOCK TABLES

Initial Comment:
While MySql needs LOCK TABLE at times, other DB engines
(i.e. PostgreSQL and Oracle) don't. It would be better
to have two wrapper subroutines, doLock and doUnlock,
that map to the real SQL operations. In the case of
other DBs, they could be a no-op.

Please note that my implementation of doLock on MySql
may be flawed since I am no expert of that DB.

----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=304421&aid=413645&group_id=4421

[Slashcode-development] Section page tweaking.

[CertIndex.com W. <web...@ce...>]

greetings,

is it possible to make section pages 'paged' like the comment page is? like
if you have 30 stories and you're displaying 10 per page, show the 1, 2, 3
at the top and bottom of the page like would happen with the comment page.

hopefully not only this is possible, but also limiting stories per date with
some url string var so that if you have 50 stories in the db, 30 of the
stories were posted in the past 1 month, and the 'date filter' is set to 30
days, only 30 would be dealt with. combined with the paging stuff from
above, you'd have 30 dealt with, and 10 per page. giving you 3 pages.

hope you guys can understand me, very, very tired right now.

thanks!

Re: [Slashcode-development] sub Slash::DB::PostgreSQL::getSubmissionForUser

[Alessio B. <al...@al...>]

Brian Aker wrote:

> At this point, all date formatting is occuring
> outside fo the database. This is probably a hold over from when
> that was not the case.

Ok. There are still two occurences of date_format inside the SQL.
They are in Slash::DB::MySQL::autoUrl and getStoryList.

Do you plan to change them to Perl code, too? Because otherwise we need
a wrapper to map MySql's date_format into PostgreSql's to_char. It's
ready, but it would require plpgsql so it'd be better to leave it out.

-- 
Alessio F. Bragadini		al...@al...
APL Financial Services		http://village.albourne.com
Nicosia, Cyprus		 	phone: +357-2-755750

"It is more complicated than you think"
		-- The Eighth Networking Truth from RFC 1925

[Slashcode-development] Bender Conf Problems

[Eric D. <eri...@ja...>]

I'm wondering if this line in the conf files for Bender is working....

# Enable the line below if you are worried about performance
    PerlTransHandler Slash::Apache::IndexHandler

I've tried it, and if I'm a registered user, the only page I get is ones
with only the left blocks on it. An anonymous person seems to get a real
page with stories and everything.

Re: [Slashcode-development] sub Slash::DB::PostgreSQL::getSubmissionForUser

[Brian A. <br...@ta...>]

Alessio Bragadini wrote:
> What is correct? The SQL line for PostgreSQL sure it's not :-)
> I believe that date_format should be to_char and 'm/d H:i' with '%'s, at
> least.
The one with just time. At this point, all date formatting is occuring
outside fo the database. This is probably a hold over from when
that was not the case.
	-Brian

[Slashcode-development] subroutine Slash::DB::PostgreSQL::getDay

[Alessio B. <al...@al...>]

What should return?

-- 
Alessio F. Bragadini		al...@al...
APL Financial Services		http://village.albourne.com
Nicosia, Cyprus		 	phone: +357-2-755750

"It is more complicated than you think"
		-- The Eighth Networking Truth from RFC 1925

[Slashcode-development] sub Slash::DB::PostgreSQL::getSubmissionForUser

[Alessio B. <al...@al...>]

package Slash::DB::{Postgre|My}SQL, routine getSubmissionForUser,
updated from CVS.

I am a little confused by the SELECT:

in PostgreSQL.pm:

SELECT subid,subj,date_format($dateformat,'m/d  H:i') [...]

in MySQL.pm:

SELECT subid,subj,time [...]

and later

formatDate($submission, 2, 2, '%m/%d  %H:%M');

in Perl.

What is correct? The SQL line for PostgreSQL sure it's not :-)
I believe that date_format should be to_char and 'm/d H:i' with '%'s, at
least.

-- 
Alessio F. Bragadini		al...@al...
APL Financial Services		http://village.albourne.com
Nicosia, Cyprus		 	phone: +357-2-755750

"It is more complicated than you think"
		-- The Eighth Networking Truth from RFC 1925

Re: [Slashcode-development] PgSql no LAST_INSERT_ID()

[Alessio B. <al...@al...>]

Brian Aker wrote:

> Send patches :)

I will do it probably later today or tomorrow. I think it's wise to
change as little code as follow, but use Zach Beane's "mini-library"
<http://www.xach.com/aolserver/mysql-to-postgresql.html>. 

Once we have from_unixtime, to_days etc., there is only need for an
handful of changes.

-- 
Alessio F. Bragadini		al...@al...
APL Financial Services		http://village.albourne.com
Nicosia, Cyprus		 	phone: +357-2-755750

"It is more complicated than you think"
		-- The Eighth Networking Truth from RFC 1925

[Slashcode-development] multiple sets of articles

[Blake D. <bl...@ar...>]

Our site is being built to have 2 distinct areas of the forum. The first
area will be the standard news slashdot-style area where people submit
articles and if they are interesting, they are posted. The second area will
use the bulletin board plugin we're making so that every article submitted
is automatically posted. These will act as 'threads' or 'topics' as they are
referred to in bulletin board software. Since the 2 areas of the site will
use the same sections, we want to be able to keep the articles posted to the
news area separate so they aren't shown on the bulletin board area. The same
with articles posted to the bulletin board area separate so they don't show
up on the news area. The idea we had for this was to assign every article a
flag in the extratext column. 'news' and 'bb', depending upon which area
they were submitted from. So if you go to the news area it'll only show
stories with the extratext of news and if you go to the bulletin board area
it'll only show stories with the extratext of bb. Does anyone have any
better ideas? If not, does anyone have any ideas to make getStories
recognize the extratext flag we pass it and filter the stories it returns
based on that?

Also, how many other places in the code would we possibly need to modify
this?

[Slashcode-development] multiple sets of articles

[Blake D. <bla...@ch...>]

Our site is being built to have 2 distinct areas of the forum. The first
area will be the standard news slashdot-style area where people submit
articles and if they are interesting, they are posted. The second area will
use the bulletin board plugin we're making so that every article submitted
is automatically posted. These will act as 'threads' or 'topics' as they are
referred to in bulletin board software. Since the 2 areas of the site will
use the same sections, we want to be able to keep the articles posted to the
news area separate so they aren't shown on the bulletin board area. The same
with articles posted to the bulletin board area separate so they don't show
up on the news area. The idea we had for this was to assign every article a
flag in the extratext column. 'news' and 'bb', depending upon which area
they were submitted from. So if you go to the news area it'll only show
stories with the extratext of news and if you go to the bulletin board area
it'll only show stories with the extratext of bb. Does anyone have any
better ideas? If not, does anyone have any ideas to make getStories
recognize the extratext flag we pass it and filter the stories it returns
based on that?

Also, how many other places in the code would we possibly need to modify
this?

[Slashcode-development] slashstyle

[Chris N. <pu...@po...>]

We've written a document called "slashstyle" which is now in effect as the
primary coding guideline for Slash.  It is on CVS (in fry, /docs/), and it
is on http://slashcode.com/docs/ and http://slashcode.com/faq.shtml, along
with the boilerplates.

All new code going into Slash sould follow it.  If you have any problems,
please let us know.  The core team may make changes to the document at any
time (and especially the boilerplates), and we will note significant
changes when they are made.

	http://slashcode.com/article.pl?sid=01/03/30/1924249

-- 
Chris Nandor                      pu...@po...    http://pudge.net/
Open Source Development Network    pu...@os...     http://osdn.com/

[Slashcode-development] slashcode-i18n

[Chris N. <pu...@po...>]

There is a new mailing list in town, slashcode-i18n for discussion of
internationalization support in Slash.

	http://sourceforge.net/mail/?group_id=4421

-- 
Chris Nandor                      pu...@po...    http://pudge.net/
Open Source Development Network    pu...@os...     http://osdn.com/