semanticscuttle-devel — General discussion about SemanticScuttle development

Re: [Semanticscuttle-devel] Privatekey

[Mark P. <mpe...@gm...>]

Yep, made some final changes last night and didn't get a chance to commit.
 I just committed my work so please review.  Ok, I confess, I haven't done
the below:

"The API tests look really good. What I miss is tests that query the
actual feed URLs with parameters; see the API tests that do
->getRequest()->send()."

I was hoping to do them last week when I thought I would have the time, but
never got to it.  I will try to complete that this evening.

Cheers!
Mark

On Wed, Apr 6, 2011 at 1:02 PM, Christian Weiske <cw...@cw...> wrote:

> Hello Mark,
>
>
> > Well that was easy.... :)  Thanks for the tip!  I'll finish up on the
> > unit tests and hopefully we'll be done.
> Any updates?
>
> --
> Regards/Mit freundlichen Grüßen
> Christian Weiske
>
> -=≡ Geeking around in the name of science since 1982 ≡=-
>
>
> ------------------------------------------------------------------------------
> Xperia(TM) PLAY
> It's a major breakthrough. An authentic gaming
> smartphone on the nation's most reliable network.
> And it wants your games.
> http://p.sf.net/sfu/verizon-sfdev
> _______________________________________________
> Semanticscuttle-devel mailing list
> Sem...@li...
> https://lists.sourceforge.net/lists/listinfo/semanticscuttle-devel
>
>

Re: [Semanticscuttle-devel] Privatekey

[Christian W. <cw...@cw...>]

Hello Mark,


> Well that was easy.... :)  Thanks for the tip!  I'll finish up on the
> unit tests and hopefully we'll be done.
Any updates?

-- 
Regards/Mit freundlichen Grüßen
Christian Weiske

-=≡ Geeking around in the name of science since 1982 ≡=-

Re: [Semanticscuttle-devel] Privatekey / Opensearch problem / Configurable privacy

[Christian W. <cw...@cw...>]

Hello Brett,


> >> Also, I ran BookmarkTest.php in PHPUnit and there was one failure:
> >> 1) BookmarkTest::testHardCharactersInBookmarks
> >> Failed asserting that two strings are equal.
> >> --- Expected
> >> +++ Actual
> >> @@ @@
> >> -#{|`^@]³¹¡¿<&é__(-è\_çà)
> >> +#{|`^@]
> >>
> >> I had not changed anything in this file.
> > Does it suddenly work when running the tests the second (or third)
> > time? i know of one unstable tests (which is crap and needs to be
> > fixed).
> I ran the test five times consecutively, and the assert failed each
> time.

That really looks like DOS characters. Maybe it'll solve itself when
you get a unix box or run the tests on the normal windows command line.

-- 
Regards/Mit freundlichen Grüßen
Christian Weiske

-=≡ Geeking around in the name of science since 1982 ≡=-

Re: [Semanticscuttle-devel] Status for configurable privacy ....

[Christian W. <cw...@cw...>]

Hello Brett,


> I have some concerns about the fact that there were Windows
> carriage returns in my last check-in. I am working on a Windows
> environment--I will eventually set up a Linux environment, but not
> for a few weeks at least--but I've been editing files with vi on
> Cygwin and MKS Toolkit, so there should not have been any Windows
> carriage returns.
No problem. Someone noticed it, now it's fixed, all is fine :)

> Any tips/advice you have concerning this would be
> appreciated.
Sorry, I've never worked with cygwin or so. I know that many
editors/IDEs on windows are able to save files with unix newlines only,
i.e. Notepad++ and all the PHP IDEs.


> Here are a few things I noticed when testing import.php and
> importNetscape.php:
> 1. import.php points the user to http://del.icio.us/api/posts/all to
> save bookmarks to an XML file. This is evidently not a good option any
> longer for "new" del.icio.us users who are using a Yahoo! ID to access
> their bookmarks. A better option for these users is to go to the
> Delicious XML Exporter at http://deliciousxml.com/. Otherwise, they
> must use http://api.del.icio.us/v2/posts/all with open authorization,
> and that complicates the process somewhat.
Yep, we should change that.

> 2. I think I may have found a bug in importNetscape.php, which ignores
> the last bookmark in the specified HTML file. I think this is
> occurring in the regular expression given as the first parameter of
> the preg_match_all function call. Consequently, my test data contains
> a last entry that I have assumed will be ignored. The test will need
> to be updated if this behavior in importNetscape.php changes.
Ok, I'll see if I can fix that.


About your tests:
- You assume that the default privacy is configured to the default one,
  but that's something you cannot assume. If you do such assumptions,
  at least do a assertion on your assumed variable so the test fails
  with an explanatory message.
- You assume a default privacy setting since there was no way to
  configure the privacy permanently for http requests - up to now! In
  the jquery branch, I had the same problem and solved it; the
  testbaseapi class has a way to set default configuration values.
- You had one big method in BookmarksTest that checked all the
  different api methods. In general, keeping test methods small is a
  really good idea since in case of a failure, you can narrow down the
  problematic code line much faster. The rule of thumb is to test one
  thing in a test (but I don't want only one assertion in one test
  method, that'd be overkill).
- I began to split up the big test method and
  moved the api/posts_add test to their corresponding Api test
  class. By doing that I could utilize the TestBaseApi methods that
  prepare authorized HTTP requests automatically for you if you wish
  that. Do the same for the rest of the tests in there.
- In tests, always use example.org as hostname, or example.com and
  example.net (of course, you can use foo.example.org,
  example.org/foobar and such).
- You assume that new bookmarks always get ID 1 in your tests. That
  might not always be the case, especially with database systems other
  than mysql. Better than fetching bookmark ID 1 is to fetch all
  bookmarks of the user with the ID you just created which keeps you on
  the safe side.
- Assuming http://localhost/ as URL fails for my setup since I have
  "http://bm.bogo/" as URL under which my dev instance of SC is
  available. TestBaseApi takes care of this.

I cherry-picked the two relevant changes to testbaseapi into your
branch. You still need to merge master, though.

-- 
Regards/Mit freundlichen Grüßen
Christian Weiske

-=≡ Geeking around in the name of science since 1982 ≡=-

[Semanticscuttle-devel] jquery branch merged to master!

[Christian W. <cw...@cw...>]

Hi fellow developers,


After months of hacking, I am proud to announce that the jquery branch
has been merged into master!
The former javascript library - dojo+dijit - has been dropped and
replaced with jquery, jstree and jquery-ui for autocompletion.

This change gives us two big advantages:
1. No more "tree model deprecated" developer warnings
2. Offline capabilities - up to now, dojo and dijit from google's
   ajaxapis were used. I can finally use and thus test all
   semanticscuttle features on the way to and from work in the train.

While there might be bugs in it, I consider the implementation good
enough for wide testing and inclusion into the next version 0.98.0.

Now I do also have time to review the privatekey and configurable
privacy branches, so that 0.98 should not be too far away now.

-- 
Regards/Mit freundlichen Grüßen
Christian Weiske

-=≡ Geeking around in the name of science since 1982 ≡=-

Re: [Semanticscuttle-devel] "Further updates for configurable-privacy2"

[Brett D. <bs...@fr...>]

Hello, Christian.

I think I've removed all the carriage returns now.

Rgrds,
Brett

On Thu, 31 Mar 2011 10:43:54 +0200, Christian Weiske
<cw...@cw...> wrote:
> Hi Brett,
> 
> Looking at
> http://semanticscuttle.git.sourceforge.net/git/gitweb.cgi?p=semanticscuttle/sc;a=commitdiff;h=ef88147
> I see that you have misconfigured your editor to not use unix newlines,
> but added \r everywhere. Please revert that changes.

Re: [Semanticscuttle-devel] "Further updates for configurable-privacy2"

[Christian W. <cw...@cw...>]

Hi Brett,

Looking at
http://semanticscuttle.git.sourceforge.net/git/gitweb.cgi?p=semanticscuttle/sc;a=commitdiff;h=ef88147
I see that you have misconfigured your editor to not use unix newlines,
but added \r everywhere. Please revert that changes.

-- 
Regards/Mit freundlichen Grüßen
Christian Weiske

-= Geeking around in the name of science since 1982 =-

Re: [Semanticscuttle-devel] Privatekey

[Christian W. <cw...@cw...>]

Hi Mark,


> I think the only two items I have left on this is the 404 issue with
> the wrong content type and the API tests along with the tests on the
> new function.  I don't see an easy fix to the 404 issue because the
> header is already being written with the predefined content type.
> I'll study it a bit longer to see what I can come up with.

As long as no output is sent, you can send as many headers as you like
- even the same headers again. Thus, sending a "HTTP/1.0 200 OK" after
a content-type is not a problem.


-- 
Regards/Mit freundlichen Grüßen
Christian Weiske

-= Geeking around in the name of science since 1982 =-

Re: [Semanticscuttle-devel] Status for configurable privacy ....

[Brett D. <bs...@fr...>]

Hello, Christian.

I have completed my unit tests and will likely check in my changes
Tuesday. I have some concerns about the fact that there were Windows
carriage returns in my last check-in. I am working on a Windows
environment--I will eventually set up a Linux environment, but not for a
few weeks at least--but I've been editing files with vi on Cygwin and
MKS Toolkit, so there should not have been any Windows carriage returns.
Any tips/advice you have concerning this would be appreciated.

Here are a few things I noticed when testing import.php and
importNetscape.php:
1. import.php points the user to http://del.icio.us/api/posts/all to
save bookmarks to an XML file. This is evidently not a good option any
longer for "new" del.icio.us users who are using a Yahoo! ID to access
their bookmarks. A better option for these users is to go to the
Delicious XML Exporter at http://deliciousxml.com/. Otherwise, they must
use http://api.del.icio.us/v2/posts/all with open authorization, and
that complicates the process somewhat.
2. I think I may have found a bug in importNetscape.php, which ignores
the last bookmark in the specified HTML file. I think this is occurring
in the regular expression given as the first parameter of the 
preg_match_all function call. Consequently, my test data contains a
last entry that I have assumed will be ignored. The test will need to be
updated if this behavior in importNetscape.php changes.

Also, I will be creating a data directory below the tests directory to
keep the test data for import.php and importNetscape.php.

Rgrds,
Brett

On Tue, 22 Mar 2011 06:57:55 +0100, Christian Weiske
<cw...@cw...> wrote:
> Hello Brett,
> 
> 
>> I've written a unit test for posts_add.php and will write additional
>> asserts for the other affected files this week.
>>
>> I think I will need to create a bookmark file with data for testing
>> import.php and importNetscape.php. I assume I should just place this
>> file in the tests directory, correct?
> Yep. Although I begin to think that we should create a data/ directory
> inside tests/ and put the file in there so they don't clutter up the
> tests dir itself. Please prefix the file names with the test class
> name, i.e. "UserTest-netscape.html".

Re: [Semanticscuttle-devel] Status for configurable privacy ....

[Christian W. <cw...@cw...>]

Hello Brett,


> I've written a unit test for posts_add.php and will write additional
> asserts for the other affected files this week.
> 
> I think I will need to create a bookmark file with data for testing
> import.php and importNetscape.php. I assume I should just place this
> file in the tests directory, correct?
Yep. Although I begin to think that we should create a data/ directory
inside tests/ and put the file in there so they don't clutter up the
tests dir itself. Please prefix the file names with the test class
name, i.e. "UserTest-netscape.html".

-- 
Regards/Mit freundlichen Grüßen
Christian Weiske

-=≡ Geeking around in the name of science since 1982 ≡=-

[Semanticscuttle-devel] Status for configurable privacy ....

[Brett D. <bs...@fr...>]

Hello, Christian.

I've written a unit test for posts_add.php and will write additional
asserts for the other affected files this week.

I think I will need to create a bookmark file with data for testing
import.php and importNetscape.php. I assume I should just place this
file in the tests directory, correct?

Rgrds,
Brett


On Tue, 15 Mar 2011 08:38:21 +0100, Christian Weiske
<cw...@cw...> wrote:
> Hello Brett,
> 
> 
>> Thanks for the feedback/review. I will work to make corrections and
>> amend my changes over the next few days.
> Great.
> 
>> Given my as-yet limited experience with git, perhaps it would be best
>> for you to delete the branch.
> I deleted the configurable-privacy branch and created a new one for
> you, configurable-privacy2. I also applied your changes to it as a
> commit from you.

Re: [Semanticscuttle-devel] Privatekey

[Christian W. <cw...@cw...>]

Hello Mark,


> Ok Christian, I've backed out the bookmark query updates, and removed
> the recently added User column, added a number of new tests, and
> addressed most (if not all) of your concerns).  I'm still a bit green
> on unit testing, so be easy on me.  :)  Cheers to all.  Have a great
> weekend!


> >>> I've got a little special configuration on my dev system:
> >>> > $sitename = 'b"m.bo\'go';
> >>> The site name here contains both single and double quotes,
> >>> something that helped me discovering quite a number of
> >>> quoting/escaping bugs. It seems that you double-htmlencode the
> >>> title in the rss feed so that I get
> >>> > title="b"m.bo'go: (private) b"m.bo'go"
> >>> instead of the
> >>> > title="b"m.bo'go: Recent bookmarks"
> >>> as it is for the normal feed in the HTML head.
The problem still exists for me. Seems I did not describe it properly
since you did remove the escaping in rss.tpl.php which needs to stay
there.
You don't double-encode it in the rss feed but in the rss feed link on
the html main page:
> <link rel="alternate" type="application/rss+xml"
>  title="b&quot;m.bo'go: Recent bookmarks"
>  href="http://bm.bogo/rss.php?sort=date_desc" />
> <link rel="alternate" type="application/rss+xml"
>  title="b&amp;quot;m.bo'go: (private) b&amp;quot;m.bo'go"
>  href="http://bm.bogo/rss.php/test?sort=date_desc&amp;privatekey=-e36a17f2717a3da7321aa742e328e92" />

It's already fixed on the main page. On the user bookmark page
you unfortunately did not encode the & in the URL, making the page
invalid. I either spot these things manually or use the W3C HTML
Validator. With the Opera browser, you just have to right-click a page
and "Validate", on firefox you may want to install the web developer
toolbar - it has a "validate local html" button on the "extras" menu.

You also removed the htmlspecialchars on the opensearchdescription
title in top.inc.php - that needs to be kept, too.


> >>> There is no fail message when an unknown privatekey is used.
> >>> SemanticScuttle should send a status 400 (Bad request) with an
> >>> explanatory message.
I see that a 404 is sent, but I don't see any error message in the
browser. This is due to the fact that the error is sent as HTML page,
but the Content-Type in the headers is application/rss+xml:

$ curl -i 'http://bm.bogo/rss.php/test?sort=date_desc&privatekey=foobar'
HTTP/1.1 404 Not Found
Date: Mon, 21 Mar 2011 06:46:04 GMT
....
Content-Type: application/rss+xml; charset=utf-8

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd"> <html
xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">


One thing I noted on the profile page:
When I uncheck the "enable" checkbox, I get a new key of
"-e36a17f2717a3da7321aa742e328e92" the the checkbox still enabled.
Also, the "enabled" text should be surrounded with a <label
for="checkbox-id"> tag so that the user can click the text and does not
have to aim on the checkbox itself.
I just see that this is due to the SQL change that I did not apply yet
on my database!

The attribute "readonly" is ok, but please make it XHTML-compatible by
using readonly="readonly".


About this:
> +     * @param string $privateKey       RSS Private Key
> +     * @param string $enablePrivateRSS RSS Private Key
From that signature, I don't understand why you need two parameters
here since they have the same type and the same description. Plus, the
type is string but the default value in the method signature is still
0 :) Use a boolean type here and keep the original name of
"enablePrivateKey", since we it does not have to be limited to the RSS
feeds in the future.

And I see that we misunderstood each other; you want to keep the
original key and be able to re-enable the key in the profile while
being able to restore the old one. I thought that setting it to NULL or
empty string would suffice, but your idea of prepending a "-" is
actually better.

I like that you hide the implementation details of
deactivating/activating the private key in the updateuser method by
providing a "enableprivatekey" parameter. Unfortunately, you don't hide
the details when checking for it and always rely on the 32 characters
length: 
> www/index.php
> +    if ($currentUser->getPrivateKey() <> null &&
> strlen($currentUser->getPrivateKey()) == 32) {
It'd be cool if you implemented and used
$currentUser->isPrivateKeyAllowed() there.


The API tests look really good. What I miss is tests that query the
actual feed URLs with parameters; see the API tests that do
->getRequest()->send().

I'm beginning to see the finish line here now :)

-- 
Regards/Mit freundlichen Grüßen
Christian Weiske

-=≡ Geeking around in the name of science since 1982 ≡=-

Re: [Semanticscuttle-devel] Privatekey

[Mark P. <mpe...@gm...>]

Ok Christian, I've backed out the bookmark query updates, and removed the
recently added User column, added a number of new tests, and addressed most
(if not all) of your concerns).  I'm still a bit green on unit testing, so
be easy on me.  :)  Cheers to all.  Have a great weekend!


On Tue, Mar 15, 2011 at 10:55 PM, Mark Pemberton <mpe...@gm...>wrote:

> Sorry for the delay on the privatekey code.  I'm finishing up with the
> tests and should be done shortly.  Cheers to all.
> Mark
>
>
> On Mon, Mar 7, 2011 at 4:08 PM, Mark Pemberton <mpe...@gm...>wrote:
>
>> Thanks for the great response Christian!  I'll work on these items and get
>> back to you.  Cheers to all!
>>
>> On Mon, Mar 7, 2011 at 2:03 PM, Christian Weiske <cw...@cw...>wrote:
>>
>>> Hi Mark,
>>>
>>>
>>>
>>> > Christian, I've updated privatekey with the following changes:
>>> > 1) I added an enable checkbox to allow user to enable/disable the
>>> > Private RSS feed.
>>> That's a good addition since it allows people to easily deactivate
>>> their private key without manually deleting the key.
>>>
>>> I saw that you added a second column beside the private key;
>>> enablePrivateKey. In my eyes this is not necessary since that
>>> information can be transmitted in the private key column - when it's
>>> empty/NULL, it's deactivated. Please change that.
>>>
>>> > 2) I took out the ajax call in profile.php - I ran into some issues
>>> > with trying keep the javascript for those who use it.  Didn't think
>>> > it was that big enough of a deal to worry about it.
>>> Fine with me.
>>>
>>> I've got a little special configuration on my dev system:
>>> > $sitename = 'b"m.bo\'go';
>>> The site name here contains both single and double quotes, something
>>> that helped me discovering quite a number of quoting/escaping bugs. It
>>> seems that you double-htmlencode the title in the rss feed so that I
>>> get
>>> > title="b&amp;quot;m.bo'go: (private) b&amp;quot;m.bo'go"
>>> instead of the
>>> > title="b&quot;m.bo'go: Recent bookmarks"
>>> as it is for the normal feed in the HTML head. Also add a newline
>>> before the private <link> so the html looks cleaner. You also do not
>>> escape the & in the link before privatekey=, which makes the page fail
>>> to validate.
>>>
>>> There is no fail message when an unknown privatekey is used.
>>> SemanticScuttle should send a status 400 (Bad request) with an
>>> explanatory message.
>>>
>>> The user's bookmark list does also not contain a privatekey feed link.
>>> I think that all pages that link feeds should also link the
>>> associated private feed (i.e. useful when someone wants to bookmark a
>>> search that may contain private bookmarks).
>>>
>>> And unit tests are a must here:
>>> - fetch feed with private key, do the private bookmarks appear?
>>> - same without priv key
>>> - are the private feed links on all relevant pages when logged in and
>>>  the user has a private key?
>>> - same for logged out -> no private links
>>> - same for logged in bug no private key -> no private feed links
>>>
>>>
>>> > 3) on a separate commit, I took a stab at updating the getBookmarks()
>>> > function to make the queries a little more efficient.  The base of the
>>> > problem was doing both a GROUP BY and an ORDER BY in the same query
>>> > (as well as the use of multiple tables on the same query).  I got
>>> > spoiled on Oracle because it tends to be a little more efficient.
>>> > MySQL as well as other RDBS's don't optimize it enough to make it
>>> > very efficient.  So I tweaked the queries to get around some of the
>>> > issues.  I haven't tested it on MySQL 4 so I can't confirm if the
>>> > query will work on that version.  There was one component that I
>>> > couldn't make more efficient and that is searching by a tag, or by a
>>> > user, or searching.  Those queries are simply too complex to make
>>> > efficient.  However, they are fast if there is only a small
>>> > resultset. Anyway, outside of doing a full redesign of the data
>>> > tables, this is the best I could come up with.  My last commit
>>> > contains the updates to the user table and the bookmarks table.  I
>>> > simply added an additional index on the bookmarks table to help with
>>> > the query.  Please let me know if you have any questions.
>>>
>>> Do the tests do still run? I get a bunch of errors when running them.
>>> Please also do that in a different branch; we're using one branch per
>>> feature which makes it easier to merge the relevant features in the
>>> master branch later. Just imagine that the private key feature is
>>> ready, but your sql optimizations are buggy but we want to release a
>>> new version. If we have clean feature branches, we just merge the ones
>>> that are ready and are done. Currently this is not possible since an
>>> unfinished feature is in the same branch as the other one.
>>>
>>> --
>>> Regards/Mit freundlichen Grüßen
>>> Christian Weiske
>>>
>>> -=≡ Geeking around in the name of science since 1982 ≡=-
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> What You Don't Know About Data Connectivity CAN Hurt You
>>> This paper provides an overview of data connectivity, details
>>> its effect on application quality, and explores various alternative
>>> solutions. http://p.sf.net/sfu/progress-d2d
>>>
>>> _______________________________________________
>>> Semanticscuttle-devel mailing list
>>> Sem...@li...
>>> https://lists.sourceforge.net/lists/listinfo/semanticscuttle-devel
>>>
>>>
>>
>

Re: [Semanticscuttle-devel] Privatekey

[Mark P. <mpe...@gm...>]

Sorry for the delay on the privatekey code.  I'm finishing up with the tests
and should be done shortly.  Cheers to all.
Mark


On Mon, Mar 7, 2011 at 4:08 PM, Mark Pemberton <mpe...@gm...>wrote:

> Thanks for the great response Christian!  I'll work on these items and get
> back to you.  Cheers to all!
>
> On Mon, Mar 7, 2011 at 2:03 PM, Christian Weiske <cw...@cw...>wrote:
>
>> Hi Mark,
>>
>>
>>
>> > Christian, I've updated privatekey with the following changes:
>> > 1) I added an enable checkbox to allow user to enable/disable the
>> > Private RSS feed.
>> That's a good addition since it allows people to easily deactivate
>> their private key without manually deleting the key.
>>
>> I saw that you added a second column beside the private key;
>> enablePrivateKey. In my eyes this is not necessary since that
>> information can be transmitted in the private key column - when it's
>> empty/NULL, it's deactivated. Please change that.
>>
>> > 2) I took out the ajax call in profile.php - I ran into some issues
>> > with trying keep the javascript for those who use it.  Didn't think
>> > it was that big enough of a deal to worry about it.
>> Fine with me.
>>
>> I've got a little special configuration on my dev system:
>> > $sitename = 'b"m.bo\'go';
>> The site name here contains both single and double quotes, something
>> that helped me discovering quite a number of quoting/escaping bugs. It
>> seems that you double-htmlencode the title in the rss feed so that I
>> get
>> > title="b&amp;quot;m.bo'go: (private) b&amp;quot;m.bo'go"
>> instead of the
>> > title="b&quot;m.bo'go: Recent bookmarks"
>> as it is for the normal feed in the HTML head. Also add a newline
>> before the private <link> so the html looks cleaner. You also do not
>> escape the & in the link before privatekey=, which makes the page fail
>> to validate.
>>
>> There is no fail message when an unknown privatekey is used.
>> SemanticScuttle should send a status 400 (Bad request) with an
>> explanatory message.
>>
>> The user's bookmark list does also not contain a privatekey feed link.
>> I think that all pages that link feeds should also link the
>> associated private feed (i.e. useful when someone wants to bookmark a
>> search that may contain private bookmarks).
>>
>> And unit tests are a must here:
>> - fetch feed with private key, do the private bookmarks appear?
>> - same without priv key
>> - are the private feed links on all relevant pages when logged in and
>>  the user has a private key?
>> - same for logged out -> no private links
>> - same for logged in bug no private key -> no private feed links
>>
>>
>> > 3) on a separate commit, I took a stab at updating the getBookmarks()
>> > function to make the queries a little more efficient.  The base of the
>> > problem was doing both a GROUP BY and an ORDER BY in the same query
>> > (as well as the use of multiple tables on the same query).  I got
>> > spoiled on Oracle because it tends to be a little more efficient.
>> > MySQL as well as other RDBS's don't optimize it enough to make it
>> > very efficient.  So I tweaked the queries to get around some of the
>> > issues.  I haven't tested it on MySQL 4 so I can't confirm if the
>> > query will work on that version.  There was one component that I
>> > couldn't make more efficient and that is searching by a tag, or by a
>> > user, or searching.  Those queries are simply too complex to make
>> > efficient.  However, they are fast if there is only a small
>> > resultset. Anyway, outside of doing a full redesign of the data
>> > tables, this is the best I could come up with.  My last commit
>> > contains the updates to the user table and the bookmarks table.  I
>> > simply added an additional index on the bookmarks table to help with
>> > the query.  Please let me know if you have any questions.
>>
>> Do the tests do still run? I get a bunch of errors when running them.
>> Please also do that in a different branch; we're using one branch per
>> feature which makes it easier to merge the relevant features in the
>> master branch later. Just imagine that the private key feature is
>> ready, but your sql optimizations are buggy but we want to release a
>> new version. If we have clean feature branches, we just merge the ones
>> that are ready and are done. Currently this is not possible since an
>> unfinished feature is in the same branch as the other one.
>>
>> --
>> Regards/Mit freundlichen Grüßen
>> Christian Weiske
>>
>> -=≡ Geeking around in the name of science since 1982 ≡=-
>>
>>
>> ------------------------------------------------------------------------------
>> What You Don't Know About Data Connectivity CAN Hurt You
>> This paper provides an overview of data connectivity, details
>> its effect on application quality, and explores various alternative
>> solutions. http://p.sf.net/sfu/progress-d2d
>>
>> _______________________________________________
>> Semanticscuttle-devel mailing list
>> Sem...@li...
>> https://lists.sourceforge.net/lists/listinfo/semanticscuttle-devel
>>
>>
>

Re: [Semanticscuttle-devel] Added branch for configurable privacy ....

[Christian W. <cw...@cw...>]

Hello Brett,


> Thanks for the feedback/review. I will work to make corrections and
> amend my changes over the next few days.
Great.

> Given my as-yet limited experience with git, perhaps it would be best
> for you to delete the branch.
I deleted the configurable-privacy branch and created a new one for
you, configurable-privacy2. I also applied your changes to it as a
commit from you.

-- 
Regards/Mit freundlichen Grüßen
Christian Weiske

-=≡ Geeking around in the name of science since 1982 ≡=-

Re: [Semanticscuttle-devel] Added branch for configurable privacy ....

[Brett D. <bs...@fr...>]

Hello, Christian.

Thanks for the feedback/review. I will work to make corrections and
emend my changes over the next few days.

Given my as-yet limited experience with git, perhaps it would be best
for you to delete the branch.

Thanks.

Rgrds,
Brett

On Mon, 14 Mar 2011 08:33:56 +0100, Christian Weiske
<cw...@cw...> wrote:
> Hello Brett,
> 
> 
>> My apologies for the mix-up. I think I've corrected this by pushing to
>> the master at origin. However, now the configurable-privacy branch
>> shows two parents.
> No, not really. I had to do the diff manually since you also seemed to
> change the line breaks in all the files.
> 
> Did you notice you have a sc subdirectory? You also added files
> like Thumbs.db that should never be checked in the repository.
> 
> About the code itself:
> - You do not need to declare "$defaults = array()" in config.php.dist
>   since config.default.php is included before config.php - so
>   redeclaring the array would actually erase the default settings for
>   privacy.
> - Why do you suddenly display "public" for public bookmarks? Since
>   public is standard, there is no need to display the text. The
>   bookmark status is only shown for non-public bookmarks which helps to
>   unclutter the bookmark list display in my eyes.
> - Your unit test is flawed since it does not test the effects of a
>   privacy config setting at all - you set the variable and use the same
>   variable as parameter to the method you are calling.
>   The correct way is to set the privacy variable permanently in a
>   config and call the scripts you changed (i.e. posts_add.php, edit.php)
>   with a set of POST parameters via HTTP. You can then observe if the
>   added bookmarks have the configured status. Setting the varible
>   permanently is not easy here since you do a HTTP call, but I will add
>   that to the master branch quickly.
>   Always remember that for the unit tests to be sufficient they need to
>   fail when I remove or change a line in your SemanticScuttle source
>   code changes. Currently, I can revert the changes in www/bookmark.php
>   and all other files without the tests to fail.
> 
> Apart from that, the code seems to do what it should.
> 
> About the branch:
> Since the configurable-privacy branch is totally broken, we should just
> delete it and start afresh from master and apply your changes to it.
> Shall I do that for you?

Re: [Semanticscuttle-devel] auto-post to delicious (and others)

[Chris L. <ch...@ch...>]

Christian: that makes sense and is how I would want to implement it.
It would also make it easy to make it configurable to potentially post
to other services as well.

Now to find the time!

c

On Sun, Mar 13, 2011 at 10:50 PM, Christian Weiske <cw...@cw...> wrote:
> Hello Chris,
>
>
>> I'm interested in being able to auto-add my bookmarks to Delicious
>> and/or Diigo -- SemanticScuttle works great for me, but many of my
>> colleagues use these other services and I'd like to feed bookmarks
>> there so they can see/use/subscribe. Seems like a relatively easy
>> thing to do, but I didn't want to embark on trying to add this if
>> others were working on it?
>
> There is currently nobody working on that, so I'd be grateful if you
> contributed that feature.
>
> How do you plan to implement it?
>
> My ideas in this direction are to add a "hook" that plugins (via
> config.php) can be registered to and that gets called whenever the
> bookmarks have been saved.
>
> Example:
> config.php:
>> $hooks['bookmark-save'][] = array(
>>     'SemanticScuttle_Plugin_Delicious', 'saveBookmark'
>> );
> That way, other plugins/parts of semanticscuttle are able to hook into
> save bookmark processing as well.
>
> Now the method SemanticScuttle_Plugin_Delicious::saveBookmark will get
> called after a bookmark has been stored/updated in the database, and
> the plugin can do what it wants.
>
> What I would absolutely not want is hard coding the calls to delicious
> in the core SemanticScuttle code. That must be configurable.
>
> --
> Regards/Mit freundlichen Grüßen
> Christian Weiske
>
> -=≡ Geeking around in the name of science since 1982 ≡=-
>
> ------------------------------------------------------------------------------
> Colocation vs. Managed Hosting
> A question and answer guide to determining the best fit
> for your organization - today and in the future.
> http://p.sf.net/sfu/internap-sfd2d
> _______________________________________________
> Semanticscuttle-devel mailing list
> Sem...@li...
> https://lists.sourceforge.net/lists/listinfo/semanticscuttle-devel
>
>

Re: [Semanticscuttle-devel] auto-post to delicious (and others)

[Christian W. <cw...@cw...>]

Hello Chris,


> I'm interested in being able to auto-add my bookmarks to Delicious
> and/or Diigo -- SemanticScuttle works great for me, but many of my
> colleagues use these other services and I'd like to feed bookmarks
> there so they can see/use/subscribe. Seems like a relatively easy
> thing to do, but I didn't want to embark on trying to add this if
> others were working on it?

There is currently nobody working on that, so I'd be grateful if you
contributed that feature.

How do you plan to implement it?

My ideas in this direction are to add a "hook" that plugins (via 
config.php) can be registered to and that gets called whenever the
bookmarks have been saved.

Example:
config.php:
> $hooks['bookmark-save'][] = array(
>     'SemanticScuttle_Plugin_Delicious', 'saveBookmark'
> );
That way, other plugins/parts of semanticscuttle are able to hook into
save bookmark processing as well.

Now the method SemanticScuttle_Plugin_Delicious::saveBookmark will get
called after a bookmark has been stored/updated in the database, and
the plugin can do what it wants.

What I would absolutely not want is hard coding the calls to delicious
in the core SemanticScuttle code. That must be configurable.

-- 
Regards/Mit freundlichen Grüßen
Christian Weiske

-=≡ Geeking around in the name of science since 1982 ≡=-

Re: [Semanticscuttle-devel] Added branch for configurable privacy ....

[Christian W. <cw...@cw...>]

Hello Brett,


> My apologies for the mix-up. I think I've corrected this by pushing to
> the master at origin. However, now the configurable-privacy branch
> shows two parents.
No, not really. I had to do the diff manually since you also seemed to
change the line breaks in all the files.

Did you notice you have a sc subdirectory? You also added files
like Thumbs.db that should never be checked in the repository.

About the code itself:
- You do not need to declare "$defaults = array()" in config.php.dist
  since config.default.php is included before config.php - so
  redeclaring the array would actually erase the default settings for
  privacy.
- Why do you suddenly display "public" for public bookmarks? Since
  public is standard, there is no need to display the text. The
  bookmark status is only shown for non-public bookmarks which helps to
  unclutter the bookmark list display in my eyes.
- Your unit test is flawed since it does not test the effects of a
  privacy config setting at all - you set the variable and use the same
  variable as parameter to the method you are calling.
  The correct way is to set the privacy variable permanently in a
  config and call the scripts you changed (i.e. posts_add.php, edit.php)
  with a set of POST parameters via HTTP. You can then observe if the
  added bookmarks have the configured status. Setting the varible
  permanently is not easy here since you do a HTTP call, but I will add
  that to the master branch quickly.
  Always remember that for the unit tests to be sufficient they need to
  fail when I remove or change a line in your SemanticScuttle source
  code changes. Currently, I can revert the changes in www/bookmark.php
  and all other files without the tests to fail.

Apart from that, the code seems to do what it should.

About the branch:
Since the configurable-privacy branch is totally broken, we should just
delete it and start afresh from master and apply your changes to it.
Shall I do that for you?

-- 
Regards/Mit freundlichen Grüßen
Christian Weiske

-=≡ Geeking around in the name of science since 1982 ≡=-

Re: [Semanticscuttle-devel] Short Links?

[Christian W. <cw...@cw...>]

Hello Chris,


> I see some code for short links and it looks like my URLs in
> SemanticScuttle are getting short links, but I don't see any way to
> use it from the UI... is this a work in progress or am I missing
> something?
The short links do actually work already but the bookmark edit/add form
provides no way to set them yet. This is a task that I wanted to finish
after the HTML_QuickForm2 transition, but I'll also accept patches for
the current code :)

-- 
Regards/Mit freundlichen Grüßen
Christian Weiske

-=≡ Geeking around in the name of science since 1982 ≡=-

[Semanticscuttle-devel] Short Links?

[Chris L. <ch...@ch...>]

I see some code for short links and it looks like my URLs in
SemanticScuttle are getting short links, but I don't see any way to
use it from the UI... is this a work in progress or am I missing
something?

c
--
Chris Lott

[Semanticscuttle-devel] auto-post to delicious (and others)

[Chris L. <ch...@ch...>]

I'm interested in being able to auto-add my bookmarks to Delicious
and/or Diigo -- SemanticScuttle works great for me, but many of my
colleagues use these other services and I'd like to feed bookmarks
there so they can see/use/subscribe. Seems like a relatively easy
thing to do, but I didn't want to embark on trying to add this if
others were working on it?

c
--
Chris Lott

Re: [Semanticscuttle-devel] Privatekey / Opensearch problem / Configurable privacy

[Brett D. <bs...@fr...>]

Hi, Christian.

I ran the test five times consecutively, and the assert failed each
time.

Rgrds,
Brett

On Mon, 7 Mar 2011 20:04:43 +0100, Christian Weiske
<cw...@cw...> wrote:
> Hi Brett,
> 
> 
>> I'm pretty close to having configurable privacy implemented. I'm
>> wondering where I should place my unit test. I was thinking I should
>> add it to BookmarkTest.php in the tests directory, correct?
> Yep, that's the correct one.
> 
> 
>> Also, I ran BookmarkTest.php in PHPUnit and there was one failure:
>> 1) BookmarkTest::testHardCharactersInBookmarks
>> Failed asserting that two strings are equal.
>> --- Expected
>> +++ Actual
>> @@ @@
>> -#{|`^@]³¹¡¿<&é__(-è\_çà)
>> +#{|`^@]
>>
>> I had not changed anything in this file.
> 
> Does it suddenly work when running the tests the second (or third)
> time? i know of one unstable tests (which is crap and needs to be
> fixed).

Re: [Semanticscuttle-devel] Added branch for configurable privacy ....

[Brett D. <bs...@fr...>]

Hello, Christian.

My apologies for the mix-up. I think I've corrected this by pushing to
the master at origin. However, now the configurable-privacy branch shows
two parents.

Rgrds,
Brett

On Mon, 7 Mar 2011 14:00:02 +0100, Christian Weiske
<cw...@cw...> wrote:
> Hi Brett,
> 
> 
>> I've made changes for this feature and have added a test to
>> BookmarkTest.php.
>>
>> This is the first time I've pushed a new branch using git, so I'm not
>> sure if I've done it right. Please have a look at the
>> configurable-privacy branch and let me know if this looks okay.
> 
> No, it's not ok. You created a completely fresh branch starting from
> nothing, but normally you start from an existing branch - i.e. master.
> 
> $ git clone ...
> $ git checkout master
>  -> branch you want to base from
> $ git checkout -b my-new-branch
>  -> create new branch
> $ git checkout my-new-branch
>  -> switch to new branch
> 
> Now you do your changes and commit.
> Currently, I cannot just "diff" the changes to see what you changed.

Re: [Semanticscuttle-devel] Privatekey

[Mark P. <mpe...@gm...>]

Thanks for the great response Christian!  I'll work on these items and get
back to you.  Cheers to all!

On Mon, Mar 7, 2011 at 2:03 PM, Christian Weiske <cw...@cw...> wrote:

> Hi Mark,
>
>
>
> > Christian, I've updated privatekey with the following changes:
> > 1) I added an enable checkbox to allow user to enable/disable the
> > Private RSS feed.
> That's a good addition since it allows people to easily deactivate
> their private key without manually deleting the key.
>
> I saw that you added a second column beside the private key;
> enablePrivateKey. In my eyes this is not necessary since that
> information can be transmitted in the private key column - when it's
> empty/NULL, it's deactivated. Please change that.
>
> > 2) I took out the ajax call in profile.php - I ran into some issues
> > with trying keep the javascript for those who use it.  Didn't think
> > it was that big enough of a deal to worry about it.
> Fine with me.
>
> I've got a little special configuration on my dev system:
> > $sitename = 'b"m.bo\'go';
> The site name here contains both single and double quotes, something
> that helped me discovering quite a number of quoting/escaping bugs. It
> seems that you double-htmlencode the title in the rss feed so that I
> get
> > title="b&amp;quot;m.bo'go: (private) b&amp;quot;m.bo'go"
> instead of the
> > title="b&quot;m.bo'go: Recent bookmarks"
> as it is for the normal feed in the HTML head. Also add a newline
> before the private <link> so the html looks cleaner. You also do not
> escape the & in the link before privatekey=, which makes the page fail
> to validate.
>
> There is no fail message when an unknown privatekey is used.
> SemanticScuttle should send a status 400 (Bad request) with an
> explanatory message.
>
> The user's bookmark list does also not contain a privatekey feed link.
> I think that all pages that link feeds should also link the
> associated private feed (i.e. useful when someone wants to bookmark a
> search that may contain private bookmarks).
>
> And unit tests are a must here:
> - fetch feed with private key, do the private bookmarks appear?
> - same without priv key
> - are the private feed links on all relevant pages when logged in and
>  the user has a private key?
> - same for logged out -> no private links
> - same for logged in bug no private key -> no private feed links
>
>
> > 3) on a separate commit, I took a stab at updating the getBookmarks()
> > function to make the queries a little more efficient.  The base of the
> > problem was doing both a GROUP BY and an ORDER BY in the same query
> > (as well as the use of multiple tables on the same query).  I got
> > spoiled on Oracle because it tends to be a little more efficient.
> > MySQL as well as other RDBS's don't optimize it enough to make it
> > very efficient.  So I tweaked the queries to get around some of the
> > issues.  I haven't tested it on MySQL 4 so I can't confirm if the
> > query will work on that version.  There was one component that I
> > couldn't make more efficient and that is searching by a tag, or by a
> > user, or searching.  Those queries are simply too complex to make
> > efficient.  However, they are fast if there is only a small
> > resultset. Anyway, outside of doing a full redesign of the data
> > tables, this is the best I could come up with.  My last commit
> > contains the updates to the user table and the bookmarks table.  I
> > simply added an additional index on the bookmarks table to help with
> > the query.  Please let me know if you have any questions.
>
> Do the tests do still run? I get a bunch of errors when running them.
> Please also do that in a different branch; we're using one branch per
> feature which makes it easier to merge the relevant features in the
> master branch later. Just imagine that the private key feature is
> ready, but your sql optimizations are buggy but we want to release a
> new version. If we have clean feature branches, we just merge the ones
> that are ready and are done. Currently this is not possible since an
> unfinished feature is in the same branch as the other one.
>
> --
> Regards/Mit freundlichen Grüßen
> Christian Weiske
>
> -=≡ Geeking around in the name of science since 1982 ≡=-
>
>
> ------------------------------------------------------------------------------
> What You Don't Know About Data Connectivity CAN Hurt You
> This paper provides an overview of data connectivity, details
> its effect on application quality, and explores various alternative
> solutions. http://p.sf.net/sfu/progress-d2d
> _______________________________________________
> Semanticscuttle-devel mailing list
> Sem...@li...
> https://lists.sourceforge.net/lists/listinfo/semanticscuttle-devel
>
>