naviserver-devel

[naviserver-devel] nsmemcached

[Daniel S. <moo...@av...>]

I have noticed a quirk in the nsmemcache module.  On the very first
"get" command from a fresh restart of nsd, it will not read past about
1379 bytes and returns a partial chunk of data.  The length is
correct, but the data past 1379 is corrupt.

To set, I performed:

   ns_memcache set bigdata [string repeat "X" 2048]

Next, I killed and restarted nsd and performed:

   ns_puts [ns_memcache get bigdata text]<br>        ' returns 1
   ns_puts [string length $text]<br/>                           ' returns 2048
   ns_puts $text

All subsequent reads are correct.  Only the very first read is affected.

Daniel

-- 
| ---------------------------------------------------------------
| Daniel P. Stasinski | http://www.saidsimple.com
| moo...@av... | http://www.disabilities-r-us.com
| XMMP: moo...@av... | http://www.avenues.org
| Google Talk: mooooooo | http://www.scriptkitties.com

Re: [naviserver-devel] nsmemcached

[Vlad S. <vl...@cr...>]

Just tested it, works fine regardless is it first time or not

I am using latest version from CVS of server and the module on Archlinux 
2.6.24-ARCH

Daniel Stasinski wrote:
> I have noticed a quirk in the nsmemcache module.  On the very first
> "get" command from a fresh restart of nsd, it will not read past about
> 1379 bytes and returns a partial chunk of data.  The length is
> correct, but the data past 1379 is corrupt.
> 
> To set, I performed:
> 
>    ns_memcache set bigdata [string repeat "X" 2048]
> 
> Next, I killed and restarted nsd and performed:
> 
>    ns_puts [ns_memcache get bigdata text]<br>        ' returns 1
>    ns_puts [string length $text]<br/>                           ' returns 2048
>    ns_puts $text
> 
> All subsequent reads are correct.  Only the very first read is affected.
> 
> Daniel
> 

Re: [naviserver-devel] nsmemcached

[Stephen D. <sd...@gm...>]

On Tue, May 27, 2008 at 6:30 PM, Vlad Seryakov <vl...@cr...> wrote:
>
> Daniel Stasinski wrote:
>>
>> I have noticed a quirk in the nsmemcache module.  On the very first
>> "get" command from a fresh restart of nsd, it will not read past about
>> 1379 bytes and returns a partial chunk of data.  The length is
>> correct, but the data past 1379 is corrupt.
>>
>> To set, I performed:
>>
>>    ns_memcache set bigdata [string repeat "X" 2048]
>>
>> Next, I killed and restarted nsd and performed:
>>
>>    ns_puts [ns_memcache get bigdata text]<br>        ' returns 1
>>    ns_puts [string length $text]<br/>                           ' returns 2048
>>    ns_puts $text
>>
>> All subsequent reads are correct.  Only the very first read is affected.
>>
>
> Just tested it, works fine regardless is it first time or not
>
> I am using latest version from CVS of server and the module on Archlinux
> 2.6.24-ARCH
>


Looks like the 'expires' and 'flags' variables might be uninitialised
for the cmdSet etc. commands if values aren't passed from Tcl.

Re: [naviserver-devel] nsmemcached

[Daniel S. <moo...@av...>]

On Tue, May 27, 2008 at 10:30 AM, Vlad Seryakov <vl...@cr...> wrote:
> Just tested it, works fine regardless is it first time or not

I spent an hour going through it and tracked down the problem.

   rc = mc_conn_read(conn, BUFSIZE, 1, &line);

On the first read, the line arg is set to point to an offset within
conn->ds.dstring.   The problem is that mc_conn_read() there are calls
to Ns_DStringSetLength() which can (and does) relocate
conn->ds.dstring, therefor leaving &line pointing to a deallocated
memory block.   The only time ds.dstring is in a static location is
when it's 200 bytes or less.

I just happened come across a perhaps a platform specific set of data
that could duplicate the problem over and over.

Will patch and update cvs.

Daniel

-- 
| ---------------------------------------------------------------
| Daniel P. Stasinski | http://www.saidsimple.com
| moo...@av... | http://www.disabilities-r-us.com
| XMMP: moo...@av... | http://www.avenues.org
| Google Talk: mooooooo | http://www.scriptkitties.com

Re: [naviserver-devel] nsmemcached

[Vlad S. <vl...@cr...>]

Nice catch

Daniel Stasinski wrote:
> On Tue, May 27, 2008 at 10:30 AM, Vlad Seryakov <vl...@cr...> wrote:
>> Just tested it, works fine regardless is it first time or not
> 
> I spent an hour going through it and tracked down the problem.
> 
>    rc = mc_conn_read(conn, BUFSIZE, 1, &line);
> 
> On the first read, the line arg is set to point to an offset within
> conn->ds.dstring.   The problem is that mc_conn_read() there are calls
> to Ns_DStringSetLength() which can (and does) relocate
> conn->ds.dstring, therefor leaving &line pointing to a deallocated
> memory block.   The only time ds.dstring is in a static location is
> when it's 200 bytes or less.
> 
> I just happened come across a perhaps a platform specific set of data
> that could duplicate the problem over and over.
> 
> Will patch and update cvs.
> 
> Daniel
> 

Re: [naviserver-devel] nsmemcached

[Vlad S. <vl...@cr...>]

> 
> 
> Looks like the 'expires' and 'flags' variables might be uninitialised
> for the cmdSet etc. commands if values aren't passed from Tcl.

In CVS version 1.5 they are initialized with 0

Re: [naviserver-devel] nsmemcached

[Stephen D. <sd...@gm...>]

On Tue, May 27, 2008 at 8:12 PM, Vlad Seryakov <vl...@cr...> wrote:
>>
>>
>> Looks like the 'expires' and 'flags' variables might be uninitialised
>> for the cmdSet etc. commands if values aren't passed from Tcl.
>
> In CVS version 1.5 they are initialized with 0
>


Thinking about flags...

If you remove -flags from the Tcl interface then you could use it to
distinguish between urf8 and byte arrays. ATM all data is sent as a
byte array, which is flexible. But probably a lot of the data will be
utf8 strings, and in that case it gets converted on the way in, and
again on the way out.  It would be more efficient to save the data in
it's native format.