Steel Bank Common Lisp

On 10 December 2011 12:15, Tamas Papp <tkpapp@...> wrote:

> Redefining structures (with a different layout, or factoring out parts
> to an included structure, etc) sometimes causes glitches like
>
> The value
>  #S(AXIS
>     :TITLE "y"
>     :TITLE-MARGIN 20
>     :DECORATION-MARGIN #S(XY :X 25 :Y 45)
>     :MARK 3
>     :ANNOTATION-DISTANCE 6
>     :SPACING #S(XY :X 50 :Y 30))
> is not of type
>  AXIS.
>   [Condition of type TYPE-ERROR]

This is pretty much expected when redefining structures unless you're
exceedingly careful.

You have in your image instances of structures using the old layout,
which then break when used with the new accessors and predicates.
(Usual suspects include DEFVARs, LOAD-TIME-VALUEs, DEFGLOBALs, closed
over values, EQL-specializers, and slots with :ALLOCATION :CLASS)

Now, in /some/ cases it would be possibly to destructively update the
instance to match the new layout. (Eg. same slots but in different
order.) It is not, however, possible in the general.

For things where allocation or slot access speed is not at premium
(AXIS above doesn't look like it), you're propably better off using
DEFCLASS, since it allows instances to be updated.

You may also consider writing a DEFOBJECT that can expand to either
DEFSTRUCT or DEFCLASS as needed, so that you can switch to DEFCLASS
when you're working on the system, and DEFSTRUCT when deploying.

Cheers,

 - nikodemus

On Sat, 10 Dec 2011, Nikodemus Siivola wrote:

> On 10 December 2011 12:15, Tamas Papp <tkpapp@...> wrote:
> 
> > Redefining structures (with a different layout, or factoring out parts
> > to an included structure, etc) sometimes causes glitches like
> >
> > The value
> >  #S(AXIS
> >     :TITLE "y"
> >     :TITLE-MARGIN 20
> >     :DECORATION-MARGIN #S(XY :X 25 :Y 45)
> >     :MARK 3
> >     :ANNOTATION-DISTANCE 6
> >     :SPACING #S(XY :X 50 :Y 30))
> > is not of type
> >  AXIS.
> >   [Condition of type TYPE-ERROR]
> 
> This is pretty much expected when redefining structures unless you're
> exceedingly careful.
> 
> You have in your image instances of structures using the old layout,
> which then break when used with the new accessors and predicates.
> (Usual suspects include DEFVARs, LOAD-TIME-VALUEs, DEFGLOBALs, closed
> over values, EQL-specializers, and slots with :ALLOCATION :CLASS)

I thought about that, but I got this error even when I regenerated the
instances according to the new layout.  Of course, I don't know how
deeply SBCL inlines the layout, so it could happen that I forgot to
recompile something, but I did an asdf reload and still got the error
message.

> For things where allocation or slot access speed is not at premium
> (AXIS above doesn't look like it), you're propably better off using
> DEFCLASS, since it allows instances to be updated.

Speed is irrelevant per se in this application, I am using structures
because of the extremely nice read/write syntax (I guess one could
write a mixin class with some MOP magic to emulate this though, but am
not comfortable enough with the MOP just yet).

> You may also consider writing a DEFOBJECT that can expand to either
> DEFSTRUCT or DEFCLASS as needed, so that you can switch to DEFCLASS
> when you're working on the system, and DEFSTRUCT when deploying.

Great suggestion, I think I may end up doing that.

Thanks,

Tamas

On 10 December 2011 17:10, Tamas Papp <tkpapp@...> wrote:

> I thought about that, but I got this error even when I regenerated the
> instances according to the new layout.

Do you inline MAKE-AXIS? If so, you need to also recompile sites that
call it, or they will use the old layout.

Cheers,

 -- nikodemus

On Dec 10, 2011, at 16:10 , Tamas Papp wrote:
> On Sat, 10 Dec 2011, Nikodemus Siivola wrote:
> 
>> On 10 December 2011 12:15, Tamas Papp <tkpapp@...> wrote:
>> 
>>> Redefining structures (with a different layout, or factoring out parts
>>> to an included structure, etc) sometimes causes glitches like
>>> 
>>> The value
>>>  #S(AXIS
>>>     :TITLE "y"
>>>     :TITLE-MARGIN 20
>>>     :DECORATION-MARGIN #S(XY :X 25 :Y 45)
>>>     :MARK 3
>>>     :ANNOTATION-DISTANCE 6
>>>     :SPACING #S(XY :X 50 :Y 30))
>>> is not of type
>>>  AXIS.
>>>   [Condition of type TYPE-ERROR]
>> 
>> This is pretty much expected when redefining structures unless you're
>> exceedingly careful.
>> 
>> You have in your image instances of structures using the old layout,
>> which then break when used with the new accessors and predicates.
>> (Usual suspects include DEFVARs, LOAD-TIME-VALUEs, DEFGLOBALs, closed
>> over values, EQL-specializers, and slots with :ALLOCATION :CLASS)
> 
> I thought about that, but I got this error even when I regenerated the
> instances according to the new layout.  Of course, I don't know how
> deeply SBCL inlines the layout, so it could happen that I forgot to
> recompile something, but I did an asdf reload and still got the error
> message.


Quote from The Hyperspec:

"The consequences of redefining a defstruct structure are undefined."

Maybe this has been mentioned already, and maybe SBCL makes more of an effort to support structure redefinition, but still...

On Sat, 10 Dec 2011, Raymond Wiker wrote:

> On Dec 10, 2011, at 16:10 , Tamas Papp wrote:
> 
> > I thought about that, but I got this error even when I regenerated the
> > instances according to the new layout.  Of course, I don't know how
> > deeply SBCL inlines the layout, so it could happen that I forgot to
> > recompile something, but I did an asdf reload and still got the error
> > message.
> 
> 
> Quote from The Hyperspec:
> 
> "The consequences of redefining a defstruct structure are undefined."
> 
> Maybe this has been mentioned already, and maybe SBCL makes more of
> an effort to support structure redefinition, but still... 

Sure, the standard basically says that I should not do it, and should
not complain if I do and bad things happen.  Which is a reasonable
attitude I guess.

Is there a way I could have quick-dirty print-object methods defined
for classes?  Does not need to be readable, but it is so nice to punt
on the nice printed representations of structures when I am trace'ing.
Otherwise, I would be much better off with classes.  Maybe someone
already has a mixin class or something similar in a library that I
have missed.

Tamas

On Dec 10, 2011, at 20:02 , Tamas Papp wrote:
> On Sat, 10 Dec 2011, Raymond Wiker wrote:
> 
>> On Dec 10, 2011, at 16:10 , Tamas Papp wrote:
>> 
>>> I thought about that, but I got this error even when I regenerated the
>>> instances according to the new layout.  Of course, I don't know how
>>> deeply SBCL inlines the layout, so it could happen that I forgot to
>>> recompile something, but I did an asdf reload and still got the error
>>> message.
>> 
>> 
>> Quote from The Hyperspec:
>> 
>> "The consequences of redefining a defstruct structure are undefined."
>> 
>> Maybe this has been mentioned already, and maybe SBCL makes more of
>> an effort to support structure redefinition, but still... 
> 
> Sure, the standard basically says that I should not do it, and should
> not complain if I do and bad things happen.  Which is a reasonable
> attitude I guess.
> 
> Is there a way I could have quick-dirty print-object methods defined
> for classes?  Does not need to be readable, but it is so nice to punt
> on the nice printed representations of structures when I am trace'ing.
> Otherwise, I would be much better off with classes.  Maybe someone
> already has a mixin class or something similar in a library that I
> have missed.
> 
> Tamas


Maybe something like this (which uses a bit of MOP, namely class-slots and slot-definition-name):

(defmethod print-object ((object my-base-class-or-mixin) stream)
  (print-unreadable-object (object stream :type t :identity t)
    (dolist (slot-def (class-slots (class-of object)))
      (let ((slot-name (slot-definition-name slot-def)))
        (format stream "~&~a: ~a~%" slot-name (slot-value object slot-name))))))

On Sat, 10 Dec 2011, Raymond Wiker wrote:

> On Dec 10, 2011, at 20:02 , Tamas Papp wrote:
> > On Sat, 10 Dec 2011, Raymond Wiker wrote:
> > 
> >> On Dec 10, 2011, at 16:10 , Tamas Papp wrote:
> >> 
> >>> I thought about that, but I got this error even when I regenerated the
> >>> instances according to the new layout.  Of course, I don't know how
> >>> deeply SBCL inlines the layout, so it could happen that I forgot to
> >>> recompile something, but I did an asdf reload and still got the error
> >>> message.
> >> 
> >> 
> >> Quote from The Hyperspec:
> >> 
> >> "The consequences of redefining a defstruct structure are undefined."
> >> 
> >> Maybe this has been mentioned already, and maybe SBCL makes more of
> >> an effort to support structure redefinition, but still... 
> > 
> > Sure, the standard basically says that I should not do it, and should
> > not complain if I do and bad things happen.  Which is a reasonable
> > attitude I guess.
> > 
> > Is there a way I could have quick-dirty print-object methods defined
> > for classes?  Does not need to be readable, but it is so nice to punt
> > on the nice printed representations of structures when I am trace'ing.
> > Otherwise, I would be much better off with classes.  Maybe someone
> > already has a mixin class or something similar in a library that I
> > have missed.
> > 
> > Tamas
> 
> 
> Maybe something like this (which uses a bit of MOP, namely class-slots and slot-definition-name):
> 
> (defmethod print-object ((object my-base-class-or-mixin) stream)
>   (print-unreadable-object (object stream :type t :identity t)
>     (dolist (slot-def (class-slots (class-of object)))
>       (let ((slot-name (slot-definition-name slot-def)))
>         (format stream "~&~a: ~a~%" slot-name (slot-value object slot-name))))))

Thanks, I also found fare-mop:simple-print-object-mixin which does
something like this.

Best,

Tamas

>Is there a tutorial somewhere on writing non-ascii characters to a file?
>
>I have latin-1 encoded characters in a db, and I'm not sure how to write
>those out.  I can use flexi-streams:string-to-octets and back, but how can
>I use with-open-file to write the correct characters to a text file?
>
>Sorry if this is a simple question, but this is my first foray into
>encoding, and I'm sticking with SBCL to do it.
>

you can either say :external-format or use :element-type

In the first case you specify encoding, in the second type you simply 
use (unsigned-byte 8) and write octets

On 4 December 2011 17:48,  <jcm@...> wrote:
> Is there a tutorial somewhere on writing non-ascii characters to a file?
>
> I have latin-1 encoded characters in a db, and I'm not sure how to write
> those out.  I can use flexi-streams:string-to-octets and back, but how can
> I use with-open-file to write the correct characters to a text file?

So, first you get stuff from the DB.

If it arrives as raw octets you can just use pass :ELEMENT-TYPE
'(UNSIGNED-BYTE 8) WITH-OPEN-FILE to write it to file using its
original (in this case latin-1) encoding. If you must write it in, say
:UTF-8 instead, you need to first convert it to a string using eg.
SB-EXT:OCTETS-TO-STRING with :EXTERNAL-FORMAT :LATIN-1 and then
proceed as in the next paragraph.

If it arrives as correctly decoded string, pass :EXTERNAL-FORMAT
:UTF-8 to WITH-OPEN-FILE instead. (Or :LATIN-1, or whatever encoding
you want to write it in.)

If it arrives as an /incorrectly/ decoded string, then you need to
figure out where it goes wrong, and make sure the right external
format is used there.

Cheers,

 -- nikodemus

Thank you both for your answer.  The initial issue I had was quickly solved.

I'm trying to use this knowledge for a web app as well.  I have to present
a web page in a number of languages, and accept input.  I might even have
to allow for (as an example) someone at an arabic configured machine to
enter their name in mandarin.  I would assume I need everything in UTF-8
at that point.  But how can I get those characters to appear correctly in
the terminal, file or web browser?

For latin-1, it's fine.  I can see the brazilian characters in my user's
name.  But will the same approach handle non-roman characters just as
easily?

On 5 December 2011 10:27, Tamas Papp <tkpapp@...> wrote:

> (defcrazy 5000)

You can request a larger control stack by doing eg.

  sbcl --control-stack-size 128

which nets you 128 megabytes of stack, enough to compiler this monster.

Cheers,

 -- Nikodemus

On 5 December 2011 10:33, Nikodemus Siivola <nikodemus@...> wrote:

>  sbcl --control-stack-size 128

I should probably also mention that /all/ threads have the same size
control stack at the moment, so spawner beware.

Cheers,

 -- nikodemus

On Mon, 05 Dec 2011, Nikodemus Siivola wrote:

> On 5 December 2011 10:27, Tamas Papp <tkpapp@...> wrote:
> 
> > (defcrazy 5000)
> 
> You can request a larger control stack by doing eg.
> 
>   sbcl --control-stack-size 128
> 
> which nets you 128 megabytes of stack, enough to compiler this monster.

Is there a simple heuristic that would allow me to estimate the
required control stack, eg as a function of elementary operations or
anything else?  Saying that it is O(n^a) with some a would be enough,
I just need it for intuition.

I am asking because some problems that I am working on are actually
even more complex when translated into elementary operations (~1e5),
_but_ there may be a way to break them up (called "checkpointing" in
the AD literature, essentially I would just break up the function
using knowledge on the structure of the problem), but I need to know
how far need to go with that approach.

Tamas

In article <20111205084920.GA16906@...>,
 Tamas Papp <tkpapp@...> wrote:

> On Mon, 05 Dec 2011, Nikodemus Siivola wrote:
> 
> > On 5 December 2011 10:27, Tamas Papp <tkpapp@...> wrote:
> > 
> > > (defcrazy 5000)
> > 
> > You can request a larger control stack by doing eg.
> > 
> >   sbcl --control-stack-size 128
> > 
> > which nets you 128 megabytes of stack, enough to compiler this monster.
> 
> Is there a simple heuristic that would allow me to estimate the
> required control stack, eg as a function of elementary operations or
> anything else?  Saying that it is O(n^a) with some a would be enough,
> I just need it for intuition.

I'm pretty sure that the maximum recursion depth is O(n). Compilation 
time, however, I'm also pretty sure is at least quadratic (even with 
speed optimisations disabled).

Not flattening nested scopes might help a bit here, i.e.:

 (let ((x (let ((y ...)) ...))) ...)
instead of
 (let* ((y ...)) (x ...)) ...)

If multiple values are needed, multiple-value-bind should work just the 
same.  ISTM that'll be closer to the original expression as well.

> I am asking because some problems that I am working on are actually
> even more complex when translated into elementary operations (~1e5),
> _but_ there may be a way to break them up (called "checkpointing" in
> the AD literature, essentially I would just break up the function
> using knowledge on the structure of the problem), but I need to know
> how far need to go with that approach.

In that case, something else that could help is putting broken-up pieces 
in notinline local functions. That'll easily get you specialised calling 
conventions (everything unboxed, a la C, but with better support for 
multiple return values), but should still keep the compiler from having 
enough static information to derive further information for eons.

Paul Khuong

Thanks everyone.

The issue wasn't that it broke, its that I got different mathematical results - I bound the two lambda's as a list to a debug variable so I could show the code being run, the function actually returned two values.

I'm running 1.0.53 now.  The issue I've seen is a confluence of speed and safety declarations, as well as floating point mode settings.

I believe I've now resolved my particular issue.

Thanks again

-----Original Message-----
>From: Stig Hemmer <stighemmer@...>
>Sent: Dec 1, 2011 9:33 AM
>To: billpeople@...
>Cc: Nikodemus Siivola <nikodemus@...>, sbcl-help@...
>Subject: Re: [Sbcl-help] compile function v. dynamic eval of defun
>
>Your make-rk4-model returns a LIST of two lambdas.  The calling
>function then tries to pick these apart using MULTIPLE VALUES.
>
>Either return multiple values using VALUES or pick apart the list
>using DESTRUCTURING-BIND.
>
>Stig Hemmer
>
>On 11/30/11, billpeople@... <billpeople@...> wrote:
>> Thanks for your rapid reply.
>>
>> I realize a reproducible test case is what you need to go further, and I'll
>> try to develop that - and I'll move to 1.0.53 as well as trying this on a
>> linux box.
>>
>> That said, I was trying to establish whether I should expect different
>> results (or if this is a known issue) from the two different methods of
>> producing a compiled function dynamically, hence showing the compilations
>> are made in essentially identical lexical environments, and that the code
>> being compiled is basically identical.
>>
>> To directly answer your question on EQUATION-AGGREGATOR, here's the source,
>> and in this case, fn is bound to #'+:
>>
>> (defun equation-aggregator (fn &rest args)
>>   (declare (optimize (speed 3) (safety 0)))
>>   (let ((tmp (delete nil args)))
>>     (if tmp
>> 	(reduce fn tmp)
>> 	0d0)))
>>
>> -----Original Message-----
>>>From: Nikodemus Siivola <nikodemus@...>
>>>Sent: Nov 30, 2011 4:04 AM
>>>To: billpeople@...
>>>Cc: sbcl-help@...
>>>Subject: Re: [Sbcl-help] compile function v. dynamic eval of defun
>>>
>>>On 30 November 2011 11:08,  <billpeople@...> wrote:
>>>
>>>> I haven't been able to generate a simpler case to show the issue - the
>>>> simpler cases
>>>> all work as they ought, yielding identical results.
>>>
>>>Hi,
>>>
>>>First of, you may want to try with 1.0.53 (or current master), as FP
>>>traps on OS X have seen some work since 1.0.52.
>>>
>>>Second, can you please provide definitions for the functions called in
>>>your test-cast, and a sample run where you show how to call them and
>>>the results you get.
>>>
>>>Ie. something I can just copy and paste into my repl (or a file I can
>>>load, etc) to see the issue without needing to wonder what
>>>EQUATION-AGGREGATOR is or what kind of arguments I'm supposed to
>>>provide.
>>>
>>>Cheers,
>>>
>>> -- Nikodemus
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> All the data continuously generated in your IT infrastructure
>> contains a definitive record of customers, application performance,
>> security threats, fraudulent activity, and more. Splunk takes this
>> data and makes sense of it. IT sense. And common sense.
>> http://p.sf.net/sfu/splunk-novd2d
>> _______________________________________________
>> Sbcl-help mailing list
>> Sbcl-help@...
>> https://lists.sourceforge.net/lists/listinfo/sbcl-help
>>

Thanks everyone.

The issue wasn't that it broke, its that I got different mathematical results - I bound the two lambda's as a list to a debug variable so I could show the code being run, the function actually returned two values.

I'm running 1.0.53 now.  The issue I've seen is a confluence of speed and safety declarations, as well as floating point mode settings.

I believe I've now resolved my particular issue.

Thanks again

-----Original Message-----
>From: Stig Hemmer <stighemmer@...>
>Sent: Dec 1, 2011 9:33 AM
>To: billpeople@...
>Cc: Nikodemus Siivola <nikodemus@...>, sbcl-help@...
>Subject: Re: [Sbcl-help] compile function v. dynamic eval of defun
>
>Your make-rk4-model returns a LIST of two lambdas.  The calling
>function then tries to pick these apart using MULTIPLE VALUES.
>
>Either return multiple values using VALUES or pick apart the list
>using DESTRUCTURING-BIND.
>
>Stig Hemmer
>
>On 11/30/11, billpeople@... <billpeople@...> wrote:
>> Thanks for your rapid reply.
>>
>> I realize a reproducible test case is what you need to go further, and I'll
>> try to develop that - and I'll move to 1.0.53 as well as trying this on a
>> linux box.
>>
>> That said, I was trying to establish whether I should expect different
>> results (or if this is a known issue) from the two different methods of
>> producing a compiled function dynamically, hence showing the compilations
>> are made in essentially identical lexical environments, and that the code
>> being compiled is basically identical.
>>
>> To directly answer your question on EQUATION-AGGREGATOR, here's the source,
>> and in this case, fn is bound to #'+:
>>
>> (defun equation-aggregator (fn &rest args)
>>   (declare (optimize (speed 3) (safety 0)))
>>   (let ((tmp (delete nil args)))
>>     (if tmp
>> 	(reduce fn tmp)
>> 	0d0)))
>>
>> -----Original Message-----
>>>From: Nikodemus Siivola <nikodemus@...>
>>>Sent: Nov 30, 2011 4:04 AM
>>>To: billpeople@...
>>>Cc: sbcl-help@...
>>>Subject: Re: [Sbcl-help] compile function v. dynamic eval of defun
>>>
>>>On 30 November 2011 11:08,  <billpeople@...> wrote:
>>>
>>>> I haven't been able to generate a simpler case to show the issue - the
>>>> simpler cases
>>>> all work as they ought, yielding identical results.
>>>
>>>Hi,
>>>
>>>First of, you may want to try with 1.0.53 (or current master), as FP
>>>traps on OS X have seen some work since 1.0.52.
>>>
>>>Second, can you please provide definitions for the functions called in
>>>your test-cast, and a sample run where you show how to call them and
>>>the results you get.
>>>
>>>Ie. something I can just copy and paste into my repl (or a file I can
>>>load, etc) to see the issue without needing to wonder what
>>>EQUATION-AGGREGATOR is or what kind of arguments I'm supposed to
>>>provide.
>>>
>>>Cheers,
>>>
>>> -- Nikodemus
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> All the data continuously generated in your IT infrastructure
>> contains a definitive record of customers, application performance,
>> security threats, fraudulent activity, and more. Splunk takes this
>> data and makes sense of it. IT sense. And common sense.
>> http://p.sf.net/sfu/splunk-novd2d
>> _______________________________________________
>> Sbcl-help mailing list
>> Sbcl-help@...
>> https://lists.sourceforge.net/lists/listinfo/sbcl-help
>>