gdalgorithms-list

Re: [Algorithms] General purpose task parallel threading approach

[Conor S. <bor...@ya...>]

"Other than classroom examples, software is NOT written through
composition of 'general components'. And even in the small subset
of software writing where your statement may hold true, writing
those components was a very small amount of work of the total."

No, every piece of software in a modern language is written through composition of "general components", from the machine code op-codes, up to the language constructs they come from, up to the libraries, patterns and algorithms they're constructed with. Unless you dislike modern conveniences like "arrays" and "for loops", or you feel the need to write binary searches, quicksorts, disk-io and image-decompression routines inline every time you come across the need for them (in a language of silicon, plastic and copper no less!). Those may be small pieces of code, but serialization frameworks, database frameworks and communications frameworks that are used in extremely general ways and can often be much larger than the small pieces of business logic that ties them together. 

Someone writes the middleware and the infrastructure code, the VHDL, the compilers, the drivers, the operating system and the standard libraries, it may not be you or the coder next to you, but I can assure you there is a huge amount of it in the world (have a look on sourceforge at the number of different component libraries relative the number of "application" projects) and it's a lot of work for somebody. Then again, the better general components you have, the less often you need to re-write them (or go looking for them) and the rarer they might seem from the perspective of someone who spends a lot of time solving specific problems with them. 

Cheers,
Conor


      Yahoo!7 recommends that you update your browser to the new Internet Explorer 8.Get it now.

Re: [Algorithms] General purpose task parallel threading approach

[<chr...@pl...>]

> "Other than classroom examples, software is NOT written through
> composition of 'general components'. And even in the small subset
> of software writing where your statement may hold true, writing
> those components was a very small amount of work of the total."
>
> No, every piece of software in a modern language is written through
> composition of "general components", from the machine code op-codes,
> up to the language constructs they come from, up to the libraries,
> patterns and algorithms they're constructed with.

The fact that you have to rely on labelling assembly instructions
and for-loops as "general components" to provide support for your
initial statement shows how weak your statement was.


Christer Ericson, Director of Tools and Technology
Sony Computer Entertainment, Santa Monica

Re: [Algorithms] General purpose task parallel threading approach

[Ola O. <ola...@gm...>]

You know guys, the sweng-gamedev has been without one of these endless flame 
wars for quite some time, why don't you move it there and leave the 
algorithms list in peace?

peace
.ola

----- Original Message ----- 
From: "Tony Cox" <to...@mi...>
To: "Game Development Algorithms" <gda...@li...>
Sent: Thursday, April 16, 2009 9:02 PM
Subject: Re: [Algorithms] General purpose task parallel threading approach


>I think you guys are getting tied up in an argument about semantics.
>
> The real point here, IMHO, is that software development is a *tradeoff*. 
> There are many things we'd like in any piece of software:
>
> - Good performance (both in time and space)
> - Robustness
> - Maintainability
> - Shipped on time
> - Shipped on budget
> - Features
> - Interoperability
> - Testability
> - Portability
> - Reusability (perhaps this means being more rather than less general)
> - And many others.
>
> Clearly, holding all other variables equals, it's better to have more of 
> these good things than less. But you don't get to hold the other variables 
> equal - you're always trading off things against each other. Would I like 
> any piece of code I write to be more general purpose and more reusable? 
> Sure, but (a) do I even know that how to achieve that goal, and (b) am I 
> willing to trade off something else to get it? Maybe not. Perhaps almost 
> always not.
>
> - Tony
>
> -----Original Message-----
> From: chr...@pl... 
> [mailto:chr...@pl...]
> Sent: Thursday, April 16, 2009 11:14 AM
> To: Game Development Algorithms
> Subject: Re: [Algorithms] General purpose task parallel threading approach
>
>> "Other than classroom examples, software is NOT written through
>> composition of 'general components'. And even in the small subset
>> of software writing where your statement may hold true, writing
>> those components was a very small amount of work of the total."
>>
>> No, every piece of software in a modern language is written through
>> composition of "general components", from the machine code op-codes,
>> up to the language constructs they come from, up to the libraries,
>> patterns and algorithms they're constructed with.
>
> The fact that you have to rely on labelling assembly instructions
> and for-loops as "general components" to provide support for your
> initial statement shows how weak your statement was.
>
>
> Christer Ericson, Director of Tools and Technology
> Sony Computer Entertainment, Santa Monica
>
>
> ------------------------------------------------------------------------------
> Stay on top of everything new and different, both inside and
> around Java (TM) technology - register by April 22, and save
> $200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
> 300 plus technical and hands-on sessions. Register today.
> Use priority code J9JMT32. http://p.sf.net/sfu/p
> _______________________________________________
> GDAlgorithms-list mailing list
> GDA...@li...
> https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
> Archives:
> http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
>
>
> ------------------------------------------------------------------------------
> Stay on top of everything new and different, both inside and
> around Java (TM) technology - register by April 22, and save
> $200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
> 300 plus technical and hands-on sessions. Register today.
> Use priority code J9JMT32. http://p.sf.net/sfu/p
> _______________________________________________
> GDAlgorithms-list mailing list
> GDA...@li...
> https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
> Archives:
> http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list 

Re: [Algorithms] General purpose task parallel threading approach

[Conor S. <bor...@ya...>]

"The fact that you have to rely on labelling assembly instructions
and for-loops as "general components" to provide support for your
initial statement shows how weak your statement was."

No, that's a very good strawman you have there though. The point was that programming is nothing but composition of general components from the ground up. Assembly instructions and for-loops weren't all I mentioned, or indeed what I emphasised. Libraries and frameworks are rather strong arguments for generality and composition.

Cheers,
Conor 


      The new Internet Explorer 8 optimised for Yahoo!7: Faster, Safer, Easier.

Re: [Algorithms] General purpose task parallel threading approach

[Gregory J. <gj...@da...>]

>> "Other than classroom examples, software is NOT written through
>> composition of 'general components'. And even in the small subset
>> of software writing where your statement may hold true, writing
>> those components was a very small amount of work of the total."
>>
>> No, every piece of software in a modern language is written through
>> composition of "general components", from the machine code op-codes,
>> up to the language constructs they come from, up to the libraries,
>> patterns and algorithms they're constructed with. 

> The fact that you have to rely on labelling assembly instructions
> and for-loops as "general components" to provide support for your
> initial statement shows how weak your statement was.

Agreed. "Reductio ad absurdum" is a technique most often used by parties who
cannot defend their position within the original bounds of the debate, and
therefore find they need to expand the playing field, so to speak. 

Perhaps the meta-issue is that there is no agreement in the past 205 posts
on what "general" actually means? According to dictionary.com, "general" can
be said to mean "not specific or definite". Using this definition then,
anything above the subatomic level cannot be said to be "general" because
the instant protons, neutrons and electrons assemble themselves into a
particular number and order of each, they achieve specificity and discard
generality.

Greg

Re: [Algorithms] General purpose task parallel threading approach

[Conor S. <bor...@ya...>]

"Agreed. "Reductio ad absurdum" is a technique most often used by parties who
cannot defend their position within the original bounds of the debate, and
therefore find they need to expand the playing field, so to speak. "

Reductio ad absurdum is a great technique often used to show that another argument is absurd by taking your own argument to a level of absurdity. It's still a formal logical argument and it's still a formal logical fallacy to dismiss a reductio ad absurdum. Just because it's silly, doesn't mean it's not perfectly valid. The point was that programming is nothing but composition of general components from the very low to the very high level.

"Perhaps the meta-issue is that there is no agreement in the past 205 posts
on what "general" actually means? According to dictionary.com, "general" can
be said to mean "not specific or definite". Using this definition then,
anything above the subatomic level cannot be said to be "general" because
the instant protons, neutrons and electrons assemble themselves into a
particular number and order of each, they achieve specificity and discard
generality."


Now we are getting back to the crux of the matter and has been mentioned. Generality in my mind is where you can use a component to solve multiple different unforeseen problems, not just the problem you may originally have used it to solve. For example, the generic functions and collections in the standard library are good examples of that kind of generality without much loss of functionality. It's not a good idea every time to make things that general, but it is a good idea enough of the time not to dismiss the idea out of hand by assigning the absolute truth of a "thou shalt not" to it. That was why originally I said, it depends on what you're doing an if it makes economic sense to do it. 

For a lot of companies that are selling frameworks and middleware, it makes good economic sense to be general. For someone coding a specific piece of game logic, it doesn't. That said, you can take "general" to stupid levels as well if you really want to, to the point where it doesn't actually do anything, but why would any reasonable person do that?

Cheers,
Conor


      The new Internet Explorer 8 optimised for Yahoo!7: Faster, Safer, Easier.

Re: [Algorithms] General purpose task parallel threading approach

[Tony C. <to...@mi...>]

I think you guys are getting tied up in an argument about semantics.

The real point here, IMHO, is that software development is a *tradeoff*. There are many things we'd like in any piece of software:

- Good performance (both in time and space)
- Robustness
- Maintainability
- Shipped on time
- Shipped on budget
- Features
- Interoperability
- Testability
- Portability
- Reusability (perhaps this means being more rather than less general)
- And many others.

Clearly, holding all other variables equals, it's better to have more of these good things than less. But you don't get to hold the other variables equal - you're always trading off things against each other. Would I like any piece of code I write to be more general purpose and more reusable? Sure, but (a) do I even know that how to achieve that goal, and (b) am I willing to trade off something else to get it? Maybe not. Perhaps almost always not.

- Tony

-----Original Message-----
From: chr...@pl... [mailto:chr...@pl...] 
Sent: Thursday, April 16, 2009 11:14 AM
To: Game Development Algorithms
Subject: Re: [Algorithms] General purpose task parallel threading approach

> "Other than classroom examples, software is NOT written through
> composition of 'general components'. And even in the small subset
> of software writing where your statement may hold true, writing
> those components was a very small amount of work of the total."
>
> No, every piece of software in a modern language is written through
> composition of "general components", from the machine code op-codes,
> up to the language constructs they come from, up to the libraries,
> patterns and algorithms they're constructed with.

The fact that you have to rely on labelling assembly instructions
and for-loops as "general components" to provide support for your
initial statement shows how weak your statement was.


Christer Ericson, Director of Tools and Technology
Sony Computer Entertainment, Santa Monica


------------------------------------------------------------------------------
Stay on top of everything new and different, both inside and 
around Java (TM) technology - register by April 22, and save
$200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
300 plus technical and hands-on sessions. Register today. 
Use priority code J9JMT32. http://p.sf.net/sfu/p
_______________________________________________
GDAlgorithms-list mailing list
GDA...@li...
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list

Re: [Algorithms] General purpose task parallel threading approach

[Tony C. <to...@mi...>]

Also, to expand on this point:

"Sure, but (a) do I even know that how to achieve that goal...?"

Intentionally writing "general purpose" code is an exercise in predicting the future. Sometimes, the future is easy to predict. If I'm writing a "List" class, it's a fairly safe prediction that people will want lists of all kinds of arbitrary types I don't know about. On the other hand, if I'm writing a large and complex component, my crystal ball is a lot foggier about how people might want to use it in future. Any additional work I do to allow for these future scenarios is based on my team's best efforts to guess about the future, an activity which is intrinsically fraught with peril.

That doesn't mean that it's always wrong to try to do this. Rather, one should have the humility to realize than one's guesses may be wrong, and that therefore one should discount the value of that work accordingly when trading it off against other considerations.

- Tony

-----Original Message-----
From: Tony Cox [mailto:to...@mi...] 
Sent: Thursday, April 16, 2009 12:03 PM
To: Game Development Algorithms
Subject: Re: [Algorithms] General purpose task parallel threading approach

I think you guys are getting tied up in an argument about semantics.

The real point here, IMHO, is that software development is a *tradeoff*. There are many things we'd like in any piece of software:

- Good performance (both in time and space)
- Robustness
- Maintainability
- Shipped on time
- Shipped on budget
- Features
- Interoperability
- Testability
- Portability
- Reusability (perhaps this means being more rather than less general)
- And many others.

Clearly, holding all other variables equals, it's better to have more of these good things than less. But you don't get to hold the other variables equal - you're always trading off things against each other. Would I like any piece of code I write to be more general purpose and more reusable? Sure, but (a) do I even know that how to achieve that goal, and (b) am I willing to trade off something else to get it? Maybe not. Perhaps almost always not.

- Tony

-----Original Message-----
From: chr...@pl... [mailto:chr...@pl...] 
Sent: Thursday, April 16, 2009 11:14 AM
To: Game Development Algorithms
Subject: Re: [Algorithms] General purpose task parallel threading approach

> "Other than classroom examples, software is NOT written through
> composition of 'general components'. And even in the small subset
> of software writing where your statement may hold true, writing
> those components was a very small amount of work of the total."
>
> No, every piece of software in a modern language is written through
> composition of "general components", from the machine code op-codes,
> up to the language constructs they come from, up to the libraries,
> patterns and algorithms they're constructed with.

The fact that you have to rely on labelling assembly instructions
and for-loops as "general components" to provide support for your
initial statement shows how weak your statement was.


Christer Ericson, Director of Tools and Technology
Sony Computer Entertainment, Santa Monica


------------------------------------------------------------------------------
Stay on top of everything new and different, both inside and 
around Java (TM) technology - register by April 22, and save
$200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
300 plus technical and hands-on sessions. Register today. 
Use priority code J9JMT32. http://p.sf.net/sfu/p
_______________________________________________
GDAlgorithms-list mailing list
GDA...@li...
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list


------------------------------------------------------------------------------
Stay on top of everything new and different, both inside and 
around Java (TM) technology - register by April 22, and save
$200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
300 plus technical and hands-on sessions. Register today. 
Use priority code J9JMT32. http://p.sf.net/sfu/p
_______________________________________________
GDAlgorithms-list mailing list
GDA...@li...
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list

Re: [Algorithms] General purpose task parallel threading approach

[Gregory J. <gj...@da...>]

YAGNI, in other words. ;)

Greg

-----Original Message-----
From: Tony Cox [mailto:to...@mi...] 
Sent: Thursday, April 16, 2009 12:22 PM
To: Game Development Algorithms
Subject: Re: [Algorithms] General purpose task parallel threading approach

Also, to expand on this point:

"Sure, but (a) do I even know that how to achieve that goal...?"

Intentionally writing "general purpose" code is an exercise in predicting
the future. Sometimes, the future is easy to predict. If I'm writing a
"List" class, it's a fairly safe prediction that people will want lists of
all kinds of arbitrary types I don't know about. On the other hand, if I'm
writing a large and complex component, my crystal ball is a lot foggier
about how people might want to use it in future. Any additional work I do to
allow for these future scenarios is based on my team's best efforts to guess
about the future, an activity which is intrinsically fraught with peril.

That doesn't mean that it's always wrong to try to do this. Rather, one
should have the humility to realize than one's guesses may be wrong, and
that therefore one should discount the value of that work accordingly when
trading it off against other considerations.

- Tony

-----Original Message-----
From: Tony Cox [mailto:to...@mi...] 
Sent: Thursday, April 16, 2009 12:03 PM
To: Game Development Algorithms
Subject: Re: [Algorithms] General purpose task parallel threading approach

I think you guys are getting tied up in an argument about semantics.

The real point here, IMHO, is that software development is a *tradeoff*.
There are many things we'd like in any piece of software:

- Good performance (both in time and space)
- Robustness
- Maintainability
- Shipped on time
- Shipped on budget
- Features
- Interoperability
- Testability
- Portability
- Reusability (perhaps this means being more rather than less general)
- And many others.

Clearly, holding all other variables equals, it's better to have more of
these good things than less. But you don't get to hold the other variables
equal - you're always trading off things against each other. Would I like
any piece of code I write to be more general purpose and more reusable?
Sure, but (a) do I even know that how to achieve that goal, and (b) am I
willing to trade off something else to get it? Maybe not. Perhaps almost
always not.

- Tony

-----Original Message-----
From: chr...@pl...
[mailto:chr...@pl...] 
Sent: Thursday, April 16, 2009 11:14 AM
To: Game Development Algorithms
Subject: Re: [Algorithms] General purpose task parallel threading approach

> "Other than classroom examples, software is NOT written through
> composition of 'general components'. And even in the small subset
> of software writing where your statement may hold true, writing
> those components was a very small amount of work of the total."
>
> No, every piece of software in a modern language is written through
> composition of "general components", from the machine code op-codes,
> up to the language constructs they come from, up to the libraries,
> patterns and algorithms they're constructed with.

The fact that you have to rely on labelling assembly instructions
and for-loops as "general components" to provide support for your
initial statement shows how weak your statement was.


Christer Ericson, Director of Tools and Technology
Sony Computer Entertainment, Santa Monica


----------------------------------------------------------------------------
--
Stay on top of everything new and different, both inside and 
around Java (TM) technology - register by April 22, and save
$200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
300 plus technical and hands-on sessions. Register today. 
Use priority code J9JMT32. http://p.sf.net/sfu/p
_______________________________________________
GDAlgorithms-list mailing list
GDA...@li...
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list


----------------------------------------------------------------------------
--
Stay on top of everything new and different, both inside and 
around Java (TM) technology - register by April 22, and save
$200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
300 plus technical and hands-on sessions. Register today. 
Use priority code J9JMT32. http://p.sf.net/sfu/p
_______________________________________________
GDAlgorithms-list mailing list
GDA...@li...
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list


----------------------------------------------------------------------------
--
Stay on top of everything new and different, both inside and 
around Java (TM) technology - register by April 22, and save
$200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
300 plus technical and hands-on sessions. Register today. 
Use priority code J9JMT32. http://p.sf.net/sfu/p
_______________________________________________
GDAlgorithms-list mailing list
GDA...@li...
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list

Re: [Algorithms] General purpose task parallel threading approach

[Gregory J. <gj...@da...>]

> If I'm writing a "List" class, it's a fairly safe prediction 
> that people will want lists of all kinds of 
> arbitrary types I don't know about. 

Additional thought on this: 

Not to mention, whether they want the list to be intrusive or
non-intrusive...

Greg

Re: [Algorithms] General purpose task parallel threading approach

[Gregory J. <gj...@da...>]

> I think you guys are getting tied up in an argument about semantics.

I've been waiting for someone to trot that one out...

The problem is, words have meanings. And if someone is going to offer a
system as "general" and applicable as a feasible solution to a wide range of
problems, then they actually *do* need to define what they mean by
"general". 

I've stated (and provided reasons why), as have many others, that there is
no such thing as a truly "general" system -- that the moment you put code to
an idea, you have instituted restrictions and rules on how your code is
intended to be used. Use outside those defined parameters is unsupported and
undefined. This is why we have documentation -- to describe the operational
parameters of whatever it is we have implemented.

Alexander has created a system that worked for his needs. In order to do so,
he (like anyone else creating a system) had to make assumptions about the
conditions under which that system was designed to operate. Now he would
like to make his system accessible to a wider audience. The problem with
Alexander's approach in this thread, however, is that he is asking what
specific features people would find useful in asynclib. This approach is a
problem because it likely will introduce additional complexity (depending on
the feature), and more specific restrictions on when and how a particular
feature is to be used, and that actually leads to a less general solution. 

So perhaps a better question is not about a "general purpose" system, but
instead one about a system that supports the particular needs of game
development? The list of criteria you provided certainly is more in line
with the interests of game development (not that I would agree that
"reusability" has anything to do with "generality"), and that list is
arguably the baseline from which most on this side of the argument have been
using. 
 
Greg

Re: [Algorithms] General purpose task parallel threading approach

[Nicholas \Indy\ R. <ar...@gm...>]

On Thu, Apr 16, 2009 at 11:14 AM,
<chr...@pl...> wrote:
> The fact that you have to rely on labelling assembly instructions
> and for-loops as "general components" to provide support for your
> initial statement shows how weak your statement was.

Additionally with over 200 assembly instructions in x86, with more and
more special case assembly instructions being added, I wouldn't really
call composition of assembly instructions to also be composition of
general purpose software components. An additional point is that the
main things that code optimizers do, is remove generality from
assembly compositions.

Indy

[Algorithms] Complexity of new hardware

[Jon W. <jw...@gm...>]

Nicholas "Indy" Ray wrote:
> On Thu, Apr 16, 2009 at 11:14 AM,
> <chr...@pl...> wrote:
>   
>> The fact that you have to rely on labelling assembly instructions
>> and for-loops as "general components" to provide support for your
>> initial statement shows how weak your statement was.
>>     
>
> Additionally with over 200 assembly instructions in x86, with more and
> more special case assembly instructions being added, I wouldn't really
>   

My favorite new x86 instruction:
|
vmadd231ps v0 {k1}, v5, [rbx+rcx*4] {4to16}

Dealing with complexity is really becoming quite a chore, and our tools 
just aren't keeping up IMO.
What good ways are you seeing of lifting game design and implementation 
up onto higher levels of abstraction? Is it just middleware, or will the 
middleware guys in turn need newer, better ways of doing things?

Sincerely,

jw

|

Re: [Algorithms] Complexity of new hardware

[Nicholas \Indy\ R. <ar...@gm...>]

On 4/16/09, Jon Watte <jw...@gm...> wrote:
>  Dealing with complexity is really becoming quite a chore, and our tools
>  just aren't keeping up IMO.
>  What good ways are you seeing of lifting game design and implementation
>  up onto higher levels of abstraction? Is it just middleware, or will the
>  middleware guys in turn need newer, better ways of doing things?

I find that middleware while often built to be as general as possible,
it's still too specialized to be the end all of game design
abstraction. all middleware is designed for a specific purpose and a
subset of game types.

Since we have been topping out at progress we can make in the graphics
sense, that leaves us with much more progress to be made with core
design decisions, as evidenced by the wealth of new game mechanics
being constantly introduced by the indie game community.

Yet at the AAA end of the scale the scope of our games are so large,
and the amount of work involved is so much that we aren't yet able to
experiment so freely. As often such changes don't fit very well into
our existing pipelines/engines/middleware.

Additionally we are constantly developing tools for our artists and
designers and giving them more freedom. But here we are still
developing with the same old tools we have been using for 10+ years,
that were created with no thought to the specific challenges of game
development.

Thusly I feel that some large improvements need to be done in the
tools that help us develop our games, otherwise programming will begin
to be a larger and larger bottleneck, and we will fall into the
pattern of making the same old sort of games.

We need improvement in the languages we use, compilers, and
IDE/Editors; It's absurd that I am using tools/languages that have
been only incrementally improved in the last 10 or so years, and have
evolved naught with considerations from the problems we face in game
development.

Nicholas "Indy" Ray

Re: [Algorithms] Complexity of new hardware

[Jeff R. <je...@8m...>]

Working in C++ all the time I feel like the language has become a problem.
The whole industry seems *really* stuck on it, probably because it has the
"when in doubt, add a feature" mindset that so many programmers have (and I
count myself). I'd love to see something a bit more powerful in a language
that still affords very good performance.

Jeff

On Thu, Apr 16, 2009 at 4:08 PM, Nicholas "Indy" Ray <ar...@gm...>wrote:

> On 4/16/09, Jon Watte <jw...@gm...> wrote:
> >  Dealing with complexity is really becoming quite a chore, and our tools
> >  just aren't keeping up IMO.
> >  What good ways are you seeing of lifting game design and implementation
> >  up onto higher levels of abstraction? Is it just middleware, or will the
> >  middleware guys in turn need newer, better ways of doing things?
>
> I find that middleware while often built to be as general as possible,
> it's still too specialized to be the end all of game design
> abstraction. all middleware is designed for a specific purpose and a
> subset of game types.
>
> Since we have been topping out at progress we can make in the graphics
> sense, that leaves us with much more progress to be made with core
> design decisions, as evidenced by the wealth of new game mechanics
> being constantly introduced by the indie game community.
>
> Yet at the AAA end of the scale the scope of our games are so large,
> and the amount of work involved is so much that we aren't yet able to
> experiment so freely. As often such changes don't fit very well into
> our existing pipelines/engines/middleware.
>
> Additionally we are constantly developing tools for our artists and
> designers and giving them more freedom. But here we are still
> developing with the same old tools we have been using for 10+ years,
> that were created with no thought to the specific challenges of game
> development.
>
> Thusly I feel that some large improvements need to be done in the
> tools that help us develop our games, otherwise programming will begin
> to be a larger and larger bottleneck, and we will fall into the
> pattern of making the same old sort of games.
>
> We need improvement in the languages we use, compilers, and
> IDE/Editors; It's absurd that I am using tools/languages that have
> been only incrementally improved in the last 10 or so years, and have
> evolved naught with considerations from the problems we face in game
> development.
>
> Nicholas "Indy" Ray
>
>
> ------------------------------------------------------------------------------
> Stay on top of everything new and different, both inside and
> around Java (TM) technology - register by April 22, and save
> $200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
> 300 plus technical and hands-on sessions. Register today.
> Use priority code J9JMT32. http://p.sf.net/sfu/p
> _______________________________________________
> GDAlgorithms-list mailing list
> GDA...@li...
> https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
> Archives:
> http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
>

Re: [Algorithms] Complexity of new hardware

[Gregory J. <gj...@da...>]

> We need improvement in the languages we use, compilers, and
> IDE/Editors; It's absurd that I am using tools/languages that have
> been only incrementally improved in the last 10 or so years, and have
> evolved naught with considerations from the problems we face in game
> development.

NaughtyDog tried -- you have to admit that Lisp *in theory* should be
perfectly suited to game development: it's core design intent is list
processing (indeed, it's the name), which arguably games *are*, and it's
functional, which ought to be perfect for increasingly data-parallel game
technology designs. If the syntax is just "too weird" -- that can be
changed, while leaving the core language design intact (again, GOAL). 

http://www.gamasutra.com/features/20020710/white_02.htm

Of course, for the past ten years we've been developing for (until recently)
single-threaded Von Neumann architectures, so I'm not surprised the tools
haven't changed -- they still target the same basic hardware. Until we break
away from that processing model you're not likely to see many changes in the
tools we use. Larrabee (driven largely by GPU design) is a step in that
direction, but it's still (by business necessity) rooted in the x86 legacy
that has been around for over 30 years now.

They define the difficulties in moving to something different in that
Gamasutra article: you can't hire anyone who knows (new language X), you
can't find middleware or tools written in it (because we've used the same
tools and hardware for the past 30 years). Even using common higher-level
languages such as Python, C#, Ruby, etc., those are still based on the same
processing paradigm we use in C++...so why change when the tools, arguably
(unless you are developing for the Wii), are actually fairly usable, and the
middleware and libraries are so readily available?

Short answer: we use C++ and its tools because it is still the best
compromise, using multiple criteria, among all of the possible solutions.
Granted, C++0x may change that, but luckily it's still a draft...

I would ask -- to which game-development-specific problems do you refer?

Greg

Re: [Algorithms] Complexity of new hardware

[Nicholas \Indy\ R. <ar...@gm...>]

On 4/16/09, Jeff Russell <je...@8m...> wrote:
> Working in C++ all the time I feel like the language has become a problem.
> The whole industry seems *really* stuck on it, probably because it has the
> "when in doubt, add a feature" mindset that so many programmers have (and I
> count myself). I'd love to see something a bit more powerful in a language
> that still affords very good performance.

I think the reason the industry is stuck on it is because c++ is the
last real systems programming language. Game Development is a soft
real-time app and performance critical, General GC doesn't seem to be
good enough for Game Development. Our Special case just doesn't come
up in language development otherwise.

On 4/16/09, Gregory Junker <gj...@da...> wrote:
> NaughtyDog tried -- you have to admit that Lisp *in theory* should be
>  perfectly suited to game development: it's core design intent is list
>  processing (indeed, it's the name), which arguably games *are*, and it's
>  functional, which ought to be perfect for increasingly data-parallel game
>  technology designs. If the syntax is just "too weird" -- that can be
>  changed, while leaving the core language design intact (again, GOAL).

For me it's not a matter of admitting, outside of game development,
scheme is my language of choice. It's a pity that Andy had left
Naughty Dog, I think something like GOAL is just the sort of thing
that is needed. Regarding syntax, considering how trivial it is to
parse and rewrite S-expressions it should also be trivial to create
IDE's with all sorts of "views" into the code, as opposed to how the
complexity of parsing C++ requires the "source" to be the only view
into the code. (To be fair, IDEs like Visual studio do provide
additional views, but none of them are editable)

>  They define the difficulties in moving to something different in that
>  Gamasutra article: you can't hire anyone who knows (new language X),

I think this is a matter of hiring skilled professionals who can learn
new languages, they already have to learn so much else at new studios,
I don't figure it to be a large issue.

>  can't find middleware or tools written in it (because we've used the same
>  tools and hardware for the past 30 years).

At least early on, the key is to have *really good* compatibility with
C++ libraries.

>  I would ask -- to which game-development-specific problems do you refer?

My point is that they don't exist, other sects of development are
constantly creating new tools (web development for example) yet no one
creates them with game development in mind. Perhaps that's because of
the tradition we seem to have, or otherwise in not paying the kind of
wages required to bring in a lot of people who are capable of creating
those tools, I'm not sure.

Nicholas "Indy" Ray

Re: [Algorithms] Complexity of new hardware

[Mat N. <mat...@bu...>]

> My point is that they don't exist, other sects of development are constantly creating new tools (web development for example) yet no one creates them with game development in mind. Perhaps that's because of the tradition we seem to have, or otherwise in not paying the kind of wages required to bring in a lot of people who are capable of creating those tools, I'm not sure.

LLVM (www.llvm.org) is one approach that is more useful than most. Or Microsoft's Phoenix, if that ever becomes publically available.

Most programming development endeavors try to capture an entire end of the programming spectrum: either from users of a programming language down or a specific virtual platform on up. Unfortunately for game development, the more we can control the entire spectrum the better.

MSN
-----Original Message-----
From: Nicholas "Indy" Ray [mailto:ar...@gm...]
Sent: Thursday, April 16, 2009 3:14 PM
To: Game Development Algorithms
Subject: Re: [Algorithms] Complexity of new hardware

On 4/16/09, Jeff Russell <je...@8m...> wrote:
> Working in C++ all the time I feel like the language has become a problem.
> The whole industry seems *really* stuck on it, probably because it has the
> "when in doubt, add a feature" mindset that so many programmers have (and I
> count myself). I'd love to see something a bit more powerful in a language
> that still affords very good performance.

I think the reason the industry is stuck on it is because c++ is the
last real systems programming language. Game Development is a soft
real-time app and performance critical, General GC doesn't seem to be
good enough for Game Development. Our Special case just doesn't come
up in language development otherwise.

On 4/16/09, Gregory Junker <gj...@da...> wrote:
> NaughtyDog tried -- you have to admit that Lisp *in theory* should be
>  perfectly suited to game development: it's core design intent is list
>  processing (indeed, it's the name), which arguably games *are*, and it's
>  functional, which ought to be perfect for increasingly data-parallel game
>  technology designs. If the syntax is just "too weird" -- that can be
>  changed, while leaving the core language design intact (again, GOAL).

For me it's not a matter of admitting, outside of game development,
scheme is my language of choice. It's a pity that Andy had left
Naughty Dog, I think something like GOAL is just the sort of thing
that is needed. Regarding syntax, considering how trivial it is to
parse and rewrite S-expressions it should also be trivial to create
IDE's with all sorts of "views" into the code, as opposed to how the
complexity of parsing C++ requires the "source" to be the only view
into the code. (To be fair, IDEs like Visual studio do provide
additional views, but none of them are editable)

>  They define the difficulties in moving to something different in that
>  Gamasutra article: you can't hire anyone who knows (new language X),

I think this is a matter of hiring skilled professionals who can learn
new languages, they already have to learn so much else at new studios,
I don't figure it to be a large issue.

>  can't find middleware or tools written in it (because we've used the same
>  tools and hardware for the past 30 years).

At least early on, the key is to have *really good* compatibility with
C++ libraries.

>  I would ask -- to which game-development-specific problems do you refer?

My point is that they don't exist, other sects of development are
constantly creating new tools (web development for example) yet no one
creates them with game development in mind. Perhaps that's because of
the tradition we seem to have, or otherwise in not paying the kind of
wages required to bring in a lot of people who are capable of creating
those tools, I'm not sure.

Nicholas "Indy" Ray

------------------------------------------------------------------------------
Stay on top of everything new and different, both inside and
around Java (TM) technology - register by April 22, and save
$200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
300 plus technical and hands-on sessions. Register today.
Use priority code J9JMT32. http://p.sf.net/sfu/p
_______________________________________________
GDAlgorithms-list mailing list
GDA...@li...
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list

Re: [Algorithms] Complexity of new hardware

[Darren G. <dg...@ke...>]

At 02:44 PM 4/16/2009, Gregory Junker wrote:

> > We need improvement in the languages we use, compilers, and
> > IDE/Editors; It's absurd that I am using tools/languages that have
> > been only incrementally improved in the last 10 or so years, and have
> > evolved naught with considerations from the problems we face in game
> > development.
>
>NaughtyDog tried -- you have to admit that Lisp *in theory* should be
>perfectly suited to game development: it's core design intent is list
>processing (indeed, it's the name), which arguably games *are*, and it's
>functional, which ought to be perfect for increasingly data-parallel game
>technology designs. If the syntax is just "too weird" -- that can be
>changed, while leaving the core language design intact (again, GOAL).
>
>http://www.gamasutra.com/features/20020710/white_02.htm
>
>Of course, for the past ten years we've been developing for (until recently)
>single-threaded Von Neumann architectures, so I'm not surprised the tools
>haven't changed -- they still target the same basic hardware. Until we break
>away from that processing model you're not likely to see many changes in the
>tools we use. Larrabee (driven largely by GPU design) is a step in that
>direction, but it's still (by business necessity) rooted in the x86 legacy
>that has been around for over 30 years now.

Remember-- two market categories, which might as well be called 
custom and public.  Custom is simple, if you have the means go 
nuts.  But public changes need to be considered for marketing, 
migration, ethical and user transparency impact as well.  Because of 
the huge established base for x86 PC's, I would never bemoan Intel 
favouring a bridge technology for public consumption.  That has the 
appearance of a productive decision.  Going off on a slight tangent, 
an example of an "odd" decision would be Intel shelving support for 
numerous integrated graphics chipsets still out in the wild.  That is 
surely costly for developers and customers.


>They define the difficulties in moving to something different in that
>Gamasutra article: you can't hire anyone who knows (new language X), you
>can't find middleware or tools written in it (because we've used the same
>tools and hardware for the past 30 years). Even using common higher-level
>languages such as Python, C#, Ruby, etc., those are still based on the same
>processing paradigm we use in C++...so why change when the tools, arguably
>(unless you are developing for the Wii), are actually fairly usable, and the
>middleware and libraries are so readily available?
>
>Short answer: we use C++ and its tools because it is still the best
>compromise, using multiple criteria, among all of the possible solutions.
>Granted, C++0x may change that, but luckily it's still a draft...

I don't think that there is a high-level language that is unaware of 
its linearity since this is usually considered a strength of 
simplicity for a design dialect.  In fact the current trend is that a 
language with deep first-class reflection and standard library 
integration is almost guaranteed to become a platform for researching 
new application tools and a few paradigm candidates, some of which 
are for concurrent design.  MS CLR is an obvious example of 
this.  You can quantify the boom in their own research productivity 
by the number of unique results developed with c# especially since 
.net 2.  Pure C++ development by comparison has always been fractured 
and dragging its heels as developers keep muddling around the same 
design problems over and over again.  Python has seen great research 
adoption as well, although the numbers are somewhat steadier and 
fuzzier because of the large diffuse audience.

Good point about the dangers of working in an ivory tower.  It is 
beneficial to approach research respectfully with measures of 
reservation and imagination, so that if or when a good application is 
recognized it is ready to go.  You really can't expect all research 
material to be answerable to the pragmatic concerns of a wide 
audience all the time.  On the other hand, the languages you mention 
are very pragmatic, addressing very broad concerns.


Cheers,
Darren

Re: [Algorithms] Complexity of new hardware

[Jon W. <jw...@gm...>]

Gregory Junker wrote:
> Short answer: we use C++ and its tools because it is still the best
> compromise, using multiple criteria, among all of the possible solutions.
> Granted, C++0x may change that, but luckily it's still a draft...
>
> I would ask -- to which game-development-specific problems do you refer?
>   

Games often need high-performance data metadata, more than business or 
device driver development. A high-performance compile-time and run-time 
introspection feature in C++ would be great; especially if it could be 
integrated with templates! The compiler already has all the information; 
the standards people just don't want to add the necessary accessors to 
class type_info :-(

Sincerely,

jw

Re: [Algorithms] Complexity of new hardware

[Nicholas \Indy\ R. <ar...@gm...>]

On Thu, Apr 16, 2009 at 10:21 PM, Jon Watte <jw...@gm...> wrote:
> A high-performance compile-time and run-time
> introspection feature in C++ would be great; especially if it could be
> integrated with templates! The compiler already has all the information;
> the standards people just don't want to add the necessary accessors to
> class type_info :-(

Sure, the Data exists, but it is in *very* different formats between
compilers (compare even open source C language front ends you'll see
huge variances in data formats). And if you're familiar with how
complicated the C++ spec is, you'd know that there is *way* too much
information to ever get compiler introspection of in a standard sort
of way. If you don't need something portable or standard, look into a
gcc extension such as dehydra for gcc. Then again, if you don't need
something standard it'd be trivial to generate c++ declarations
through a custom preprocessor.

Nicholas "Indy" Ray

Re: [Algorithms] Complexity of new hardware

[Jon W. <jw...@gm...>]

Nicholas "Indy" Ray wrote:
> Sure, the Data exists, but it is in *very* different formats between
> compilers (compare even open source C language front ends you'll see
> huge variances in data formats). And if you're familiar with how
>   

We already have type_info. Having an array of 
offset-size-name-type_info* for each member of a compound data type (or 
any data type) would not be rocket science, would not be hard to 
implement in a standard way, and would be very useful. It could be a 
very simple extension to the existing type_info class, returning 
entirely static data.
Doing the same for templates (iterating through member names and types) 
would be harder to implement cleanly, but would be even more useful.
The necessity to express yourself two or three times in various macros, 
declarations, definitions, code generators etc is one big wart in C++. 
Don't Repeat Yourself!
... and I probably should take further discussion to something like 
comp.lang.cplusplus ;-)

Sincerely,

jw

Re: [Algorithms] Complexity of new hardware

[Jarkko L. <al...@gm...>]

I don't think you would even want to parse the compiler generated data for
introspection because it wouldn't provide enough necessary metadata
information for the purpose. From my experience you need e.g. different
level of constness altogether and other custom variable properties for
introspection purposes which the language doesn't provide. I have
implemented class introspection in C++ with some macro & template magic
which works reasonably well and is quite easy to use, but of course it would
be nicer to be able to define all the introspection attributes along with
the class variable if there was a language support for it.


Cheers, Jarkko


-----Original Message-----
From: Nicholas "Indy" Ray [mailto:ar...@gm...] 
Sent: Friday, April 17, 2009 9:05 AM
To: Game Development Algorithms
Subject: Re: [Algorithms] Complexity of new hardware

On Thu, Apr 16, 2009 at 10:21 PM, Jon Watte <jw...@gm...> wrote:
> A high-performance compile-time and run-time
> introspection feature in C++ would be great; especially if it could be
> integrated with templates! The compiler already has all the information;
> the standards people just don't want to add the necessary accessors to
> class type_info :-(

Sure, the Data exists, but it is in *very* different formats between
compilers (compare even open source C language front ends you'll see
huge variances in data formats). And if you're familiar with how
complicated the C++ spec is, you'd know that there is *way* too much
information to ever get compiler introspection of in a standard sort
of way. If you don't need something portable or standard, look into a
gcc extension such as dehydra for gcc. Then again, if you don't need
something standard it'd be trivial to generate c++ declarations
through a custom preprocessor.

Nicholas "Indy" Ray

----------------------------------------------------------------------------
--
Stay on top of everything new and different, both inside and 
around Java (TM) technology - register by April 22, and save
$200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
300 plus technical and hands-on sessions. Register today. 
Use priority code J9JMT32. http://p.sf.net/sfu/p
_______________________________________________
GDAlgorithms-list mailing list
GDA...@li...
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list

Re: [Algorithms] Complexity of new hardware

[Alen L. <ale...@cr...>]

Friday, April 17, 2009, 9:48:32 AM, Jarkko wrote:
> I don't think you would even want to parse the compiler generated data for
> introspection because it wouldn't provide enough necessary metadata
> information for the purpose.

IME, parsing the headers can get you a long way and alows you to easly
add additional info to members in-place. Handling all the cornercases
of C++ would be hard, but we don't want to use those anyway, so that
is not a problem.

> From my experience you need e.g. different level of constness
> altogether and other custom variable properties for introspection
> purposes which the language doesn't provide.

Haven't needed different constness yet, what do you mean by that?

We use flags for object ownership on pointers (in contrast to just
linking), barriers to define deep copy limits, versioning info for
serialization, hints for GUI for loading/streaming and a bunch of
others.

Cheers,
Alen

Re: [Algorithms] Complexity of new hardware

[Mat N. <mat...@bu...>]

You could always flip that around and have a to-C++ compiler.

MSN
-----Original Message-----
From: Alen Ladavac [mailto:ale...@cr...]
Sent: Friday, April 17, 2009 12:26 PM
To: Jarkko Lempiainen
Cc: 'Game Development Algorithms'
Subject: Re: [Algorithms] Complexity of new hardware

Friday, April 17, 2009, 9:48:32 AM, Jarkko wrote:
> I don't think you would even want to parse the compiler generated data for
> introspection because it wouldn't provide enough necessary metadata
> information for the purpose.

IME, parsing the headers can get you a long way and alows you to easly
add additional info to members in-place. Handling all the cornercases
of C++ would be hard, but we don't want to use those anyway, so that
is not a problem.

> From my experience you need e.g. different level of constness
> altogether and other custom variable properties for introspection
> purposes which the language doesn't provide.

Haven't needed different constness yet, what do you mean by that?

We use flags for object ownership on pointers (in contrast to just
linking), barriers to define deep copy limits, versioning info for
serialization, hints for GUI for loading/streaming and a bunch of
others.

Cheers,
Alen


------------------------------------------------------------------------------
Stay on top of everything new and different, both inside and
around Java (TM) technology - register by April 22, and save
$200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
300 plus technical and hands-on sessions. Register today.
Use priority code J9JMT32. http://p.sf.net/sfu/p
_______________________________________________
GDAlgorithms-list mailing list
GDA...@li...
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list