#296 Prune operation to remove empty hyperplanes

[Steve Keen]

We may debate this one, but I think that if you divide one series (with say 100 dimension values) by another (with say 10 values that contain data) then the outputted series should have just ten dimension values rather than 100).

I saw that this wasn't the case at present by dividing a House Price Index series with 62 entries (where 2010=100) by the index values for 1970, where only a handful of the countries had data (say 5). I got a variable with 62 entries on the country dimension, the vast majority of which were blank.

I've attached the source data file here, from the BIS. I'll pop the movie in via Google again.

[Steve Keen]

DataReductionNullDataShouldReduceDimension20240306.mp4
https://drive.google.com/file/d/1i8ab_OgpttJ8khVTeI2Qc4X1r9Yib5NI/view?usp=drive_web

Best, Steve
Professor Steve Keen
Want to rebuild economics? Support me on
Patreon: https://www.patreon.com/ProfSteveKeen
https://www.patreon.com/ProfSteveKeen
My latest book, The New Economics, is now available from Polity:
http://politybooks.com/bookdetail/?isbn=9781509545285
@ProfSteveKeen
Mobile +66 (0) 99-257-2692

Honorary Professor, UCL &ISRS Distinguished Research Fellow
www.profstevekeen.com

On Wed, Mar 6, 2024 at 4:30 PM Steve Keen profstevekeen@users.sourceforge.net wrote:

---

[ravel:#490] https://sourceforge.net/p/minsky/ravel/490/ Operations
should reduce dimension count for null values

Status: open
Milestone: Pascal
Created: Wed Mar 06, 2024 03:30 PM UTC by Steve Keen
Last Updated: Wed Mar 06, 2024 03:30 PM UTC
Owner: nobody
Attachments:

• WS_SPP_csv_col.csv
  https://sourceforge.net/p/minsky/ravel/490/attachment/WS_SPP_csv_col.csv
  (336.4 kB; text/csv)

We may debate this one, but I think that if you divide one series (with
say 100 dimension values) by another (with say 10 values that contain data)
then the outputted series should have just ten dimension values rather than
100).

I saw that this wasn't the case at present by dividing a House Price Index
series with 62 entries (where 2010=100) by the index values for 1970, where
only a handful of the countries had data (say 5). I got a variable with 62
entries on the country dimension, the vast majority of which were blank.

I've attached the source data file here, from the BIS. I'll pop the movie
in via Google again.

---

Sent from sourceforge.net because you indicated interest in
https://sourceforge.net/p/minsky/ravel/490/

To unsubscribe from further messages, please visit
https://sourceforge.net/auth/subscriptions/

[Steve Keen]

Ravel file.

[High Performance Coder]

You're right - we may debate this one :). I would not consider the current behaviour a bug, however, I can see the use of another operation that removes non-existent data (non-existent hyperplanes of a hypercube, actually). Its actually a rather data intensive operation to do so, so it should only be introduced deliberately by the user.

So lets discuss this - I'm thinking it may be a feature request for a future version.

[Steve Keen]

Yes, that makes sense. Sometimes a user will want to do an initial
operation that generates an array with null values for later analysis.

How about a "Reduce" operator for Ravels?

[High Performance Coder]

Reduce, or reduction is already used for the operations of sum,
product, max, min etc. So we'd need another name for it.

As they say - there are only two really hard problems in computer science:

0) Naming things
1) Cache invalidation
2) Off by one errors.

Cheers

On Mon, Mar 11, 2024 at 06:59:13AM -0000, Steve Keen wrote:

Yes, that makes sense. Sometimes a user will want to do an initial
operation that generates an array with null values for later analysis.

How about a "Reduce" operator for Ravels?

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

[ravel:#490] Operations should reduce dimension count for null values

Status: awaiting-user
Milestone: Pascal
Created: Wed Mar 06, 2024 03:30 PM UTC by Steve Keen
Last Updated: Mon Mar 11, 2024 06:17 AM UTC
Owner: nobody
Attachments:

• WS_SPP_csv_col.csv (336.4 kB; text/csv)

We may debate this one, but I think that if you divide one series (with say 100
dimension values) by another (with say 10 values that contain data) then the
outputted series should have just ten dimension values rather than 100).

I saw that this wasn't the case at present by dividing a House Price Index
series with 62 entries (where 2010=100) by the index values for 1970, where
only a handful of the countries had data (say 5). I got a variable with 62
entries on the country dimension, the vast majority of which were blank.

I've attached the source data file here, from the BIS. I'll pop the movie in
via Google again.

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Sent from sourceforge.net because you indicated interest in https://
sourceforge.net/p/minsky/ravel/490/

To unsubscribe from further messages, please visit https://sourceforge.net/auth
/subscriptions/

--

---

Dr Russell Standish Phone 0425 253119 (mobile)
Principal, High Performance Coders hpcoder@hpcoders.com.au
http://www.hpcoders.com.au

---

[Steve Keen]

How about "Trim" or "Prune" then? I think Prune is better.

[High Performance Coder]

Prune sound good. But let's backlog this for a later release.

[Steve Keen]

Hmmm. I'd prefer this to turn up earlier than that. Maybe raise it's level
from 6 to say 4.

[High Performance Coder]

Ticket moved from /p/minsky/ravel/490/