Re: [Algorithms] sparse bitset compression

[Oscar F. <os...@tr...>]

Pretty much.

The space filling curve might not be necessary but you may be able to use
it to keep smaller gaps between the entries which are local to each other
spatially.


On 14 January 2014 16:52, Richard Fabian <ra...@gm...> wrote:

> That sounds good. Something like the JPEG VLC for the actual distance
> between entries and just store deltas on a space filling curve should be
> good then.
>
>
> On 14 January 2014 13:11, Oscar Forth <os...@tr...>wrote:
>
>> Just a random though, so apologies if i'm talking about my arse, but ...
>> couldn't you just use something like the bit plane compression from Wavelet
>> Difference Reduction (WDR)?
>>
>> A lot depends on your data, I appreciate, so this may not work out as any
>> significant saving ...
>>
>> I wrote about the compression method in my blog a few years back:
>>
>> http://trueharmoniccolours.co.uk/Blog/?p=55
>>
>> Obviously thats in the context of a full wavelet image compressor so you
>> will have to just grab the part I explain in the link above.
>>
>> Also apologies for the code formatting on my blog ... ill fix it one day
>> ;)
>>
>> Oscar
>>
>> On 14 January 2014 12:37, Richard Fabian <ra...@gm...> wrote:
>>
>>> The web is full of different solutions for compressing things, but some
>>> of you have probably already done this and found a really good ratio
>>> compression scheme that fits what I'm asking about.
>>>
>>> I've got a load of entites, (4-7k) that live on a grid (1024x1024), and
>>> I need to persist them to a savegame which gets sent across the network to
>>> keep the save safe. I need to compress it. I'm compressing the entities in
>>> the world already, ones that have been affected cost a few bytes, but the
>>> but for the ones that don't have any changes, all I'm left with is the
>>> coords. There are only a few that need the fatter compression, but at the
>>> numbers I'm looking at, those "unchanged entity" coords add up to a lot of
>>> data when storing as 20 bit structs. This is too much for sending over the
>>> network regularly as part of a save.
>>>
>>> I've had some ideas on how to compress the data, but they might be crap.
>>> I don't know.
>>>
>>> I can't easily regenerate the set so thought someone who wanted to
>>> compress consumables might know the name of a good solution for sparse
>>> bitset compression. The bits are clumped around different areas of the
>>> grid, so I feel that something that leverages spatial coherence might do
>>> well.
>>>
>>> Any leads would be highly appreciated.
>>>
>>> --
>>> fabs();
>>> "The fact that an opinion has been widely held is no evidence whatever
>>> that it is not utterly absurd." - Bertrand Russell
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> CenturyLink Cloud: The Leader in Enterprise Cloud Services.
>>> Learn Why More Businesses Are Choosing CenturyLink Cloud For
>>> Critical Workloads, Development Environments & Everything In Between.
>>> Get a Quote or Start a Free Trial Today.
>>>
>>> http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
>>> _______________________________________________
>>> GDAlgorithms-list mailing list
>>> GDA...@li...
>>> https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
>>> Archives:
>>> http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
>>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> CenturyLink Cloud: The Leader in Enterprise Cloud Services.
>> Learn Why More Businesses Are Choosing CenturyLink Cloud For
>> Critical Workloads, Development Environments & Everything In Between.
>> Get a Quote or Start a Free Trial Today.
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
>> _______________________________________________
>> GDAlgorithms-list mailing list
>> GDA...@li...
>> https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
>> Archives:
>> http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
>>
>
>
>
> --
> fabs();
> "The fact that an opinion has been widely held is no evidence whatever
> that it is not utterly absurd." - Bertrand Russell
>
>
> ------------------------------------------------------------------------------
> CenturyLink Cloud: The Leader in Enterprise Cloud Services.
> Learn Why More Businesses Are Choosing CenturyLink Cloud For
> Critical Workloads, Development Environments & Everything In Between.
> Get a Quote or Start a Free Trial Today.
>
> http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
> _______________________________________________
> GDAlgorithms-list mailing list
> GDA...@li...
> https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
> Archives:
> http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
>