Re: [Algorithms] sparse bitset compression
Brought to you by:
vexxed72
From: Chris B. <cb...@gm...> - 2014-01-14 19:11:47
|
You might also look into morton order encoding, as it is a simple and efficient locality preserving way to do lossless n dimensional tree structures - this is of course best for applications like map data. Your mileage may vary. -Chria ἀπὸ μηχανῆς iPhone 5 > On Jan 14, 2014, at 9:01 AM, Oscar Forth <os...@tr...> wrote: > > Pretty much. > > The space filling curve might not be necessary but you may be able to use it to keep smaller gaps between the entries which are local to each other spatially. > > >> On 14 January 2014 16:52, Richard Fabian <ra...@gm...> wrote: >> That sounds good. Something like the JPEG VLC for the actual distance between entries and just store deltas on a space filling curve should be good then. >> >> >>> On 14 January 2014 13:11, Oscar Forth <os...@tr...> wrote: >>> Just a random though, so apologies if i'm talking about my arse, but ... couldn't you just use something like the bit plane compression from Wavelet Difference Reduction (WDR)? >>> >>> A lot depends on your data, I appreciate, so this may not work out as any significant saving ... >>> >>> I wrote about the compression method in my blog a few years back: >>> >>> http://trueharmoniccolours.co.uk/Blog/?p=55 >>> >>> Obviously thats in the context of a full wavelet image compressor so you will have to just grab the part I explain in the link above. >>> >>> Also apologies for the code formatting on my blog ... ill fix it one day ;) >>> >>> Oscar >>> >>>> On 14 January 2014 12:37, Richard Fabian <ra...@gm...> wrote: >>>> The web is full of different solutions for compressing things, but some of you have probably already done this and found a really good ratio compression scheme that fits what I'm asking about. >>>> >>>> I've got a load of entites, (4-7k) that live on a grid (1024x1024), and I need to persist them to a savegame which gets sent across the network to keep the save safe. I need to compress it. I'm compressing the entities in the world already, ones that have been affected cost a few bytes, but the but for the ones that don't have any changes, all I'm left with is the coords. There are only a few that need the fatter compression, but at the numbers I'm looking at, those "unchanged entity" coords add up to a lot of data when storing as 20 bit structs. This is too much for sending over the network regularly as part of a save. >>>> >>>> I've had some ideas on how to compress the data, but they might be crap. I don't know. >>>> >>>> I can't easily regenerate the set so thought someone who wanted to compress consumables might know the name of a good solution for sparse bitset compression. The bits are clumped around different areas of the grid, so I feel that something that leverages spatial coherence might do well. >>>> >>>> Any leads would be highly appreciated. >>>> >>>> -- >>>> fabs(); >>>> "The fact that an opinion has been widely held is no evidence whatever that it is not utterly absurd." - Bertrand Russell >>>> >>>> ------------------------------------------------------------------------------ >>>> CenturyLink Cloud: The Leader in Enterprise Cloud Services. >>>> Learn Why More Businesses Are Choosing CenturyLink Cloud For >>>> Critical Workloads, Development Environments & Everything In Between. >>>> Get a Quote or Start a Free Trial Today. >>>> http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk >>>> _______________________________________________ >>>> GDAlgorithms-list mailing list >>>> GDA...@li... >>>> https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list >>>> Archives: >>>> http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list >>> >>> >>> ------------------------------------------------------------------------------ >>> CenturyLink Cloud: The Leader in Enterprise Cloud Services. >>> Learn Why More Businesses Are Choosing CenturyLink Cloud For >>> Critical Workloads, Development Environments & Everything In Between. >>> Get a Quote or Start a Free Trial Today. >>> http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk >>> _______________________________________________ >>> GDAlgorithms-list mailing list >>> GDA...@li... >>> https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list >>> Archives: >>> http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list >> >> >> >> -- >> fabs(); >> "The fact that an opinion has been widely held is no evidence whatever that it is not utterly absurd." - Bertrand Russell >> >> ------------------------------------------------------------------------------ >> CenturyLink Cloud: The Leader in Enterprise Cloud Services. >> Learn Why More Businesses Are Choosing CenturyLink Cloud For >> Critical Workloads, Development Environments & Everything In Between. >> Get a Quote or Start a Free Trial Today. >> http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk >> _______________________________________________ >> GDAlgorithms-list mailing list >> GDA...@li... >> https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list >> Archives: >> http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list > > ------------------------------------------------------------------------------ > CenturyLink Cloud: The Leader in Enterprise Cloud Services. > Learn Why More Businesses Are Choosing CenturyLink Cloud For > Critical Workloads, Development Environments & Everything In Between. > Get a Quote or Start a Free Trial Today. > http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk > _______________________________________________ > GDAlgorithms-list mailing list > GDA...@li... > https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list > Archives: > http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list |