From: Matej S. <mat...@co...> - 2011-08-30 18:12:08
|
A while ago I looked at compression (not specific, but for general byte stream). I find this very suitable (or version of it): http://www.quicklz.com/ It's very simple and fast - 300MB/s. Compression rate is 50%. I say work keeping this in mind. I am curious what Theo Larrieu at JLab has done? Matej On Tue, Aug 30, 2011 at 6:23 PM, Dalesio, Leo <da...@bn...> wrote: > The flow control needs to be supported. If it is supported all of the > way down to the hardware, then it would seem to be most efficient. This > would help with the throughput when the data is bursty - but sustained > data rates have to be supported and we need to be able to drop samples > in a reasonable way. This may also be in the interface to the hardware. > How does the application know the data rate it is trying to achieve? I > think it is reasonable to require that this run on a dedicated VLan. I > would like to be able to achieve 80% of the bandwidth of the link. Is > that possible? Should we be considering compression? Some tests of > compression would be interesting. At BNL, we are working closely with > the detector developers and have our fingers down on the FPGA code. It > could be that support of compression in the IOC side can give us a first > set of numbers on processor and throughput. But the support for > compression opens the door to push it down into the detector and allow > us to kick up the overall throughput when they have detectors that > support the compression algorithm. Theo Larrieu at JLab did a lot of > work on compression. Should we contact him? > > > > ------------------------------------------------------------------------------ > Special Offer -- Download ArcSight Logger for FREE! > Finally, a world-class log management solution at an even better > price-free! And you'll get a free "Love Thy Logs" t-shirt when you > download Logger. Secure your free ArcSight Logger TODAY! > http://p.sf.net/sfu/arcsisghtdev2dev > |