Re: [Cppcms-users] Big cache
Brought to you by:
artyom-beilis
Menu
▾
▴
Re: [Cppcms-users] Big cache
|
From: Artyom B. <art...@ya...> - 2012-12-29 21:05:30
|
See, I don't know what exactly the data you have I can suggest: Separete your date into 3 categories: 1. In memory so the access is very fast (think how do you update/invalidate it) also think what is good to keep in memory and what is not 2. In cache what you need, remember that serialization and deserialization may cost as well. 3. Prefer to cache HTML rather than objects (as you save HTML generation) If you have some data that you can prefetch and it is not updated frequently do it. In the application I developed for my clients I use in-memory data structures/in-memory db very intensivly as it makes things faster and easier. But you should keep in mind how would you scale and when (now in near future in far future). Artyom Beilis -------------- CppCMS - C++ Web Framework: http://cppcms.com/ CppDB - C++ SQL Connectivity: http://cppcms.com/sql/cppdb/ >________________________________ > From: Lee Elenbaas <lee...@gm...> >To: cpp...@li... >Sent: Saturday, December 29, 2012 9:55 PM >Subject: Re: [Cppcms-users] Big cache > > >since this type of data is already stored in some form of a db - why not get most of this information prepared before hand in the db? >intersection of lists of friends for people who are likely to be used can be kept as a new table, or as an additional column and you can have a ready and waiting index ready for you to iterate over. >the only thing you can not prepare before hand is the online status >i would not go for any already made caching solution as a cure for the problem - i would have created approached it as a data mining problem - and calculate everything possible before hand in the background > > > > >On Thu, Dec 27, 2012 at 2:12 PM, Sergey Shutov <ser...@gm...> wrote: > >Artyom Beilis <artyomtnk@...> writes: >> >> >>> >>> See, you don't cache objects of 500MB in memory...Caching system is designed >> >>for pages and other objects of reasonable sizethat their generation is >>costly,Also remember that saving objects to cache and using it from cache >>involves somememcpy... and than serving.if you want to serve huge files/data >>(and >>cache them) it is much better to use file system for this.Put them on disk and >> >>tell lighttpd, nginx or even apache to serve them, it would do it much >>better.For >>example lighttpd would know to transfer them using sendfile directly from disk >>to >>socketwithout even copying the data to user space. >> >> >> >>Thanks for answer. But i need little bit explain. For example we have social >>network (real big social network). We have 3,4,5,6... million users. >>And we have common friends. >>Algorithm: >> >>1: >>select friend_id from friends where user_id = 1 (10000 rows) >>select friend_id from friends where user_id = 2 (6000 rows) >> >>(without join's) >> >>2: >>std::set_intersection (return data we need caching) >> >>So i need store friends, common friends, online friends etc... >> >> >> >>------------------------------------------------------------------------------ >>Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, >>MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current >>with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft >>MVPs and experts. ON SALE this month only -- learn more at: >>http://p.sf.net/sfu/learnmore_122712 >>_______________________________________________ >>Cppcms-users mailing list >>Cpp...@li... >>https://lists.sourceforge.net/lists/listinfo/cppcms-users >> > > >-- >-- >lee >Lee Elenbaas >lee...@gm... > >------------------------------------------------------------------------------ >Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, >MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current >with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft >MVPs and experts. SALE $99.99 this month only -- learn more at: >http://p.sf.net/sfu/learnmore_122912 >_______________________________________________ >Cppcms-users mailing list >Cpp...@li... >https://lists.sourceforge.net/lists/listinfo/cppcms-users > > > |
