Re: [Dooble-Team] welcome orbiter to dooble team
Brought to you by:
textfield
From: Michael C. <mc...@ya...> - 2008-09-14 16:33:34
|
Hi Max! >> For >> example: a web browser needs a built-in http client and a web cache. > > Of course, if the basics is done, we can further integrate. oh no, this is not an option, you need a web cache. If you load a web page and then click on another page of the same domain you always have a lot of image links for the next page that you already have loaded on the previous page. So you need a cache and some logic that states if the cache entry is fresh or stale. We have such rules in YaCy because it is also a caching proxy and the proxy needs the exacly same fresh/ stale rules as a browser cache. > For now, I > would suggest to keep priority on releasing a first release. Not to > bring in most details. We can do that later, because the project needs > a first "as is" release. Then the integration starts.. > > We need to be more explained, what the http-client in the browser > needs to do and what the web cache is for, > The http client is responsible for downloading e.g.? and the webcache > has which function to do? How should we organize that, if we compile > the code from gecko? then we need to fork or to send patches there..? from what I know of gecko, khtml, webkit is, that these are only rendering engines and have no web protocol handling integrated. I don't know anything abount such rendering things, so I am not shure. But I believe that loading and caching of pages should be outside of these rendering functions, so they shouldn't have that. >> and so you do not >> need to care about web protocols and caching. >> > > If that is a plus for yacy, we should discuss that and just do it. So > you would as well step into c++ qt maybe XUl code? Or would that show > the java YaCy side? we see in time which ideas we can develop. We just > need to be continously in touch and the mailinglist just has started. I don't know anything about qt and XUI and I would prefer not to do coding in c++. There is so much to do for YaCy and therefore I cannot help in these qt/XUI/rendering tasks. But we must define how an interface between c++ and YaCy/java is done. Any ideas? > >> The next idea I had was using a distributed web cache for a complete >> 'stealth-mode' web browsing experience. > > Interesting concept. so you mean a second Tor? how could we imagine > this idea? Hopping over YaCy nodes? or do you want to integrate i2p > (well this has no exit nodes). But YaCy would allow to surf with the > IP of any other peer, right. a worldwide mix cascade.. great idea!! no, not a second tor. tor aims for complete anonymous browsing, but the result is really slow. If we want to produce a new browser it must compete with all others and that means dooble must be _better_ that all others. My idea was just brainstorming and could mean that we use a DHT for web pages on YaCy peers maybe in different 'stealth-modes': (0) no DHT: just load the web page from the origin server (1) partial DHT: if page exists in a YaCy cache and that cache is considered fresh, use it, othervise load it from the origin (2) partial DHT plus load balancing: like (1) but load also from origin and take that file that appears faster. This has no 'stealth'- function, but could cause faster browsing (3) full DHT: load only from other YaCy peers. If they don't have the page, they load them for you (4) full DHT plus anonymous mode: tell another peer to load the page from another DHT position (plus more hops): this could work like Tor but would be equally slow like Tor. as I said, this is just brainstorming. As we can see we have one option (2) that doesn't bring more privacy but possibly more efficiency. > we only need a webproxy in each browser. > The browser of another user then does not know, if the Website was > loaded remote for indexing or for reading... Cool Concept that is > really revolutionary. Did you get this idea, while thinking with > dooble or do you have it already longer in mind? we sometimes played with such ideas when we discussed options for the proxy. I believe we shouldn't do anything that reduces performance at this time. >> There is currently no DHT for >> web pages (only for words in the index) in YaCy but I can start to >> think about that. > > So you mean for the content of urls? a website or a webpage? With > pictures? for all objects: web pages, images, flash, whatever. > You know, that google had the approach to download the whole internet. > With a webpage DHT we have one url and ZIP with the webpage containing > all html and pictures.. > That could be stored in the DHT. That allows the YaCy search to > download the snippets and to download a website from the Cache like > google, but it would allow as well, to surf the internet without any > REAL contact to the internet. we currently load the snippets from the hosting web page, because this is simple and fast. A DHT would create more privacy, but probably not more speed. >> At this time it would be only an option for the >> future if dooble uses YaCys web cache and http client protocol >> methods. >> > > We would like to start on that as soon as possible, Can some things of > these ideas considered in java in the YaCy code? otherwise we need to > code them in the browser with c++, you can support that as well? This ideas above could all be part of YaCy, so its Java code. >> What do you think? >> > > Great ideas! and we can make them!, we just need a basic release which > we then can develop and integrate these ideas. Yes, I agree: lets start with just something that works before we start doing other things to extend funktionality. Greetings, Michael |