Menu
▾
▴
Re: [Bigdata-developers] Quorum Problems
|
From: Bryan T. <br...@sy...> - 2015-03-25 11:01:26
|
I see several different manners in which ip addresses are being expressed (host name, 10.x.x.x network, 127.x.x,x network). This would appear to be the status for the one host that can not reach the others. While it has discovered zookeeper and obtained R I proxies for two other services (those 127.0.1.1 addresses look odd) the services are not responding at those addresses. This could be an ip configuration issue (I would try consistently using the private network ip addresses thoughout) or a firewall issue. Sometimes you can windup with bad proxy caches when bouncing the services and making configuration changes. If this happens the easiest thing to do is bring down all 3 services, wait a minute or two for zookeeper to definitively notice that the services are dead and expire the ephemeral znodes. Also, make sure that the services really are down using jps to list java processes and use netstat to make sure that lingering ports have been released. The check the logs for each service on restart. Once you have a proper configuration, these steps are not necessary but sometimes they can help while attempting to coverage on a correct is installation. On Wednesday, March 25, 2015, Maximilian Brodhun < br...@su...> wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Thanks for the quick answer. > I'm wondering cause I put all hosts in etc/hosts and I can ping all > servers by using the name not the IP. > > I give all servers a higher timeout time but the problem still occurs. The > status tab from blazegraph gives the following response: (maybe an RMI > Problem?) > > > Quorum Services > > * http://myhostname.de:7073/bigdata > <http://textgrid-blazequorum.gwdg.de:7073/bigdata> > <http://textgrid-blazequorum.gwdg.de:7073/bigdata> : is not joined, > pipelineOrder=0, writePipelineAddr=/10.254.1.6:3090, service=self, > extendedRunState={server=Running, quorumService=SeekConsensus @ 0, > haReady=-1, haStatus=NotReady, > serviceId=791f97d3-6c12-4470-ac61-97d03c0cd43b, now=1427274241853} > * Unable to reach service: > Proxy[HAGlue,BasicInvocationHandler[BasicObjectEndpoint[704b0793-0285-4a26-ae9a-904a3fc3b5ee,TcpEndpoint[127.0.1.1:3080 > ]]]] > * Unable to reach service: > Proxy[HAGlue,BasicInvocationHandler[BasicObjectEndpoint[f0c1c096-03d8-4d49-8093-30b50d605d8b,TcpEndpoint[127.0.1.1:3080 > ]]]] > > > Zookeeper > > tgHA-1(1 children) > quorum(4 children) > com.bigdata.quorum.zk.QuorumTokenState{lastValidToken=67,currentToken=-1,replicationFactor=3} > joined(2 children) > joined0000000300 (Ephemeral165590815907643397) > com.bigdata.quorum.zk.QuorumServiceState{serviceUUID=8deaf15c-776d-48d9-84d5-2157c56dbe48} > joined0000000301 (Ephemeral93533224253980679) > com.bigdata.quorum.zk.QuorumServiceState{serviceUUID=71ef114a-b872-470a-ac9b-0ff632aa0b59} > member(3 children) > member71ef114a-b872-470a-ac9b-0ff632aa0b59 > (Ephemeral93533224253980679) > com.bigdata.quorum.zk.QuorumServiceState{serviceUUID=71ef114a-b872-470a-ac9b-0ff632aa0b59} > member791f97d3-6c12-4470-ac61-97d03c0cd43b > (Ephemeral237648408135794694) > com.bigdata.quorum.zk.QuorumServiceState{serviceUUID=791f97d3-6c12-4470-ac61-97d03c0cd43b} > member8deaf15c-776d-48d9-84d5-2157c56dbe48 > (Ephemeral165590815907643397) > com.bigdata.quorum.zk.QuorumServiceState{serviceUUID=8deaf15c-776d-48d9-84d5-2157c56dbe48} > pipeline(3 children) > pipeline0000000401 (Ephemeral237648408135794694) > com.bigdata.quorum.zk.QuorumPipelineState{serviceUUID=791f97d3-6c12-4470-ac61-97d03c0cd43b,addrSelf=/ > 10.254.1.6:3090} > pipeline0000000403 (Ephemeral165590815907643397) > com.bigdata.quorum.zk.QuorumPipelineState{serviceUUID=8deaf15c-776d-48d9-84d5-2157c56dbe48,addrSelf=/ > 10.254.1.2:3090} > pipeline0000000404 (Ephemeral93533224253980679) > com.bigdata.quorum.zk.QuorumPipelineState{serviceUUID=71ef114a-b872-470a-ac9b-0ff632aa0b59,addrSelf=/ > 10.254.1.5:3090} > votes(1 children) > 0(2 children) > vote0000000000 (Ephemeral165590815907643397) > com.bigdata.quorum.zk.QuorumServiceState{serviceUUID=8deaf15c-776d-48d9-84d5-2157c56dbe48} > vote0000000001 (Ephemeral93533224253980679) > com.bigdata.quorum.zk.QuorumServiceState{serviceUUID=71ef114a-b872-470a-ac9b-0ff632aa0b59} > > > > Am 24.03.2015 um 14:58 schrieb Bryan Thompson: > > This could very easily be DNS. Also, java can have long timeouts (30-60 > seconds) if reverse DNS is not properly configured. > > > > You can use http://localhost:port/bigdata/status to see the detailed > status (including zookeeper). This information is also available under > the "status" tab of the workbench. > > > > Thanks, > > Bryan > > > > ---- > > Bryan Thompson > > Chief Scientist & Founder > > SYSTAP, LLC > > 4501 Tower Road > > Greensboro, NC 27410 > > br...@sy... <javascript:_e(%7B%7D,'cvml','br...@sy...');> > <mailto:br...@sy...> > <javascript:_e(%7B%7D,'cvml','br...@sy...');> > > http://blazegraph.com > > http://blog.bigdata.com <http://bigdata.com> <http://bigdata.com> > > http://mapgraph.io > > > > Blazegraph™ <http://www.blazegraph.com/> <http://www.blazegraph.com/> > is our ultra high-performance graph database that supports both RDF/SPARQL > and Tinkerpop/Blueprints APIs. MapGraph™ <http://www.systap.com/mapgraph> > <http://www.systap.com/mapgraph> is our disruptive new technology to use > GPUs to accelerate data-parallel graph analytics. > > > > CONFIDENTIALITY NOTICE: This email and its contents and attachments are > for the sole use of the intended recipient(s) and are confidential or > proprietary to SYSTAP. Any unauthorized review, use, disclosure, > dissemination or copying of this email or its contents or attachments is > prohibited. If you have received this communication in error, please notify > the sender by reply email and permanently delete all copies of the email > and its contents and attachments. > > > > > > On Tue, Mar 24, 2015 at 9:54 AM, Maximilian Brodhun < > br...@su... > <javascript:_e(%7B%7D,'cvml','br...@su...');> > <mailto:br...@su...> > <javascript:_e(%7B%7D,'cvml','br...@su...');>> wrote: > > > > > > Dear All, > > > > I'm very new to blazegraph but I changed to blazegraph cause of the > > clustering possibilities. I'm poor of having trouble with this. I want > > to cluster three nodes, all three become members in the quorum but only > > two join them. > > I discover this with zooinspector. > > > > The only difference between the three servers is that one of them > > doesn't have DNS is that a problem? Maybe on you can help me. > > > > > > My config file looks like this (the same on every machine): > > > > ## Configure basic environment variables. Obviously, you must use your > > own parameters for LOCATORS and ZK_SERVERS. > > > > ## This will not override parameters in the environment. > > > > > > # Name of the federation of services (controls the Apache River GROUPS). > > > > > > if [ -z "${FEDNAME}" ]; then > > > > export FEDNAME=tgRDFCluster > > > > fi > > > > > > # Path for local storage for this federation of services. > > > > > > if [ -z "${FED_DIR}" ]; then > > > > export FED_DIR=/home/tomcat-sesame/blazegraphCluster/data > > > > fi > > > > > > # Name of the replication cluster to which this HAJournalServer will > belong. > > > > > > if [ -z "${LOGICAL_SERVICE_ID}" ]; then > > > > export LOGICAL_SERVICE_ID=tgHA-1 > > > > fi > > > > > > # Where to find the Apache River service registrars (can also use > > multicast). > > > > > > if [ -z "${LOCATORS}" ]; then > > > > #Use for a HA1+ configuration > > > > #export LOCATORS="jini://localhost/" > > > > #HA3 example > > > > export > > LOCATORS="jini:// > textgrid-test1.gwdg.de/,jini://textgrid-test1.gwdg.de/,jini://141.5.102.206/ > <http://textgrid-test1.gwdg.de/,jini://textgrid-test1.gwdg.de/,jini://141.5.102.206/> > <http://textgrid-test1.gwdg.de/,jini://textgrid-test1.gwdg.de/,jini://141.5.102.206/> > " > > > > fi > > > > > > # Where to find the Apache Zookeeper ensemble. > > > > > > if [ -z "${ZK_SERVERS}" ] ; then > > > > #Use for single node configuration > > > > export ZK_SERVERS="localhost:2181" > > > > #Use for a multiple ZK configuration > > > > #export ZK_SERVERS="bigdata15:2081,bigdata16:2081,bigdata17:2081" > > > > fi > > > > > > #Replication Factor (set to one for HA1) configuration > > > > > > if [ -z "${REPLICATION_FACTOR}" ] ; then > > > > #Use for a HA1 configuration > > > > export REPLICATION_FACTOR=3 > > > > #Use for a HA1+ configuration > > > > #export REPLICATION_FACTOR=3 > > > > fi > > > > > > #Port for the NanoSparqlServer Jetty > > > > > > if [ -z "${JETTY_PORT}" ] ; then > > > > export JETTY_PORT=7070 > > > > fi > > > > > > #Group commit (true|false) > > > > > > if [ -z "${GROUP_COMMIT}" ] ; then > > > > export GROUP_COMMIT=true > > > > > > > > > > > ------------------------------------------------------------------------------ > > Dive into the World of Parallel Programming The Go Parallel Website, > sponsored > > by Intel and developed in partnership with Slashdot Media, is your > hub for all > > things parallel software development, from weekly thought leadership > blogs to > > news, videos, case studies, tutorials and more. Take a look and join > the > > conversation now. http://goparallel.sourceforge.net/ > > _______________________________________________ > > Bigdata-developers mailing list > > Big...@li... > <javascript:_e(%7B%7D,'cvml','Big...@li...');> > <mailto:Big...@li...> > <javascript:_e(%7B%7D,'cvml','Big...@li...');> > > https://lists.sourceforge.net/lists/listinfo/bigdata-developers > > > > > > - -- > Maximilian Brodhun > > Abteilung Forschung und Entwicklung > Georg-August-Universität Göttingen > Niedersächsische Staats- und Universitätsbibliothek Göttingen > D-37070 Göttingen > > Papendiek 14 (Historisches Gebäude, Raum 2.409) > +49 551 39-4923 (Tel.) > > br...@su... > <javascript:_e(%7B%7D,'cvml','br...@su...');> > > http://www.sub.uni-goettingende/ > http://www.rdd.sub.uni-goettingen.de/ > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQIcBAEBAgAGBQJVEn0FAAoJEBDMOSiH8mYu01EP/ibqT/5sn3dcDHRQ0upDEq4r > dJh6IYUkQnCDJJ36lgozVSlEEKHAnLgMW8HcFWwA4ox74v/NJtvnvMz+mjDuExQ1 > xZbMQHGW19pXSJoiI2rWbi/j7cwdU46EUlAnAxf96UN/P4Srg3OPcqrvGo9Y6YC9 > 5Z+WmLMBZ+kVG9++Vhe0JCoqe6L8NkjVk/wvOwM5Qdh4Et99HLUA0tFBwsK+8cQK > EBrkjDsGmsvtJQGNVygN4EWDP0GfQPXU7XzPExOhKO/mPzFTRuyd0FAzXlSWWnqR > TA77bM9rqBCo3jmWyFme84gpggVeUWMNFyLEdJDKYIU3JU/wcf+5JxHqD5xMGSQ6 > Y2e4c9MNEkkernT8XedUdcttnNBvZqN4M5AILNr19uD5zz+RKpgsg+fNRvAhXvz2 > HFevFWG7+5M87b781z6gjcVvyPSsIOvNKdNljzWYSjOwO5f1hAEz0f2LuYJF9vzf > HaCQ7Gt1tCyGUn/UPteGhUOq6xWOqHZHAgwz0eGGMpxI/4vKNgPq8huObPXvJieV > gu3/RVupNsyNwyczeMDg6OSYyz791EEKfxkKhJ9rN3DpPbAdenCE4/7udpTR7R8c > AJjBttHQ2tXAFS1Tvp5ViM9lEFXT315yLhhWkHhtMt1RVygYPrVr8p3nZmpZ67iW > IwYYhOIpyrLoai1CY6Mm > =JnID > -----END PGP SIGNATURE----- > > -- ---- Bryan Thompson Chief Scientist & Founder SYSTAP, LLC 4501 Tower Road Greensboro, NC 27410 br...@sy... http://blazegraph.com http://blog.bigdata.com <http://bigdata.com> http://mapgraph.io Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new technology to use GPUs to accelerate data-parallel graph analytics. CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments. |
