From: Jerven B. <jer...@is...> - 2013-10-04 09:18:29
|
Hi Egon, Not a virtuoso expert. But about the mondica, I would not expect a different result between VOS6 and VOS7. The query mondica uses for measuring uptime is ASK { ?s ?p ?o .} This query is very cheap to answer no matter which store is used. So in the mondica is measuring the time needed for http transfer etc... This highly dependant on the mondica server and infrastructure implementation. It is around 1 second for the uniprot one and that is quite slow for a http connection setup like this. When I try these endpoints it is in the 100 ms range which is more normal. for server in "http://beta.sparql.uniprot.org/" "http://rdf.farmbio.uu.se/chembl";do time wget "$server/sparql?query=ASK {?s ?p ?o}";done So no VOS7 is not going to affect mondica response times. It is going to affect complex query times. Otherwise it looks to me like you have both the settings for 8gb and the default settings uncommented, I suspect that the defaults win in that case as these are the last ones. Regards, Jerven On 04/10/13 10:59, Egon Willighagen wrote: > Hi VOS community, > > we have recently upgraded our VOS7 SPARQL end point for the ChEMBL-RDF > data [0]: http://rdf.farmbio.uu.se/chembl/sparql > > It's crashed a few times (not sure why; I'm now monitoring it day by > day), but I was originally looking forward to the faster performance. > I started from a fresh empty database, and reloaded all data, but when > it was back online, I was expecting it to be faster, but when looking > at the Mondeca SPARQL uptime service, it reported the same response > times... does that make sense? > > I'm no expert at all in the VOS config files, and am wondering if it > is sensible at all (given below), and if there is anything I can do to > improve the performance of the SPARQL end point... I have no clue > where to start, and do not have time to deeply explore this either (as > a chemist you do not get funding for maintenance of this kind at > all...) > > What are things I can try? Did I need to do some specific indexing to > take advantage of the column store speed up? > > Am I doing something stupid in my config (below)? The machine this > runs on is an 8 core, but with VOS that should not help. It has 8GB of > memory. > > Egon > > 0.http://www.jcheminf.com/content/5/1/23 > > the config file: > > ========================================================== > ; > ; virtuoso.ini > ; > ; Configuration file for the OpenLink Virtuoso VDBMS Server > ; > ; To learn more about this product, or any other product in our > ; portfolio, please check out our web site at: > ; > ; http://virtuoso.openlinksw.com/ > ; > ; or contact us at: > ; > ; gen...@op... > ; > ; If you have any technical questions, please contact our support > ; staff at: > ; > ; tec...@op... > ; > > ; > ; Database setup > ; > [Database] > DatabaseFile = virtuoso.db > ErrorLogFile = virtuoso.log > LockFile = virtuoso.lck > TransactionFile = virtuoso.trx > xa_persistent_file = virtuoso.pxa > ErrorLogLevel = 7 > FileExtend = 200 > MaxCheckpointRemap = 2000 > Striping = 0 > TempStorage = TempDatabase > > > [TempDatabase] > DatabaseFile = virtuoso-temp.db > TransactionFile = virtuoso-temp.trx > MaxCheckpointRemap = 2000 > Striping = 0 > > > ; > ; Server parameters > ; > [Parameters] > ServerPort = 1113 > LiteMode = 0 > DisableUnixSocket = 1 > DisableTcpSocket = 0 > ;SSLServerPort = 2111 > ;SSLCertificate = cert.pem > ;SSLPrivateKey = pk.pem > ;X509ClientVerify = 0 > ;X509ClientVerifyDepth = 0 > ;X509ClientVerifyCAFile = ca.pem > ServerThreads = 20 > CheckpointInterval = 60 > O_DIRECT = 0 > CaseMode = 2 > MaxStaticCursorRows = 5000 > CheckpointAuditTrail = 0 > AllowOSCalls = 0 > SchedulerInterval = 10 > DirsAllowed = ., /var/data/egonw/chembl/vad > ThreadCleanupInterval = 0 > ThreadThreshold = 10 > ResourcesCleanupInterval = 0 > FreeTextBatchSize = 100000 > SingleCPU = 0 > VADInstallDir = > /usr/local/virtuoso-opensource/share/virtuoso/vad/ > PrefixResultNames = 0 > RdfFreeTextRulesSize = 100 > IndexTreeMaps = 256 > MaxMemPoolSize = 200000000 > PrefixResultNames = 0 > MacSpotlight = 0 > IndexTreeMaps = 64 > ;; > ;; When running with large data sets, one should configure the Virtuoso > ;; process to use between 2/3 to 3/5 of free system memory and to stripe > ;; storage on all available disks. > ;; > ;; Uncomment next two lines if there is 2 GB system memory free > ;NumberOfBuffers = 170000 > ;MaxDirtyBuffers = 130000 > ;; Uncomment next two lines if there is 4 GB system memory free > ;NumberOfBuffers = 340000 > ; MaxDirtyBuffers = 250000 > ;; Uncomment next two lines if there is 8 GB system memory free > NumberOfBuffers = 680000 > MaxDirtyBuffers = 500000 > ;; Uncomment next two lines if there is 16 GB system memory free > ;NumberOfBuffers = 1360000 > ;MaxDirtyBuffers = 1000000 > ;; Uncomment next two lines if there is 32 GB system memory free > ;NumberOfBuffers = 2720000 > ;MaxDirtyBuffers = 2000000 > ;; Uncomment next two lines if there is 48 GB system memory free > ;NumberOfBuffers = 4000000 > ;MaxDirtyBuffers = 3000000 > ;; Uncomment next two lines if there is 64 GB system memory free > ;NumberOfBuffers = 5450000 > ;MaxDirtyBuffers = 4000000 > ;; > ;; Note the default settings will take very little memory > ;; but will not result in very good performance > ;; > NumberOfBuffers = 10000 > MaxDirtyBuffers = 6000 > > > [HTTPServer] > ServerPort = 8892 > ServerRoot = > /usr/local/virtuoso-opensource/var/lib/virtuoso/vsp > ServerThreads = 20 > DavRoot = DAV > EnabledDavVSP = 0 > HTTPProxyEnabled = 0 > TempASPXDir = 0 > DefaultMailServer = localhost:25 > ServerThreads = 10 > MaxKeepAlives = 10 > KeepAliveTimeout = 10 > MaxCachedProxyConnections = 10 > ProxyConnectionCacheTimeout = 15 > HTTPThreadSize = 280000 > HttpPrintWarningsInOutput = 0 > Charset = UTF-8 > ;HTTPLogFile = logs/http.log > > [AutoRepair] > BadParentLinks = 0 > > [Client] > SQL_PREFETCH_ROWS = 100 > SQL_PREFETCH_BYTES = 16000 > SQL_QUERY_TIMEOUT = 0 > SQL_TXN_TIMEOUT = 0 > ;SQL_NO_CHAR_C_ESCAPE = 1 > ;SQL_UTF8_EXECS = 0 > ;SQL_NO_SYSTEM_TABLES = 0 > ;SQL_BINARY_TIMESTAMP = 1 > ;SQL_ENCRYPTION_ON_PASSWORD = -1 > > [VDB] > ArrayOptimization = 0 > NumArrayParameters = 10 > VDBDisconnectTimeout = 1000 > KeepConnectionOnFixedThread = 0 > > [Replication] > ServerName = db-WS1 > ServerEnable = 1 > QueueMax = 50000 > > > ; > ; Striping setup > ; > ; These parameters have only effect when Striping is set to 1 in the > ; [Database] section, in which case the DatabaseFile parameter is ignored. > ; > ; With striping, the database is spawned across multiple segments > ; where each segment can have multiple stripes. > ; > ; Format of the lines below: > ; Segment<number> = <size>, <stripe file name> [, <stripe file name> .. ] > ; > ; <number> must be ordered from 1 up. > ; > ; The <size> is the total size of the segment which is equally divided > ; across all stripes forming the segment. Its specification can be in > ; gigabytes (g), megabytes (m), kilobytes (k) or in database blocks > ; (b, the default) > ; > ; Note that the segment size must be a multiple of the database page size > ; which is currently 8k. Also, the segment size must be divisible by the > ; number of stripe files forming the segment. > ; > ; The example below creates a 200 meg database striped on two segments > ; with two stripes of 50 meg and one of 100 meg. > ; > ; You can always add more segments to the configuration, but once > ; added, do not change the setup. > ; > [Striping] > Segment1 = 100M, db-seg1-1.db, db-seg1-2.db > Segment2 = 100M, db-seg2-1.db > ;... > > ;[TempStriping] > ;Segment1 = 100M, db-seg1-1.db, db-seg1-2.db > ;Segment2 = 100M, db-seg2-1.db > ;... > > ;[Ucms] > ;UcmPath = <path> > ;Ucm1 = <file> > ;Ucm2 = <file> > ;... > > > [Zero Config] > ServerName = virtuoso (WS1) > ;ServerDSN = ZDSN > ;SSLServerName = > ;SSLServerDSN = > > > [Mono] > ;MONO_TRACE = Off > ;MONO_PATH = <path_here> > ;MONO_ROOT = <path_here> > ;MONO_CFG_DIR = <path_here> > ;virtclr.dll = > > > [URIQA] > DynamicLocal = 0 > DefaultHost = localhost:8890 > > > [SPARQL] > DefaultGraph = http://linkedchemistry.info/chembl/ > ImmutableGraphs = http://linkedchemistry.info/chembl/ > ResultSetMaxRows = 1000000 > MaxQueryCostEstimationTime = 5000 ; in seconds > MaxQueryExecutionTime = 80 ; in seconds > DefaultQuery = select distinct * where > {<http://linkedchemistry.info/chembl/molecule/m443> ?p ?o} > ;ExternalQuerySource = 1 > ;ExternalXsltSource = 1 > DeferInferenceRulesInit = 0 ; controls inference rules loading > ;PingService = http://rpc.pingthesemanticweb.com/ > > > [Plugins] > LoadPath = > /usr/local/virtuoso-opensource/lib/virtuoso/hosting > Load1 = plain, wikiv > Load2 = plain, mediawiki > Load3 = plain, creolewiki > Load4 = plain, im > ;Load5 = plain, wbxml2 > ;Load6 = plain, hslookup > ;Load7 = attach, libphp5.so > ;Load8 = Hosting, hosting_php.so > ;Load9 = Hosting,hosting_perl.so > ;Load10 = Hosting,hosting_python.so > ;Load11 = Hosting,hosting_ruby.so > ;Load12 = msdtc,msdtc_sample > ========================================================== > > -- ------------------------------------------------------------------- Jerven Bolleman Jer...@is... SIB Swiss Institute of Bioinformatics Tel: +41 (0)22 379 58 85 CMU, rue Michel Servet 1 Fax: +41 (0)22 379 58 58 1211 Geneve 4, Switzerland www.isb-sib.ch - www.uniprot.org Follow us at https://twitter.com/#!/uniprot ------------------------------------------------------------------- |