Small SPARQL, RDQL, etc. Cheat Sheet - Revision history

Admin: /* Basic Observations */

2012-09-04T01:26:10Z

‎Basic Observations

Admin: /* Basic Observations */

2012-09-04T01:24:14Z

‎Basic Observations

Admin at 02:10, 2 September 2012

2012-09-02T02:10:41Z

Admin: /* Basic Observations */ added Query Complexity Factors subsection

2012-09-02T02:08:34Z

‎Basic Observations: added Query Complexity Factors subsection

Admin at 20:54, 31 August 2012

2012-08-31T20:54:10Z

Admin: /* Basic Observations */

2012-08-30T03:18:52Z

‎Basic Observations

Admin: Created page with "For the lack of the same, I'll put here some of my notes on SPARQL, RDQL, graph databases, and semantic web related topics in general... Will probably branch out to several pages..."

2012-08-30T00:25:56Z

Created page with "For the lack of the same, I'll put here some of my notes on SPARQL, RDQL, graph databases, and semantic web related topics in general... Will probably branch out to several pages..."

New page

For the lack of the same, I'll put here some of my notes on SPARQL, RDQL, graph databases, and semantic web related topics in general... Will probably branch out to several pages in future, but for now, it's just a small mess.

== Introduction ==

I'm using Redland 1.0.13, Raptor 2.0.4, and Rasqal 0.9.26 as reference implementation of SPARQL 1.0, SPARQL 1.1, and RDQL.

== Basic Observations ==

Main rule of thumb I observed in many systems - fixing object ("RHS") of the triplet is much cheaper than fixing subject ("LHS"), sometimes in the order of 3-4 magnitudes.

In other words - having only triplets
''something'' contains ''a-thing''
and querying for "all things that given fixed ''something'' contains"
SELECT ?thing WHERE { ''something'' contains ?thing }
might take substantially longer than storing also
''a-thing'' contained-in ''something''
triplets along, and querying for "all things contained in given fixed ''something''"
SELECT ?thing WHERE { ?thing contained-in ''something'' }

@@ Line 15: / Line 15: @@
 For example, lets find all items that "user X" bought, that are blue. Lets presume that there are many more blue items in the DB than items that "user X" bought.
-Then the query (get all things that are blue, that "user X" bought):
+Then the query (get all things that "user X" bought, that are blue):
- SELECT ?thing WHERE { ?thing _:color "blue" . "user X" _:bought ?thing . }
-will typically run (much) slower, than  (get all things that "user X" bought, that are blue):
   SELECT ?thing WHERE { "user X" _:bought ?thing . ?thing _:color "blue" . }
-Note that the result set is identical, but the former query first takes all the blue things and picks those bought by "user X", while the latter takes the small set of bought items and picks just the blue ones.
+Note that the result set is identical, but the latter query first takes all the blue things and picks those bought by "user X", while the former takes the small set of bought items and picks just the blue ones.
 In general - '''graph databases are incredibly powerful tools, but it's up to you to make them smart!'''

@@ Line 36: / Line 36: @@
 In my tests, "hashes" storage was 10-100 times faster for "trivial" queries, requiring single step or single comparison, than "mysql" storage (but both in <10 ms times).
 On the other hand, the "hashes" storage explodes on queries requiring more steps through the graph, or using joins/intersections, or including boolean operations - e.g. selecting number of items that have 2+ attributes in common is 100+ times faster using "mysql" storage (where "hashes" queries go to seconds already on small graphs).
 == Turtle and SPARQL ==

@@ Line 4: / Line 4: @@
 I'm using Redland 1.0.13, Raptor 2.0.4, and Rasqal 0.9.26 as reference implementation of SPARQL 1.0, SPARQL 1.1, and RDQL.
 == Basic Observations ==
@@ Line 14: / Line 16: @@
 Then the query (get all things that are blue, that "user X" bought):
-  SELECT ?thing WHERE { ?thing _:color "blue" . "user X" _:bought ?thing }
+  SELECT ?thing WHERE { ?thing _:color "blue" . "user X" _:bought ?thing . }
 will typically run (much) slower, than  (get all things that "user X" bought, that are blue):
-  SELECT ?thing WHERE { "user X" _:bought ?thing . ?thing _:color "blue" }
+  SELECT ?thing WHERE { "user X" _:bought ?thing . ?thing _:color "blue" . }
 Note that the result set is identical, but the former query first takes all the blue things and picks those bought by "user X", while the latter takes the small set of bought items and picks just the blue ones.

← Older revision		Revision as of 02:08, 2 September 2012
Line 27:		Line 27:

	The gain is storage and application specific - e.g. Redland library loads ~100K model 5 times faster using "hashes" storage then when adding statements one by one, but the difference becomes negligible using MySQL storage.		The gain is storage and application specific - e.g. Redland library loads ~100K model 5 times faster using "hashes" storage then when adding statements one by one, but the difference becomes negligible using MySQL storage.
		+
		+	=== Query Complexity Factors ===
		+
		+	Please note that while storage system inherently makes retrieval of data slower when using persistent storage, the system might take advantage of storage's preexisting capabilities.
		+
		+	In my tests, "hashes" storage was 10-100 times faster for "trivial" queries, requiring single step or single comparison, than "mysql" storage (but both in <10 ms times).
		+	On the other hand, the "hashes" storage explodes on queries requiring more steps through the graph, or using joins/intersections, or including boolean operations - e.g. selecting number of items that have 2+ attributes in common is 100+ times faster using "mysql" storage (where "hashes" queries go to seconds already on small graphs).

	== Turtle and SPARQL ==		== Turtle and SPARQL ==

@@ Line 6: / Line 6: @@
 == Basic Observations ==
 Main rule of thumb I observed in many systems - try to guess what statement of the ''WHERE'' clause restricts the triplets set the most, and order the statements in increasing order of generality (most restrictive first).
@@ Line 19: / Line 21: @@
 In general - '''graph databases are incredibly powerful tools, but it's up to you to make them smart!'''

@@ Line 7: / Line 7: @@
 == Basic Observations ==
-Main rule of thumb I observed in many systems - fixing object ("RHS") of the triplet is much cheaper than fixing subject ("LHS"), sometimes in the order of 3-4 magnitudes.
+Main rule of thumb I observed in many systems - try to guess what statement of the ''WHERE'' clause restricts the triplets set the most, and order the statements in increasing order of generality (most restrictive first).
-In other words - having only triplets
+For example, lets find all items that "user X" bought, that are blue. Lets presume that there are many more blue items in the DB than items that "user X" bought.
- ''something'' contains ''a-thing''
-and querying for "all things that given fixed ''something'' contains"
+Then the query (get all things that are blue, that "user X" bought):
-  SELECT ?thing WHERE { ''something'' contains ?thing }
+  SELECT ?thing WHERE { ?thing _:color "blue" . "user X" _:bought ?thing }
-might take substantially longer than storing also
+will typically run (much) slower, than  (get all things that "user X" bought, that are blue):
-  ''a-thing'' contained-in ''something''
+  SELECT ?thing WHERE { "user X" _:bought ?thing . ?thing _:color "blue" }
-triplets along, and querying for "all things contained in given fixed ''something''"
-  SELECT ?thing WHERE { ?thing contained-in ''something'' }
+Note that the result set is identical, but the former query first takes all the blue things and picks those bought by "user X", while the latter takes the small set of bought items and picks just the blue ones.