SPARQL, RDQL, etc. cheat sheet

last updated 2012-09

for the lack of the same, i'll put here some of my notes on SPARQL, RDQL, graph databases, and semantic web related topics in general. will probably branch out to several pages in future, but for now, it's just a small mess…

introduction

i'm using Redland 1.0.13, Raptor 2.0.4, and Rasqal 0.9.26 as the reference implementation of SPARQL 1.0, SPARQL 1.1, and RDQL

most of the timing and optimization hints presented here are derived from experiments with Redland (and to a lesser extent, with 4store)

basic observations

optimize `WHERE`s

main rule of thumb i observed in many systems - try to guess which statement of the WHERE clause restricts the triplet set the most, and order the statements in increasing order of generality (most restrictive first)

for example, let's find all items that "user X" bought, that are blue. let's presume that there are many more blue items in the DB than items that "user X" bought

then the query (get all things that "user X" bought, that are blue):

SELECT ?thing WHERE { "user X" _:bought ?thing . ?thing _:color "blue" . }

will typically run (much) faster than (get all things that are blue, that "user X" bought):

SELECT ?thing WHERE { ?thing _:color "blue" . "user X" _:bought ?thing . }

note that the result set is identical, but the latter query first takes all the blue things and picks those bought by "user X", while the former takes the small set of bought items and picks just the blue ones

in general - graph databases are incredibly powerful tools, but it's up to you to make them smart!

loading vs insertion

in general, loading a model from a file is faster than inserting triplets one by one from code. of course, esp. if the model first loads the data and then indexes them, the gain might be significant for large(r) amounts of data

the gain is storage and application specific - e.g. the Redland library loads a ~100K model 5 times faster using "hashes" storage than when adding statements one by one, but the difference becomes negligible using MySQL storage

query complexity factors

please note that while the storage system inherently makes retrieval of data slower when using persistent storage, the system might take advantage of the storage's preexisting capabilities

in my tests, "hashes" storage was 10-100 times faster than "mysql" storage for "trivial" queries, requiring a single step or single comparison (but both in <10 ms times). on the other hand, the "hashes" storage explodes on queries requiring more steps through the graph, or using joins/intersections, or including boolean operations - e.g. selecting the number of items that have 2+ attributes in common is 100+ times faster using "mysql" storage (where "hashes" queries go to seconds already on small graphs)

avoid `FILTER BY` and other comparisons where speed matters

i know this sounds trivial, but often you'll use filtering of input data just "to make it more precise" or "to be more accurate"

but if you can avoid the filtering, or neglect the influence of the mistake, do it

e.g. if you want the average value of an attribute for all but one user, rather go for "for all", and either get rid of the one user in postprocess, or neglect the influence. the speed-up may be quite significant (5-100 times)

Turtle and SPARQL

SPARQL and Turtle share part of the syntax, and personally i prefer Turtle to other RDF syntaxes esp. due to this

the following groupings of triplets make the data easier to read, and also might give good hints to query parsers, or ease up the work for RDF importers

grouping by same subject and predicate

:a :b :c , :d , :e .

is equivalent to

:a :b :c .
:a :b :d .
:a :b :e .

grouping by same subject

:a :b :c ;
   :d :e .