Small 4store How-To - Revision history

Admin: /* Turtle File Format */

2012-09-07T20:53:33Z

‎Turtle File Format

Admin at 20:49, 7 September 2012

2012-09-07T20:49:22Z

Admin at 20:37, 7 September 2012

2012-09-07T20:37:13Z

Admin at 15:07, 7 September 2012

2012-09-07T15:07:08Z

Admin at 14:58, 7 September 2012

2012-09-07T14:58:22Z

Admin at 00:51, 6 September 2012

2012-09-06T00:51:35Z

Admin: Created page with "The [http://4store.org/ 4store] RDF storage is "an efficient, scalable and stable RDF database". Even though it's creators [http://www.garlik.com/ Garlik] are currently using ne..."

2012-09-04T21:00:34Z

Created page with "The [http://4store.org/ 4store] RDF storage is "an efficient, scalable and stable RDF database". Even though it's creators [http://www.garlik.com/ Garlik] are currently using ne..."

New page

The [http://4store.org/ 4store] RDF storage is "an efficient, scalable and stable RDF database".

Even though it's creators [http://www.garlik.com/ Garlik] are currently using new [http://4store.org/trac/wiki/5store 5store], the project is still developed, and honestly, became even more interesting with v1.1.5, esp. since it adds support for ORDER BY together with GROUP BY - e.g. ordering by average - which was not supported in previous versions.

It's recommended that you [http://4store.org/trac/wiki/Download download tarball], not the Git snapshot. Then

<pre>./configure --prefix /some/folder/ --with-storage-path=/some/folder/4store --with-config-file=/some/folder/4store.conf CFLAGS=-O2 CPPFLAGS=-O3
make -j8
make install</pre>

Of course, you can leave the folders to their defaults if you so choose.

Then to start the 4store:

<pre>4s-backend-setup [kbname]
4s-backend [kbname]
4s-httpd -p [port] [kbname]</pre>

The above will setup storage for ''kbname'', run backend to support it, and open HTTP server at ''port'' (default 8080).

Import some data using

<pre>curl -T data.ttl 'http://localhost:8000/data/data.ttl'</pre>

Then go to http://localhost:[port]/test/ and test some SPARQL queries!

@@ Line 83: / Line 83: @@
 # example - printer of Turtle files, using the condensed markup where possible
 def prep_for_load():
-     f = open("example.ttl", "wt")
+     with open("example.ttl", "wt") as f:
-    gen = gen_statement()
+        gen = gen_statement()
-    ps, pp, po = s, p, o = next(gen)
+        ps, pp, po = s, p, o = next(gen)
-    f.write("%s %s %s "  % (print_triplet_item(s), print_triplet_item(p), print_triplet_item(o)))
+        f.write("%s %s %s "  % (print_triplet_item(s), print_triplet_item(p), print_triplet_item(o)))
-    for s, p, o in gen:
+        for s, p, o in gen:
-        if ps == s and pp == p:
+            if ps == s and pp == p:
-            f.write(", %s " % print_triplet_item(o))
+                f.write(", %s " % print_triplet_item(o))
-        elif ps == s:
+            elif ps == s:
-            f.write(";\n    %s %s "  % (print_triplet_item(p), print_triplet_item(o)))
+                f.write(";\n    %s %s "  % (print_triplet_item(p), print_triplet_item(o)))
-        else:
+            else:
-            f.write(".\n%s %s %s "  % (print_triplet_item(s), print_triplet_item(p), print_triplet_item(o)))
+                f.write(".\n%s %s %s "  % (print_triplet_item(s), print_triplet_item(p), print_triplet_item(o)))
-        ps, pp, po = s, p, o
+            ps, pp, po = s, p, o
-    f.write(".")    # add the final dot
+        f.write(".")</pre>
-    f.close()</pre>

← Older revision		Revision as of 20:49, 7 September 2012
Line 59:		Line 59:
	if line != "":		if line != "":
	print line.split("\t")</pre>		print line.split("\t")</pre>
		+
		+	=== Turtle File Format ===
		+
		+	Here's a simple Turtle format printer. IMO, Python's generator functions are ideal solution for this kind of task, as you can return the generated triplet, and not worry about what it's used for (printed to file, directly POST'ed, dumped to DB, ...), making the obviously orthogonal tasks really separate also in code.
		+
		+	<pre># this is the statements generator; do whatever you need to
		+	# generate the data to store, and just 'yield' them
		+	def gen_statement():
		+	# some loop over DB data, generator, ...
		+	# the 'subject', 'predicate', and 'object' may be numbers, URI's or strings
		+	yield subject, predicate, object
		+
		+	def print_triplet_item(i):
		+	tname = type(i).__name__
		+	if tname == 'int' or tname == "float":
		+	return str(i)
		+	if tname == 'str':
		+	if i[:7] == "http://":
		+	return '<' + i + '>'
		+	return '"' + i + '"'
		+	raise "invalid type provided to print_triplet_item()"
		+
		+	# example - printer of Turtle files, using the condensed markup where possible
		+	def prep_for_load():
		+	f = open("example.ttl", "wt")
		+	gen = gen_statement()
		+	ps, pp, po = s, p, o = next(gen)
		+	f.write("%s %s %s " % (print_triplet_item(s), print_triplet_item(p), print_triplet_item(o)))
		+	for s, p, o in gen:
		+	if ps == s and pp == p:
		+	f.write(", %s " % print_triplet_item(o))
		+	elif ps == s:
		+	f.write(";\n %s %s " % (print_triplet_item(p), print_triplet_item(o)))
		+	else:
		+	f.write(".\n%s %s %s " % (print_triplet_item(s), print_triplet_item(p), print_triplet_item(o)))
		+	ps, pp, po = s, p, o
		+	f.write(".") # add the final dot
		+	f.close()</pre>

@@ Line 11: / Line 11: @@
 Of course, you can leave the folders to their defaults if you so choose.
-Then to start the 4store:
+Now, lets start the 4store:
-<pre>4s-backend-setup [kbname]
+# start the backend to support the KB
 s-backend [kbname]
-The above will setup storage for ''kbname'', run backend to support it, and open HTTP server at ''port'' (default 8080). If you want your httpd to be accessible only locally, use <tt>-H 127.0.0.1</tt> to restrict it to listen only on localhost.
+# start the HTTP SPARQL endpoint using one of these:
+s-httpd [kbname]                           # start endpoint listening on port 8080
-Import some data using
+s-httpd -p [port] [kbname]                 # -"- on given port
 <pre>curl -T data.ttl 'http://localhost:[port]/data/data.ttl'</pre>

@@ Line 30: / Line 30: @@
 Also, while PUT works fine from ''requests'', you need to load the whole file first, and then PUT it on 4store, as 4store requires you to provide data length (when PUTting a file using ''requests'' directly from file, the length is not provided).
-For this reason I prefer to use ''curl'' via ''subprocess'' - saves process' memory, and is in fact faster.
+For this reason I prefer to use ''curl'' via ''subprocess'' - saves process' memory, and is in fact faster. Or, you can use POST to add statements to KB bit-by-bit.
 The following is a simple example of loading ''example.ttl'' file to 4store, and querying it to return first 10 "rows":

@@ Line 21: / Line 21: @@
 Import some data using
-<pre>curl -T data.ttl 'http://localhost:8000/data/data.ttl'</pre>
+<pre>curl -T data.ttl 'http://localhost:[port]/data/data.ttl'</pre>
 Then go to http://localhost:[port]/test/ and test some SPARQL queries!