org.dice_research.squirrel.sink.impl.sparql

Class TDBSink

java.lang.Object

org.dice_research.squirrel.sink.impl.sparql.AbstractBufferingSink

org.dice_research.squirrel.sink.impl.sparql.TDBSink

All Implemented Interfaces:

Closeable, AutoCloseable, org.dice_research.squirrel.sink.quadbased.QuadBasedSink, org.dice_research.squirrel.sink.Sink, org.dice_research.squirrel.sink.SinkBase, org.dice_research.squirrel.sink.triplebased.AdvancedTripleBasedSink, org.dice_research.squirrel.sink.triplebased.TripleBasedSink, org.dice_research.squirrel.sink.UnstructuredDataSink

public class TDBSink
extends AbstractBufferingSink
implements org.dice_research.squirrel.sink.triplebased.AdvancedTripleBasedSink, org.dice_research.squirrel.sink.Sink

A sink which stores the data in different graphs in a sparql based db.

Field Summary

Fields

Modifier and Type	Field and Description
private static org.slf4j.Logger	LOGGER
private org.dice_research.squirrel.data.uri.CrawleableUri	metaDataGraphUri Uri for the MetaData graph, will be stored in the default graph
private String	queryDatasetURI The URI of the DB in which querys can be performed.
private String	updateDatasetURI The URI of the DB in which updates can be performed.
private String	updateMetaDataUri The URI to the metadata DB in which updates can be performed.

Fields inherited from class org.dice_research.squirrel.sink.impl.sparql.AbstractBufferingSink

bufferSize, DEFAULT_BUFFER_SIZE

Constructor Summary

Constructors

Constructor and Description
TDBSink(String host, String port, String updateAppendix, String queryAppendix, String updateMetaDataAppendix, String queryMetaDataAppendix) Constructor of SparqlBasedSink.

Method Summary

Modifier and Type	Method and Description
void	addData(org.dice_research.squirrel.data.uri.CrawleableUri uri, InputStream stream)
void	addMetaData(org.apache.jena.rdf.model.Model model)
void	flushMetadata()
String	getGraphId(org.dice_research.squirrel.data.uri.CrawleableUri uri) Get the id of the graph in which the given uri is stored.
List<org.apache.jena.graph.Triple>	getTriplesForGraph(org.dice_research.squirrel.data.uri.CrawleableUri uri)
String	getUpdateDatasetURI()
protected void	sendQuads(org.dice_research.squirrel.data.uri.CrawleableUri uri, Collection<org.apache.jena.sparql.core.Quad> buffer)
protected void	sendTriples(org.dice_research.squirrel.data.uri.CrawleableUri uri, Collection<org.apache.jena.graph.Triple> triples) Method to send all buffered triples to the database

Methods inherited from class org.dice_research.squirrel.sink.impl.sparql.AbstractBufferingSink

addQuad, addTriple, closeSinkForUri, openSinkForUri

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.dice_research.squirrel.sink.Sink

close

Methods inherited from interface org.dice_research.squirrel.sink.triplebased.TripleBasedSink

addTriple

Methods inherited from interface org.dice_research.squirrel.sink.quadbased.QuadBasedSink

addQuad

Methods inherited from interface org.dice_research.squirrel.sink.UnstructuredDataSink

addData, addData

Methods inherited from interface org.dice_research.squirrel.sink.SinkBase

closeSinkForUri, openSinkForUri

Field Detail

LOGGER

private static final org.slf4j.Logger LOGGER

updateMetaDataUri

private final String updateMetaDataUri

The URI to the metadata DB in which updates can be performed.

updateDatasetURI

private String updateDatasetURI

The URI of the DB in which updates can be performed.

queryDatasetURI

private String queryDatasetURI

The URI of the DB in which querys can be performed.

metaDataGraphUri

private org.dice_research.squirrel.data.uri.CrawleableUri metaDataGraphUri

Uri for the MetaData graph, will be stored in the default graph

Constructor Detail

TDBSink

public TDBSink(String host,
               String port,
               String updateAppendix,
               String queryAppendix,
               String updateMetaDataAppendix,
               String queryMetaDataAppendix)

Constructor of SparqlBasedSink.

Parameters:

host - The host name of the sink.

port - The port of the sink.

updateAppendix - The update appendix for the content data

queryAppendix - The query appendix for the content data

updateMetaDataAppendix - The update appendix for the meta data

queryMetaDataAppendix - The query appendix for the meta data

Method Detail

getTriplesForGraph

public List<org.apache.jena.graph.Triple> getTriplesForGraph(org.dice_research.squirrel.data.uri.CrawleableUri uri)

Specified by:

getTriplesForGraph in interface org.dice_research.squirrel.sink.triplebased.AdvancedTripleBasedSink

sendTriples

protected void sendTriples(org.dice_research.squirrel.data.uri.CrawleableUri uri,
                           Collection<org.apache.jena.graph.Triple> triples)

Method to send all buffered triples to the database

Specified by:

sendTriples in class AbstractBufferingSink

Parameters:

uri - the crawled CrawleableUri

tripleList - the list of Triples regarding that uri

addMetaData

public void addMetaData(org.apache.jena.rdf.model.Model model)

Specified by:

addMetaData in interface org.dice_research.squirrel.sink.Sink

addData

public void addData(org.dice_research.squirrel.data.uri.CrawleableUri uri,
                    InputStream stream)

Specified by:

addData in interface org.dice_research.squirrel.sink.UnstructuredDataSink

getGraphId

public String getGraphId(org.dice_research.squirrel.data.uri.CrawleableUri uri)

Get the id of the graph in which the given uri is stored.

Parameters:

uri - The given uri.

Returns:

The id of the graph.

getUpdateDatasetURI

public String getUpdateDatasetURI()

sendQuads

protected void sendQuads(org.dice_research.squirrel.data.uri.CrawleableUri uri,
                         Collection<org.apache.jena.sparql.core.Quad> buffer)

Specified by:

sendQuads in class AbstractBufferingSink

flushMetadata

public void flushMetadata()

Specified by:

flushMetadata in interface org.dice_research.squirrel.sink.Sink