org.dice_research.squirrel.sink.impl.hdt

Class HdtBasedSink

java.lang.Object

org.dice_research.squirrel.sink.impl.file.FileBasedSink

org.dice_research.squirrel.sink.impl.hdt.HdtBasedSink

All Implemented Interfaces:

Closeable, AutoCloseable, org.dice_research.squirrel.sink.quadbased.QuadBasedSink, org.dice_research.squirrel.sink.Sink, org.dice_research.squirrel.sink.SinkBase, org.dice_research.squirrel.sink.triplebased.TripleBasedSink, org.dice_research.squirrel.sink.UnstructuredDataSink

public class HdtBasedSink
extends FileBasedSink

HDT File based Sink, uses the FileBasedSink to store the triples of the URI in a temp folder and parses it to HDT when the sink is closed

Author:

gsjunior

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected class	HdtBasedSink.HDTParser

Nested classes/interfaces inherited from class org.dice_research.squirrel.sink.impl.file.FileBasedSink

FileBasedSink.StreamStatus

Field Summary

Fields

Modifier and Type	Field and Description
private ExecutorService	EXECUTION_SERVICE
protected String	inputType input type for parsing the file
private static org.slf4j.Logger	LOGGER
protected FileOutputStream	out outputstream for Metadata
protected File	outputDirectory

Fields inherited from class org.dice_research.squirrel.sink.impl.file.FileBasedSink

DEFAULT_OUTPUT_LANG, outputLang, streamMapping, useCompression

Constructor Summary

Constructors

Constructor and Description
HdtBasedSink(File outputDirectory) Creates a temp file for the FileBasedsink storage

Method Summary

Modifier and Type	Method and Description
void	addData(org.dice_research.squirrel.data.uri.CrawleableUri uri, InputStream stream)
void	addTriple(org.dice_research.squirrel.data.uri.CrawleableUri uri, org.apache.jena.graph.Triple triple)
void	closeSinkForUri(org.dice_research.squirrel.data.uri.CrawleableUri uri) Recovers the temp file generated and parse it to hdt
void	openSinkForUri(org.dice_research.squirrel.data.uri.CrawleableUri uri)

Methods inherited from class org.dice_research.squirrel.sink.impl.file.FileBasedSink

addQuad, flushMetadata, generateFileName

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.dice_research.squirrel.sink.Sink

addMetaData, close

Methods inherited from interface org.dice_research.squirrel.sink.UnstructuredDataSink

addData, addData

Field Detail

LOGGER

private static final org.slf4j.Logger LOGGER

EXECUTION_SERVICE

private final ExecutorService EXECUTION_SERVICE

outputDirectory

protected File outputDirectory

inputType

protected String inputType

input type for parsing the file

out

protected FileOutputStream out

outputstream for Metadata

Constructor Detail

HdtBasedSink

public HdtBasedSink(File outputDirectory)
             throws IOException

Creates a temp file for the FileBasedsink storage

Parameters:

outputDirectory -

Throws:

IOException

Method Detail

addTriple

public void addTriple(org.dice_research.squirrel.data.uri.CrawleableUri uri,
                      org.apache.jena.graph.Triple triple)

Specified by:

addTriple in interface org.dice_research.squirrel.sink.triplebased.TripleBasedSink

Overrides:

addTriple in class FileBasedSink

openSinkForUri

public void openSinkForUri(org.dice_research.squirrel.data.uri.CrawleableUri uri)

Specified by:

openSinkForUri in interface org.dice_research.squirrel.sink.SinkBase

Overrides:

openSinkForUri in class FileBasedSink

closeSinkForUri

public void closeSinkForUri(org.dice_research.squirrel.data.uri.CrawleableUri uri)

Recovers the temp file generated and parse it to hdt

Specified by:

closeSinkForUri in interface org.dice_research.squirrel.sink.SinkBase

Overrides:

closeSinkForUri in class FileBasedSink

addData

public void addData(org.dice_research.squirrel.data.uri.CrawleableUri uri,
                    InputStream stream)

Specified by:

addData in interface org.dice_research.squirrel.sink.UnstructuredDataSink

Overrides:

addData in class FileBasedSink