org.dice_research.squirrel.metadata

Class CrawlingActivity

java.lang.Object

org.dice_research.squirrel.metadata.CrawlingActivity

All Implemented Interfaces:

Serializable

public class CrawlingActivity
extends Object
implements Serializable

Representation of Crawling activity. A crawling activity is started by a single worker, and sends this data to the sink. So, it contains a bunch of Uris and some meta data, like timestamps for the start and end of the crawling activity.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	CrawlingActivity.CrawlingURIState

Field Summary

Fields

Modifier and Type	Field and Description
private String	activityUri A unique id.
private Calendar	dateEnded When the activity has ended.
private Calendar	dateStarted When the activity has started.
private long	numberOfTriples
private Map<String,org.apache.jena.rdf.model.Resource>	outputResource The URIs of the resources generated by this activity as well as their type as RDF Resource.
private static long	serialVersionUID
private CrawlingActivity.CrawlingURIState	state The crawling state of the uri.
private List<String>	steps
private org.dice_research.squirrel.data.uri.CrawleableUri	uri The uri for the crawling activity.
private String	workerUri URI of the worker assigned carrying out this activity.

Constructor Summary

Constructors

Constructor and Description
CrawlingActivity(org.dice_research.squirrel.data.uri.CrawleableUri uri, String workerUri) Constructor.

Method Summary

Modifier and Type	Method and Description
void	addOutputResource(String outputResource, org.apache.jena.rdf.model.Resource resourceType)
void	addStep(Class<?> clazz, String... actions)
void	finishActivity(org.dice_research.squirrel.sink.Sink sink) Finish the crawling activity and send data to sink
org.dice_research.squirrel.data.uri.CrawleableUri	getCrawleableUri()
long	getNumberOfTriples()
CrawlingActivity.CrawlingURIState	getState()
protected String	getStepsAsString()
org.dice_research.squirrel.data.uri.CrawleableUri	getUri()
org.apache.jena.rdf.model.Model	prepareMetadataModel() Prepare the metadata model and returns it.
void	setNumberOfTriples(long numberOfTriples)
void	setState(CrawlingActivity.CrawlingURIState state)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

serialVersionUID

private static final long serialVersionUID

See Also:

Constant Field Values

activityUri

private String activityUri

A unique id.

dateStarted

private Calendar dateStarted

When the activity has started.

dateEnded

private Calendar dateEnded

When the activity has ended.

uri

private org.dice_research.squirrel.data.uri.CrawleableUri uri

The uri for the crawling activity.

outputResource

private Map<String,org.apache.jena.rdf.model.Resource> outputResource

The URIs of the resources generated by this activity as well as their type as RDF Resource.

state

private CrawlingActivity.CrawlingURIState state

The crawling state of the uri.

workerUri

private String workerUri

URI of the worker assigned carrying out this activity.

numberOfTriples

private long numberOfTriples

steps

private List<String> steps

Constructor Detail

CrawlingActivity

public CrawlingActivity(org.dice_research.squirrel.data.uri.CrawleableUri uri,
                        String workerUri)

Constructor.

Parameters:

uri - the URI, which was crawled

workerUri - URI of the Worker that crawled the URI

Method Detail

setState

public void setState(CrawlingActivity.CrawlingURIState state)

finishActivity

public void finishActivity(org.dice_research.squirrel.sink.Sink sink)

Finish the crawling activity and send data to sink

prepareMetadataModel

public org.apache.jena.rdf.model.Model prepareMetadataModel()

Prepare the metadata model and returns it.

Returns:

the RDF model representing this metadata

getStepsAsString

protected String getStepsAsString()

getState

public CrawlingActivity.CrawlingURIState getState()

getCrawleableUri

public org.dice_research.squirrel.data.uri.CrawleableUri getCrawleableUri()

getUri

public org.dice_research.squirrel.data.uri.CrawleableUri getUri()

addStep

public void addStep(Class<?> clazz,
                    String... actions)

setNumberOfTriples

public void setNumberOfTriples(long numberOfTriples)

getNumberOfTriples

public long getNumberOfTriples()

addOutputResource

public void addOutputResource(String outputResource,
                              org.apache.jena.rdf.model.Resource resourceType)