org.dice_research.squirrel.fetcher.ckan.java

Class SimpleCkanFetcher

java.lang.Object

org.dice_research.squirrel.fetcher.ckan.java.SimpleCkanFetcher

All Implemented Interfaces:

Closeable, AutoCloseable, Fetcher

Direct Known Subclasses:

PaginatedCkanFetcher

public class SimpleCkanFetcher
extends Object
implements Fetcher

Simple Java-based CKAN Fetcher.

Author:

Michael Röder (michael.roeder@uni-paderborn.de)

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	checkForUriType
static String	CKAN_JSON_OBJECT_MIME_TYPE
protected File	dataDirectory
private static org.slf4j.Logger	LOGGER
protected com.fasterxml.jackson.databind.ObjectMapper	mapper
static byte	NEWLINE_CHAR

Constructor Summary

Constructors

Constructor and Description
SimpleCkanFetcher()

Method Summary

Modifier and Type	Method and Description
void	close()
File	fetch(org.dice_research.squirrel.data.uri.CrawleableUri uri, Delayer delayer) Fetches a stream of data from the given URI by following the delay implemented by the given Delayer, stores it and returns a File object pointing to the stored data.
protected void	fetchDataset(eu.trentorise.opendata.jackan.CkanClient client, String dataset, OutputStream out)
File	getDataDirectory()
boolean	isCheckForUriType()
void	setCheckForUriType(boolean checkForUriType)
void	setDataDirectory(File dataDirectory)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.dice_research.squirrel.fetcher.Fetcher

fetch

Field Detail

LOGGER

private static final org.slf4j.Logger LOGGER

CKAN_JSON_OBJECT_MIME_TYPE

public static final String CKAN_JSON_OBJECT_MIME_TYPE

See Also:

Constant Field Values

NEWLINE_CHAR

public static final byte NEWLINE_CHAR

See Also:

Constant Field Values

checkForUriType

protected boolean checkForUriType

dataDirectory

protected File dataDirectory

mapper

protected com.fasterxml.jackson.databind.ObjectMapper mapper

Constructor Detail

SimpleCkanFetcher

public SimpleCkanFetcher()

Method Detail

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException

fetch

public File fetch(org.dice_research.squirrel.data.uri.CrawleableUri uri,
                  Delayer delayer)

Description copied from interface: Fetcher

Fetches a stream of data from the given URI by following the delay implemented by the given Delayer, stores it and returns a File object pointing to the stored data. If an error occurs, null is returned.

Specified by:

fetch in interface Fetcher

Parameters:

uri - The URI from which data should be fetched.

Returns:

A File object pointing to the downloaded data or null if an error occurred.

fetchDataset

protected void fetchDataset(eu.trentorise.opendata.jackan.CkanClient client,
                            String dataset,
                            OutputStream out)

isCheckForUriType

public boolean isCheckForUriType()

Returns:

the checkForUriType

setCheckForUriType

public void setCheckForUriType(boolean checkForUriType)

Parameters:

checkForUriType - the checkForUriType to set

getDataDirectory

public File getDataDirectory()

Returns:

the dataDirectory

setDataDirectory

public void setDataDirectory(File dataDirectory)

Parameters:

dataDirectory - the dataDirectory to set