org.dice_research.squirrel.fetcher.ckan.java

Class PaginatedCkanFetcher

java.lang.Object

org.dice_research.squirrel.fetcher.ckan.java.SimpleCkanFetcher

org.dice_research.squirrel.fetcher.ckan.java.PaginatedCkanFetcher

All Implemented Interfaces:

Closeable, AutoCloseable, Fetcher

public class PaginatedCkanFetcher
extends SimpleCkanFetcher
implements Fetcher

Simple Java-based CKAN Fetcher.

Author:

Geraldo de Souza Junior (gsjunior@mail.uni-paderborn.de)

Field Summary

Fields

Modifier and Type	Field and Description
private int	DEFAULT_TIMEOUT
private static org.slf4j.Logger	LOGGER
private int	PAGESIZE
private int	timeout

Fields inherited from class org.dice_research.squirrel.fetcher.ckan.java.SimpleCkanFetcher

checkForUriType, CKAN_JSON_OBJECT_MIME_TYPE, dataDirectory, mapper, NEWLINE_CHAR

Constructor Summary

Constructors

Constructor and Description
PaginatedCkanFetcher()
PaginatedCkanFetcher(int timeout) Time out for the Ckan Client

Method Summary

Modifier and Type	Method and Description
File	fetch(org.dice_research.squirrel.data.uri.CrawleableUri uri, Delayer delayer) Fetches a stream of data from the given URI by following the delay implemented by the given Delayer, stores it and returns a File object pointing to the stored data.
protected void	fetchDataset(eu.trentorise.opendata.jackan.CkanClient client, int pageSize, int offSet, OutputStream out)
static void	main(String[] args)

Methods inherited from class org.dice_research.squirrel.fetcher.ckan.java.SimpleCkanFetcher

close, fetchDataset, getDataDirectory, isCheckForUriType, setCheckForUriType, setDataDirectory

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.dice_research.squirrel.fetcher.Fetcher

fetch

Methods inherited from interface java.io.Closeable

close

Field Detail

LOGGER

private static final org.slf4j.Logger LOGGER

DEFAULT_TIMEOUT

private final int DEFAULT_TIMEOUT

See Also:

Constant Field Values

timeout

private int timeout

PAGESIZE

private final int PAGESIZE

See Also:

Constant Field Values

Constructor Detail

PaginatedCkanFetcher

public PaginatedCkanFetcher(int timeout)

Time out for the Ckan Client

Parameters:

timeout -

PaginatedCkanFetcher

public PaginatedCkanFetcher()

Method Detail

fetch

public File fetch(org.dice_research.squirrel.data.uri.CrawleableUri uri,
                  Delayer delayer)

Description copied from interface: Fetcher

Fetches a stream of data from the given URI by following the delay implemented by the given Delayer, stores it and returns a File object pointing to the stored data. If an error occurs, null is returned.

Specified by:

fetch in interface Fetcher

Overrides:

fetch in class SimpleCkanFetcher

Parameters:

uri - The URI from which data should be fetched.

Returns:

A File object pointing to the downloaded data or null if an error occurred.

fetchDataset

protected void fetchDataset(eu.trentorise.opendata.jackan.CkanClient client,
                            int pageSize,
                            int offSet,
                            OutputStream out)

main

public static void main(String[] args)
                 throws URISyntaxException,
                        IOException

Throws:

URISyntaxException

IOException