org.dice_research.squirrel.fetcher.ftp

Class FTPFetcher

java.lang.Object

org.dice_research.squirrel.fetcher.ftp.FTPFetcher

All Implemented Interfaces:

Closeable, AutoCloseable, Fetcher

public class FTPFetcher
extends Object
implements Fetcher

A simple fetcher using the FTP protocol.

Author:

Michael Röder (michael.roeder@uni-paderborn.de)

Field Summary

Fields

Modifier and Type	Field and Description
protected static Set<String>	ACCEPTED_SCHEMES
protected File	dataDirectory
private static org.slf4j.Logger	LOGGER
private FTPRecursiveFetcher	recursiveFetcher

Constructor Summary

Constructors

Constructor and Description
FTPFetcher()

Method Summary

Modifier and Type	Method and Description
void	close()
File	fetch(org.dice_research.squirrel.data.uri.CrawleableUri uri, Delayer delayer) Fetches a stream of data from the given URI by following the delay implemented by the given Delayer, stores it and returns a File object pointing to the stored data.
private File	requestData(org.dice_research.squirrel.data.uri.CrawleableUri uri, File dataFile)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.dice_research.squirrel.fetcher.Fetcher

fetch

Field Detail

LOGGER

private static final org.slf4j.Logger LOGGER

ACCEPTED_SCHEMES

protected static final Set<String> ACCEPTED_SCHEMES

dataDirectory

protected File dataDirectory

recursiveFetcher

private FTPRecursiveFetcher recursiveFetcher

Constructor Detail

FTPFetcher

public FTPFetcher()

Method Detail

fetch

public File fetch(org.dice_research.squirrel.data.uri.CrawleableUri uri,
                  Delayer delayer)

Description copied from interface: Fetcher

Fetches a stream of data from the given URI by following the delay implemented by the given Delayer, stores it and returns a File object pointing to the stored data. If an error occurs, null is returned.

Specified by:

fetch in interface Fetcher

Parameters:

uri - The URI from which data should be fetched.

Returns:

A File object pointing to the downloaded data or null if an error occurred.

requestData

private File requestData(org.dice_research.squirrel.data.uri.CrawleableUri uri,
                         File dataFile)

close

public void close()

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable