Usage

Using Torvend in your own applications is really simple and straightforward. I’ve removed most of the complexity by provided a TorvendClient object which performs the torrent searching task optimally right out of the box.

Basic Initialization

The most simple form of initializing a client can be done like this:

from torvend.client import TorvendClient

my_client = TorvendClient()

Note

The only subpackage exposed by the torvend package is the command line package cli. This means that you will have to specify the fully qualified name to import the TorvendClient object.

# INCORRECT
import torvend
my_client = torvend.client.TorvendClient()

# CORRECT
import torvend.client
my_client = torvend.client.TorvendClient()

# CORRECT
from torvend.client import TorvendClient
my_client = TorvendClient()

Advanced Initialization

There are serveral available options that can be specified when initializing a new TorvendClient.

Specifying Spiders

The most useful of these options are the allowed and ignored lists. These lists allow you to specify the spiders to utilize or to not utilize when performing the search.

For example, if I wanted to only search torrents using the only the spiders ThePirateBaySpider and Torrentz2Spider, I could include this class in the allowed list when initializing the client.

from torvend.spiders import (ThePirateBaySpider, Torrentz2Spider)

my_client = TorvendClient(allowed=[ThePirateBaySpider, Torrentz2Spider])

However, if I only wanted to not receive torrents from LimeTorrentsSpider, I could specify that spider in the ignored list.

from torvend.spiders import (LimeTorrentsSpider,)

my_client = TorvendClient(ignored=[LimeTorrentsSpider])

Important

The use of both the allowed and ignored fields in the same initialization is not permitted. This is because it makes no sense to allow only some spiders to run and ignore others (the allowed list already ignores them).

Customize Scrapy

You can also customize the scrapy settings by passing in the settings dictionary with updated settings. For example, if I wanted to use a different name for the scrapy bot, I could pass in my new bot name in the settings dictionary.

my_client = TorvendClient(settings={'BOT_NAME': 'my-bot'})

If you need a reference for available scrapy settings, click here.

Verbose Logging

You can enable verbose logging by simply passing the verbose flag to the TorvendClient initialization.

my_client = TorvendClient(verbose=True)

Starting Spiders

You can start up the search process through the TorvendClient by starting the spiders. Lucky for you, I’ve compressed the logic into the search() method.

Because this web-scraper is asynchronous, you need to not only supply a query to the search method, but also a callback function.

def torrent_callback(item, **kwargs):
   print(('received torrent {item}').format(item=item))


my_client = TorvendClient()
my_client.search('my query', torrent_callback)

Important

This callback must specify the positional argument item and the **kwargs dictionary. Note that the positional argument must be named “item” due to how scrapy handles it’s signals.