Usage¶
Using Torvend in your own applications is really simple and straightforward.
I’ve removed most of the complexity by provided a TorvendClient
object which performs the torrent searching task optimally right out of the box.
Basic Initialization¶
The most simple form of initializing a client can be done like this:
from torvend.client import TorvendClient
my_client = TorvendClient()
Note
The only subpackage exposed by the torvend
package is the command line package cli
.
This means that you will have to specify the fully qualified name to import the TorvendClient
object.
# INCORRECT
import torvend
my_client = torvend.client.TorvendClient()
# CORRECT
import torvend.client
my_client = torvend.client.TorvendClient()
# CORRECT
from torvend.client import TorvendClient
my_client = TorvendClient()
Advanced Initialization¶
There are serveral available options that can be specified when initializing a new TorvendClient
.
Specifying Spiders¶
The most useful of these options are the allowed
and ignored
lists.
These lists allow you to specify the spiders to utilize or to not utilize when performing the search.
For example, if I wanted to only search torrents using the only the spiders ThePirateBaySpider
and Torrentz2Spider
, I could include this class in the allowed
list when initializing the client.
from torvend.spiders import (ThePirateBaySpider, Torrentz2Spider)
my_client = TorvendClient(allowed=[ThePirateBaySpider, Torrentz2Spider])
However, if I only wanted to not receive torrents from LimeTorrentsSpider
, I could specify that spider in the ignored
list.
from torvend.spiders import (LimeTorrentsSpider,)
my_client = TorvendClient(ignored=[LimeTorrentsSpider])
Important
The use of both the allowed
and ignored
fields in the same initialization is not permitted.
This is because it makes no sense to allow only some spiders to run and ignore others (the allowed list already ignores them).
Customize Scrapy¶
You can also customize the scrapy settings by passing in the settings
dictionary with updated settings.
For example, if I wanted to use a different name for the scrapy bot, I could pass in my new bot name in the settings dictionary.
my_client = TorvendClient(settings={'BOT_NAME': 'my-bot'})
If you need a reference for available scrapy settings, click here.
Verbose Logging¶
You can enable verbose logging by simply passing the verbose
flag to the TorvendClient
initialization.
my_client = TorvendClient(verbose=True)
Starting Spiders¶
You can start up the search process through the TorvendClient
by starting the spiders.
Lucky for you, I’ve compressed the logic into the search()
method.
Because this web-scraper is asynchronous, you need to not only supply a query to the search method, but also a callback function.
def torrent_callback(item, **kwargs):
print(('received torrent {item}').format(item=item))
my_client = TorvendClient()
my_client.search('my query', torrent_callback)
Important
This callback must specify the positional argument item
and the **kwargs
dictionary.
Note that the positional argument must be named “item” due to how scrapy handles it’s signals.