tget is a suite of programs: tget and plexfix.
tget grew out of my dissatisfaction with FlexGet's behavior and configuration. Don't get me wrong, FlexGet is an amazing program in its own right, but there were some things I couldn't get it to do. And configuration is clumsy, at best. Coming from the Lisp world, YAML just sucks. Go ahead and compare tget's example configuration file to that of FlexGet. The power and flexibility of Lisp make the comparison a shutout. Of course, that assumes tget does what you want it to do.
The main limitation I wanted to overcome by writing tget, though, was to set the download quality based on time: wait 6 hours (from publication date), then download the x264 SD version, but if after 12 hours after that a version isn't available, then download the x264 720p version, and if either of those aren't available for 2 days then download the XviD SD version. The latter is really my last choice, because the quality can be pretty crappy. What I just described is not possible in FlexGet. tget makes this pretty easy.
tget isn't nearly as functional as FlexGet, though, and the feed parsing only works (currently) with two sites (TvT and BTN). I'm always looking to add more.
plexfix works around deficiencies in Plex, which doesn't recognize
certain filename formats. For example, Foo-612.mp4 would not be
recognized by Plex, perhaps because the naming convention is
ambiguous--is it season 6 and espisode 12 or the June 12th episode of
Foo? Plex punts of filenames like this. plexfix creates symbolic
links so that Plex can see the correct season and episode.
How it works
Installation
Getting started
Configuration
Maintenance tasks
Cleanup
plexfix
Usage
Example configuration file
tget has these basic phases:
-
parsing the configuration file,
-
fetching the RSS feeds defined by the configuration file, and
-
downloading torrent files for episodes never seen before, which meet the criteria defined by the configuration file.
To accomplish the above, tget maintains a database of series and episodes, with various data about them.
Here are some details about the above phases:
-
The configuration file defines which series to monitor. When the configuration file is read, entries are created in the database for each series. You can examine this stored information with command line arguments to tget.
-
When an RSS feed is fetched, any episodes related to series defined by the configuration file, and thus in the database, are stored in the database, but marked as not downloaded.
-
Each series in the database has a
complete-toslot. This tells tget what episode number the series is completed to, and tget will never download anything at or before this episode.Say you've downloaded all the episodes of Breaking Bad up to S05E08. Any episodes which appear in the RSS feed at or before this episode will be ignored and never downloaded.
There are command line arguments, which start with
--catch-up, which can be used to manipulate thecomplete-toslot of a series. -
Episodes are sometimes downloaded out of order. You will not miss an episode of a series if it appears in the RSS feed out of order.
Say the
complete-tofor a series is S05E08, and episode S05E10 appears on one invocation of tget and S05E09 appears in a later invocation. Both episodes will be downloaded and thecomplete-towill not be updated to S05E10 until after episode S05E09 is downloaded. -
When the torrent file is downloaded, the resulting file can be given to transmission-remote for processing (transmission-remote is a command line tool used to control Transmission, the cross-platform BitTorrent client) or put in a specific directory watched by your BitTorrent client. See the example configuration file for information on how to use either of these methods.
tget has been tested on Linux and Mac OS X and depends on two other software packages:
transmission-remote -- [OPTIONAL] a program that can communicate to a local or
remote instance of Transmission. tget uses it to download the actual
episodes . I run Transmission on Mac OS X, on the same local
area network. transmission-remote makes this easy. On CentOS and
Fedora, install the transmission-common rpm. On a Mac, I suggest
looking to MacPorts.
Allegro Common Lisp -- For now, at least, no binary packages of tget are available and you need Allegro Common Lisp to build it.
After cloning the repo, build tget:
$ make
That should produce a directory tget/, which can be installed with:
$ make install
You can make a Makefile.local to override features of the GNU Make
file.
After installing the software, the next thing you need is a configuration file. See the next section for information on configuration, and the section at the end for an example configuration file, which you can edit for your needs.
The database used by tget is stored in $HOME/.tget.d. You can
change this location with the --root command line argument.
Next, you probably want to create a new database which has everything you care about marked as already downloaded. The following command will start from scratch and, assuming the RSS feed you are using supports it, mark 6 months of episodes as already downloaded:
$ tget --reset --learn --feed-interval 180
Next, you'll want to set up cron to periodically run tget, perhaps every day at 3AM:
0 3 * * * source $HOME/.bashrc; MAILTO=username@domain /usr/local/bin/tget --cron
Running more often is fine, but you need to be careful not to run
afoul of the site rules for how often you can download their RSS feed.
You can use the MAILTO environment variable to change the user that
receives the output of tget. $HOME/.bashrc is sourced to pick up
some environment variables used by the example configuration file.
You can omit this part if you do not need any environment variables.
At the highest level, the configuration file defines these entities:
-
transmission-remote settings - in the config example below I pull items from the environment, mainly because I don't want to share that private information in this documentation. You can store yours in the configuration file directly.
-
RSS feeds - this is where tget gets the information on what is available to download. Each feed is site specific and possibly user specific.
-
quality settings - these define the quality of the files you want to download. It's a very important aspect of the configuration, as it will directly affect your viewing pleasure.
-
download groups - if you have multiple people that are watching episodes, or multiple sites from which you are downloading, then you likely will want to use different groups with differing options for each group. Each group might, for example, have different download locations.
-
TV shows you care about - the
defseriesmacro defines the shows you want to download. A series is associated with a group, at a minimum, and can override the quality of what you want to download. For example, you may want to download 720p versions of Top Gear, but would accept a lower resolution of other shows.
That's the big picture. There is an example configuration file below. It is fully annotated and is a good place to start.
Define how .torrent files will be handled. There are two choices: have transmission-remote handle them or define a directory in which the .torrent files are placed. For the latter, the configuration file option would be:
(set-torrent-handler (pathname "~/Downloads/"))
This would cause tget to store all downloaded .torrent files in
the directory ~/Downloads/.
The second choice is to use make-tranmission-remote-handler to
define how to communicate with transmission-remote.
The values for each option have direct correspondence to transmission-remote command line arguments, given by the table below. Each keyword given in the table must be paired with a value, like this:
:host "download.example.com"
The :host part must come before the value.
| keyword(s) | transmission-remote argument |
|---|---|
:host & :port |
host:port |
:username & :password |
--auth username:password |
:add-paused t |
--start-paused |
:add-paused nil |
--no-start-paused |
:trash-torrent-file t |
--trash-torrent |
:ratio |
-sr ratio |
:download-path |
--download-dir path |
:docker-container-name |
See below |
:docker-user |
See below |
:ssh-user |
See below |
:ssh-identity |
See below |
When a subdirectory is specified by defseries, tget must ensure it exists before telling transmission to download into it. There are two methods for tget communicating with transmission: via docker or SSH.
You may specify one of the group of arguments, :docker-* or
:ssh-*, but not both.
:docker-container-name and :docker-user are the container's name
and user inside the container (for su) to create directories.
:ssh-user and :ssh-identity are the user and identity file to
create directories.
SSH example:
(set-torrent-handler
(make-transmission-remote-handler
:host (sys:getenv "TRANSMISSION_HOST")
:port (sys:getenv "TRANSMISSION_PORT")
:username (sys:getenv "TRANSMISSION_USER")
:password (sys:getenv "TRANSMISSION_PASS")
:ssh-user (sys:getenv "TRANSMISSION_SSH_USER")
:ssh-identity (sys:getenv "TRANSMISSION_SSH_IDENTITY")
:add-paused nil
:trash-torrent-file t
:ratio "1.04"))
In this example, the values for :host and other options are being
pulled from the environment with sys:getenv.
Docker example:
(set-torrent-handler
(make-transmission-remote-handler
...
:docker-container-name "transmission"
:docker-user (sys:getenv "USER")
...))
The valid values for each keyword option (the names after &key) are:
:priority -- any positive integer from 1 to 100. The higher the
number the higher the priority the that has this quality will be given
if there are multiple matching episodes.
:container -- the acceptable containers for the quality. A container
is, essentially, the file type of the downloaded file (e.g. mp4).
Valid values:
:avi, :mkv, :vob, :mpeg, :mp4, :iso, :wmv, :ts, :m4v, :m2ts
:source -- the acceptable source for the quality. The source is where
the stream originated. :hdtv is a common source. Valid values:
:pdtv, :hdtv, :dsr, :dvdrip, :tvrip, :vhsrip, :bluray, :bdrip, :brrip, :dvd5, :dvd9, :hddvd, :web-dl
:codec -- the codec used to encode the original source (e.g. XviD or
x264, aka h.264). Valid values:
:x264, :h.264, :xvid, :mpeg2, :divx, :dvdr, :vc-1, :wmv, :bd
:resolution -- the resolution of the encoded image (e.g. 720p).
Valid values:
:sd, :720p, :1080p, :1080i
name -- the name of the tracker, a keyword (e.g. :eztv).
url -- the URL that points to an RSS feed for this tracker. If you
require a password to access the tracker's RSS feed, it would need to
be contained in the URL.
debug-feed -- a filename which is to be substituted for the actual
URL, for debugging purposes.
public -- a boolean which indicates if this tracker is public or
not.
download-delay -- the delay imposed on downloads for this tracker.
This option allows you to give priority to different trackers, based
of time.
disabled -- a boolean indicating if the tracker has been disabled.
It is useful when trackers go offline for extended periods of time.
ratio -- the ratio to be applied to this specific tracker. It takes
precedence over the global ratio specified for Transmission, if that
is used.
upload-limit -- set the upload limit for this tracker. This is
useful for when the tracker unreliably gives you upload credit for
what you upload (I'm looking at you TvT!). Sometimes, if you upload
too fast the tracker will not count a significant amount of your upload
data.
name -- the name of the group, a keyword (e.g. :bruce).
:trackers -- a list of trackers which apply to this group.
:rss-url -- the URL of the RSS feed. This option has been
deprecated in favor of the :trackers option.
:delay -- nil or a positive integer, which represents the delay, in
hours, that episodes should be delayed from download. NOTE: this
is distinct from any site-specific delay that might be available in
the RSS feed.
ratio -- the share ratio given to all downloads for this group.
It's a string not a floating point number, so we don't have to worry
about the printed representation of floats. It no ratio is given,
then the default in Transmission is used. A ratio of "-1" means seed
forever.
:quality -- a user-defined quality symbol. See defquality.
:download-path -- The path used by transmission-remote to store
the downloaded file for this group. Because the path can be remote,
no checking on the validity of the path is done.
Required arguments:
name -- the name of the series. Case is not significant, and single
quotes are removed in parsing.
private -- a boolean indicating this series should only be
downloaded from a tracker listed as private.
group -- the group to which this series belongs.
Optional arguments:
:delay -- this allows the group delay to be overriden.
:quality -- this allows the group quality to be overriden.
:catch-up -- do not download any episodes of this series at or
before this season and episode. An example value is S05E08. Case
differences are ignored (e.g., you can use s or S).
:subdir -- put episodes of this series into a subdirectory of the
group download path. This is a workaround for limitations in Plex
Media Server, which will not recognize episodes of shows with dates in
their names instead of episode numbers. The Daily Show and
The Colbert Report are two examples of these types of shows.
:date-based -- indicate this series is date based and turn off the
accounting for complete seasons. This is useful for shows like
The Daily Show and The Colbert Report, which don't have seasons
and the episodes for these are ordered by date.
:aliases -- a list of aliases for name. This feature exists
because many sites use different names for series, The Daily Show with Jon Stewart vs. The Daily Show.
Here's how you specify it:
:aliases ("The Daily Show")
*repack-window-seconds* -- the number of seconds within the download
of an original file that a repack will be downloaded. The default
value is:
86400
This value is the number of seconds after an episode is downloaded that we will download a repack of that episode. A repack is a re-release of an episode to fix problems in the original.
After some time, tget can slow down due to the database growing. It has to do with how tget operates, temporarily storing episodes while it is deciding what to download. These temporarily-stored episodes are deleted almost immediately, but they leave holes in the database. To clean them out, you can
$ tget --compact-database
This will backup the current database and then compact it to make it more efficient. It will usually dramatically decrease the size of the database, if it's been in use for a month or more.
The easiest way to add a new series is to add it to the configuration file, like this:
(defseries "New Series" :group1)
However, sometimes you might want to add it in a caught up state. That would be:
(defseries "New Series" :group1 :catch-up "S05E08")
Which would cause episodes from S05E09 and later to be downloaded,
and nothing before it.
There are two ways you can remove a series.
The first method entails removing it from the configuration file and then running tget to remove it from the database. The removing from the database part would be:
$ tget --delete-series "True Blood"
The second method entails changing the configuration file to note this series should be removed:
(defseries "Series Name" :group1 :remove t)
You can remove this entry in the configuration file after tget has been run once.
You may remove a single episode or episodes.
Let's say someone uploaded pre-release versions of some really popular show, in SD with a really crappy frame rate. Say it was the first 4 episodes of the new Season 5 of the show. tget downloaded them and you want to delete them:
$ tget --delete-episode "Game of Bones S05E01"
$ tget --delete-episode "Game of Bones S05E02"
$ tget --delete-episode "Game of Bones S05E03"
$ tget --delete-episode "Game of Bones S05E04"
$ tget --catch-up-series "Game of Bones S04"
The last command was to make sure you get them in the future, but also make sure to upgrade your config file so that only high quality episodes will be downloaded in the future:
(defseries "Game of Bones :me :private t :delay 0 :quality :720p)
This is a mode of tget, when the --cleanup or -c arguments are
given. It is intended to be run manually or from cron. I recommend
that you run it manually for a while, since it will take a while to
tune the configuration so that you do not receive hit and runs from
any tracker.
With no optional arguments, nothing is removed. tget prints the status of torrents and videos:
$ tget -c
These torrents are complete:
T name %done ratio seeded left
F at midnight 2015.06.10 100% 1.22 1d 15:45:19
F the daily show 2015.06.09 100% 0.64 3d 01:45:17
F at midnight 2015.06.08 100% 1.02 3d 18:37:16
These torrents are incomplete:
T name %done ratio seeded left
F the daily show 2015.06.10 100% 0.60 2d 00:45:26 23:14:34
B the daily show 2015.06.11 100% 0.22 23:45:30 2d 00:14:30
B the nightly show with larry wilmore 2015 100% 0.22 19:43:26 2d 04:16:34
F the graham norton show S17E10 100% 1.10 14:31:43 2d 09:28:17
B the nightly show with larry wilmore 2015 100% 0.24 1d 19:37:36 1d 04:22:24
F hannibal S03E02 100% 1.10 1d 10:41:49 1d 13:18:11
F at midnight 2015.06.09 100% 1.01 2d 18:44:23 05:15:37
F at midnight 2015.06.11 100% 0.88 15:42:07 2d 08:17:53
F the ultimate fighter S21E08 100% 0.49 2d 09:36:32 14:23:28
B the nightly show with larry wilmore 2015 100% 0.23 1d 19:37:09 1d 04:22:51
B the nightly show with larry wilmore 2015 100% 0.29 1d 19:37:22 1d 04:22:38
B ridiculousness S06E29 100% 0.24 10:45:16 2d 13:14:44
F childrens hospital us S06E13 100% 0.62 09:36:38 2d 14:23:22
TV:Adrian+Kevin:
NO seeding:ridiculousness.S06E29.mp4
TV:Kevin:
NO 68h<72h:Last.Week.Tonight.With.John.Oliver.S02E16.HDTV.x264-BATV.mp4
NO seeding:the.daily.show.2015.06.09.nick.offerman.hdtv.x264-crooks.mp4
NO seeding:the.daily.show.2015.06.10.colin.quinn.hdtv.x264-crooks.mp4
NO seeding:the.daily.show.2015.06.11.mark.ruffalo.hdtv.x264-crooks.mp4
NO 68h<72h:the.daily.show.2015.06.08.nicola.sturgeon.hdtv.x264-crooks.mp4
NO seeding:at.midnight.2015.06.08.mp4
NO seeding:at.midnight.2015.06.11.mp4
NO seeding:The.Graham.Norton.Show.S17E10.720p.HDTV.x264-FTP.mkv
NO seeding:The.Ultimate.Fighter.S21E08.HDTV.x264-KOENiG.mp4
To remove the completed torrents, use --remove:
$ tget -c --remove
These torrents were removed:
T name %done ratio seeded left
F at midnight 2015.06.10 100% 1.22 1d 15:45:47
F the daily show 2015.06.09 100% 0.64 3d 01:45:45
F at midnight 2015.06.08 100% 1.02 3d 18:37:44
These torrents are incomplete:
T name %done ratio seeded left
F the daily show 2015.06.10 100% 0.60 2d 00:45:54 23:14:06
B the daily show 2015.06.11 100% 0.22 23:45:58 2d 00:14:02
B the nightly show with larry wilmore 2015 100% 0.22 19:43:54 2d 04:16:06
F the graham norton show S17E10 100% 1.10 14:32:11 2d 09:27:49
B the nightly show with larry wilmore 2015 100% 0.24 1d 19:38:04 1d 04:21:56
F hannibal S03E02 100% 1.10 1d 10:42:17 1d 13:17:43
F at midnight 2015.06.09 100% 1.01 2d 18:44:51 05:15:09
F at midnight 2015.06.11 100% 0.88 15:42:35 2d 08:17:25
F the ultimate fighter S21E08 100% 0.49 2d 09:37:00 14:23:00
B the nightly show with larry wilmore 2015 100% 0.23 1d 19:37:37 1d 04:22:23
B the nightly show with larry wilmore 2015 100% 0.29 1d 19:37:50 1d 04:22:10
B ridiculousness S06E29 100% 0.24 10:45:44 2d 13:14:16
F childrens hospital us S06E13 100% 0.62 09:37:06 2d 14:22:54
TV:Adrian+Kevin:
NO seeding:ridiculousness.S06E29.mp4
TV:Kevin:
NO 68h<72h:Last.Week.Tonight.With.John.Oliver.S02E16.HDTV.x264-BATV.mp4
NO 32h<72h:the.daily.show.2015.06.09.nick.offerman.hdtv.x264-crooks.mp4
NO seeding:the.daily.show.2015.06.10.colin.quinn.hdtv.x264-crooks.mp4
NO seeding:the.daily.show.2015.06.11.mark.ruffalo.hdtv.x264-crooks.mp4
NO 68h<72h:the.daily.show.2015.06.08.nicola.sturgeon.hdtv.x264-crooks.mp4
NO 10h<72h:at.midnight.2015.06.08.mp4
NO seeding:at.midnight.2015.06.11.mp4
NO seeding:The.Graham.Norton.Show.S17E10.720p.HDTV.x264-FTP.mkv
NO seeding:The.Ultimate.Fighter.S21E08.HDTV.x264-KOENiG.mp4
Now we have:
$ tget -c
These torrents are incomplete:
T name %done ratio seeded left
F the daily show 2015.06.10 100% 0.60 2d 00:46:03 23:13:57
B the daily show 2015.06.11 100% 0.22 23:46:07 2d 00:13:53
B the nightly show with larry wilmore 2015 100% 0.22 19:44:03 2d 04:15:57
F the graham norton show S17E10 100% 1.10 14:32:20 2d 09:27:40
B the nightly show with larry wilmore 2015 100% 0.24 1d 19:38:13 1d 04:21:47
F hannibal S03E02 100% 1.10 1d 10:42:26 1d 13:17:34
F at midnight 2015.06.09 100% 1.01 2d 18:45:00 05:15:00
F at midnight 2015.06.11 100% 0.88 15:42:44 2d 08:17:16
F the ultimate fighter S21E08 100% 0.49 2d 09:37:09 14:22:51
B the nightly show with larry wilmore 2015 100% 0.23 1d 19:37:46 1d 04:22:14
B the nightly show with larry wilmore 2015 100% 0.29 1d 19:37:59 1d 04:22:01
B ridiculousness S06E29 100% 0.24 10:45:53 2d 13:14:07
F childrens hospital us S06E13 100% 0.62 09:37:15 2d 14:22:45
TV:Adrian+Kevin:
NO seeding:ridiculousness.S06E29.mp4
TV:Kevin:
NO 68h<72h:Last.Week.Tonight.With.John.Oliver.S02E16.HDTV.x264-BATV.mp4
NO 32h<72h:the.daily.show.2015.06.09.nick.offerman.hdtv.x264-crooks.mp4
NO seeding:the.daily.show.2015.06.10.colin.quinn.hdtv.x264-crooks.mp4
NO seeding:the.daily.show.2015.06.11.mark.ruffalo.hdtv.x264-crooks.mp4
NO 68h<72h:the.daily.show.2015.06.08.nicola.sturgeon.hdtv.x264-crooks.mp4
NO 10h<72h:at.midnight.2015.06.08.mp4
NO seeding:at.midnight.2015.06.11.mp4
NO seeding:The.Graham.Norton.Show.S17E10.720p.HDTV.x264-FTP.mkv
NO seeding:The.Ultimate.Fighter.S21E08.HDTV.x264-KOENiG.mp4
To remove the watched torrents before the configured waiting period of 72h is up (set the waiting period to 0, for example):
$ tget -c --remove -h0
These torrents are incomplete:
T name %done ratio seeded left
F the daily show 2015.06.10 100% 0.60 2d 00:46:16 23:13:44
B the daily show 2015.06.11 100% 0.22 23:46:20 2d 00:13:40
B the nightly show with larry wilmore 2015 100% 0.22 19:44:16 2d 04:15:44
F the graham norton show S17E10 100% 1.10 14:32:33 2d 09:27:27
B the nightly show with larry wilmore 2015 100% 0.24 1d 19:38:26 1d 04:21:34
F hannibal S03E02 100% 1.10 1d 10:42:39 1d 13:17:21
F at midnight 2015.06.09 100% 1.01 2d 18:45:13 05:14:47
F at midnight 2015.06.11 100% 0.88 15:42:57 2d 08:17:03
F the ultimate fighter S21E08 100% 0.49 2d 09:37:22 14:22:38
B the nightly show with larry wilmore 2015 100% 0.23 1d 19:37:59 1d 04:22:01
B the nightly show with larry wilmore 2015 100% 0.29 1d 19:38:12 1d 04:21:48
B ridiculousness S06E29 100% 0.24 10:46:06 2d 13:13:54
F childrens hospital us S06E13 100% 0.62 09:37:28 2d 14:22:32
TV:Adrian+Kevin:
NO seeding:ridiculousness.S06E29.mp4
TV:Kevin:
rm /me/tplex/content/videos/kevin/Last.Week.Tonight.With.John.Oliver/Last.Week.Tonight.With.John.Oliver.S02E16.HDTV.x264-BATV.mp4
rm /me/tplex/content/videos/kevin/The.Daily.Show/the.daily.show.2015.06.09.nick.offerman.hdtv.x264-crooks.mp4
NO seeding:the.daily.show.2015.06.10.colin.quinn.hdtv.x264-crooks.mp4
NO seeding:the.daily.show.2015.06.11.mark.ruffalo.hdtv.x264-crooks.mp4
rm /me/tplex/content/videos/kevin/The.Daily.Show/the.daily.show.2015.06.08.nicola.sturgeon.hdtv.x264-crooks.mp4
rm /me/tplex/content/videos/kevin/At.Midnight/at.midnight.150608-yestv.mp4
rm /me/tplex/content/videos/kevin/At.Midnight/at.midnight.2015.06.08.mp4
NO seeding:at.midnight.2015.06.11.mp4
NO seeding:The.Graham.Norton.Show.S17E10.720p.HDTV.x264-FTP.mkv
NO seeding:The.Ultimate.Fighter.S21E08.HDTV.x264-KOENiG.mp4
$
plexfix is a simple program run from transmission after a torrent
completes. plexfix is given no arguments. The directory and name
are passed in the environment variables TR_TORRENT_DIR and
TR_TORRENT_NAME. plexfix uses these to determine if Plex needs a
symlink to see the newly download torrent. To enable it, use the
settings.json file for transmission:
"script-torrent-done-enabled": true,
"script-torrent-done-filename": "/usr/local/lib/plexfix/plexfix",
/usr/local/lib/plexfix is the default location for the installation
of plexfix.
For debugging purposes, you can also give a complete pathname to plexfix on the command line, rather than setting the above environment variables.
plexfix also accepts these arguments:
-d :: debug mode
-m :: do not send mail
-n :: do not execute, just say what would be done
-q :: be quiet, only print error messages
-v :: be verbose
An example email:
From: layer@somewhere
To: layer@somewhere
Subject: plexfix
OLD name: /me/tplex/content/videos/adrian+kevin/ridiculousness.0629-yestv.mp4
NEW name: /me/tplex/content/videos/adrian+kevin/ridiculousness.S06E29.mp4
There are two primary modes of operation: cleanup and download. Cleanup mode is activated by using the --cleanup or -c arguments. Download mode is activated by using the other primary behavior determining arguments. See below.
Cleanup mode has two parts: removing torrents from Transmission after they have seeded enough and removing video files watched by Plex.
[download] Primary behavior determining arguments (one of these must be given):
--add file-or-directory
--catch-up
--catch-up-series series-episode-name
--check-database
--clean-database
--compact-database
--delete-episodes series-name
--delete-episode episode-description
--delete-orphans
--delete-series series-name
--dump-all
--dump-complete-to
--dump-episodes series-name
--dump-orphans
--dump-series series-name
--dump-stats
--skip series-name
[download] Behavior modifying arguments:
--auto-backup condition
--config file
--cron
--db database-name
--debug
--force
--learn
--reset
--root data-directory
--verbose or -v
[cleanup] Behavior modifying arguments:
-h
--remove
--remove-bad
--remove-seeded
--remove-watched
--torrent-info
--torrents-only
The tget options are below. When there is an argument naming series, these are canonicalized by removing single quotes and converting to lower case. However, the series names presented to you will be stored in their original form.
-
--helpPrint full help text and exit.
The following are arguments controlling primary behavior:
-
--cleanupor-cActive cleanup mode. See the cleanup options below.
-
--runThe primary mode of operation, whereby RSS feeds are retrieved, searched for new episodes and those episode torrents downloaded.
-
--add thingIf
thingis a file, it should be a.torrentfile, which is manually added. Ifthingis a directory, then all the.torrentfiles in the directory are added. This circumvents any matching and assumes the episodes should be downloaded. The series name,episode and season numbers are taken directly from the file name. If the information cannot be extracted, you can rename the files to suittget.If the episode has already been downloaded, then
--forceis required to maketgetdownload it again. -
--catch-upGo through the database and make the newest episode of each series the oldest episode that will ever be downloaded for that series; this prevents old episodes, which are released from time to time, from being downloaded.
-
--catch-up-series series-epCatch series up to the episode given in the companion argument. See examples below.
-
--check-databaseReport on items in the database which can be cleaned up.
-
--clean-databaseRemove items reported by
--check-database. -
--compact-databaseThis saves and restores the database, compacting it at the same time.
-
--delete-episodes series-nameDelete episodes with series name matching
series-name. This is permanent! Using this option with --auto-backup force is recommended.--forcecan be used if there are multiple episodes that match. -
--delete-episode episode-descriptionDelete the episode matching
episode-description. This is permanent! Using this option with --auto-backup force is recommended. Example:--delete-episode "I Love Lucy S05E01"
-
--delete-orphansDelete orphaned series and episodes from the database. See
--dump-orphansfor more information. -
--delete-series series-nameDelete series with series name matching
series-name. This is permanent! Using this option with --auto-backup force is recommended. -
--dump-allDump all
episodeobjects to stdout. -
--dump-complete-toDump a table of series and last downloaded information for all series in the database to stdout. See --catch-up-series.
-
--dump-episodes series-nameDump all episode objects matching series name
series-nameto stdout. -
--dump-orphansDump orphaned series and episodes. Orphaned series are those which do not appear in the config file but exist in the database. Orphaned episodes are those are in the database but have no corresponding series object.
-
--dump-series series-nameDump all series objects matching series name
series-nameto stdout. -
--dump-statsDump information about the database to stdout.
-
--skip series-nameSkip the next episode of
series-name. It does so by using the last downloaded episode and incrementing it by 1.Note: this has no effect on date-based series. See the
:date-basedoption todefseries.
The following options augment the options above or have the stated side effects:
-
--auto-backup {reset|program-update|schema-update|restore|force|never}Perform a backup of the database under specific conditions given by the companion argument:
compact- when the database is being compactedforce- alwaysnever- neverprogram-update- when the program changed since the last db updatereset- when the database is being resetschema-update- when the schema changes
The default is to make backups for all conditions above.
-
--config fileLoad
fileas the configuration file instead of one of the built-in defaults. The default list (searched in this order) is to load the first file found in the list:config.clin the same directory as the program$HOME/.tget.clconfig.clin the tget data directory (see the --root option)
-
--cronQuiet mode, for cron jobs.
-
--db databaseThe name of the database. The default is
dbin the tget data directory (see the --root). This name will itself become a directory. -
--debugRun in debug mode. In this mode, torrents are not downloaded and the debug feed defined by the configuration file is used. Also, the program is more verbose. This is for testing and is not recommended. It implies
--learn. -
--forceThe meaning of this argument depends on the other arguments and context in which it is given.
-
--learnMark episodes which match the configuration as downloaded. This is useful when using tget for the first time. See the examples below.
-
--resetReset database before beginning operation--this removes all data from the database. The default --auto-backup settings will cause a backup to be performed before the reset.
-
--root data-directoryChange the data directory, the defaults is $HOME/.tget.d/
-
--verboseor-vVerbose mode. For now, it makes
--dump-episodesprint detailed information on episodes.
Cleanup options:
-
-h hoursSet the threshold for how long after a video is watched that it can be considered for removal. The value of 0 can be used to remove any videos which have been watched, for which seeding is complete.
The default is 72 hours, or 3 days.
-
--removeRemove videos which meet the criteria for removal. This means removing the torrent from Transmission and removing the file from disk.
-
--remove-badA special cleanup mode, which causes only unregistered torrents to be removed from Transmission. When this argument is given, without the other cleanup mode arguments (
-cand--cleanup), it is used as a pass before torrents are downloaded. Along with the torrent, the episode is deleted from the database. This allows for bad torrents to be removed and redownloaded, if they should become available again. -
--remove-seededOnly remove torrents from Transmission, if they meet the seeding criteria.
-
--remove-watchedOnly remove watched files from disk, if they meet the criteria.
-
--torrent-infoList torrents torrents currently in Transmission.
-
--torrents-onlyClean up torrents only, not files on disk.
Examples:
Toss current database and catch up on shows released in the last 180 days marking them all as `downloaded'
$ tget --reset --learn
The usefulness of this is highly dependent on how far the feed for the site you are using goes back. Many sites do not have deep feeds, but some sites have parameters that allow you to go back in time. Sadly, this feature is rare these days, as it is expensive to support.
Same, but to a `temp' database in the current directory:
$ tget --reset --learn --root $PWD --db test.db
Let's see what the series object for "Regular Show" looks like. The series name is not case sensitive:
$ tget --dump-series "regular show"
These all refer to the same series:
$ tget --dump-series "james mays man lab"
$ tget --dump-series "James Mays Man Lab"
$ tget --dump-series "James May's Man Lab"
To see the episodes of the above, you could use any of the variations on the names given, for example:
$ tget --dump-episodes "James May's Man Lab"
Compact the database:
$ tget --compact-database
Catch up series to a specific episode:
# note that episodes before s04e21 have been downloaded:
$ tget --catch-up-series "regular show s04e20"
# note that all of season 4 has been downloaded:
$ tget --catch-up-series "breaking bad s04"
;; Config file for tget
(in-package :user)
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; General options
;; This is now reset once per run, so it's not too much data to save,
;; and it's good for debugging feed issues.
(setq *log-rss* (merge-pathnames "rss.log" *tget-data-directory*))
(setq *log-xml* (merge-pathnames "error.xml" *tget-data-directory*))
;; A good resource to see why something is or isn't downloading
(setq *log-file* (merge-pathnames "ep.log" *tget-data-directory*))
;; Use transmission-remote to tell your torrent client to download the
;; episode:
(set-torrent-handler
(make-transmission-remote-handler
:host (sys:getenv "TRANSMISSION_HOST")
:port (sys:getenv "TRANSMISSION_PORT")
:username (sys:getenv "TRANSMISSION_USER")
:password (sys:getenv "TRANSMISSION_PASS")
;;;; Now using transmission in a container, with no ssh
:docker-container-name "transmission"
:docker-user (sys:getenv "USER")
;;;;
;;:ssh-user (sys:getenv "TRANSMISSION_SSH_USER")
;;:ssh-identity (sys:getenv "TRANSMISSION_SSH_IDENTITY")
:add-paused nil
:trash-torrent-file t
;; The default when there is tracker defined ratio
:ratio 1.1))
;; An alternate method for downloading .torrent files: put them into a
;; specific directory, where your torrent client will see them.
#+ignore
(set-torrent-handler (pathname "~/Downloads/"))
;; Same in and outside the transmission container
(setq *download-root* "/me/tplex/content/videos/")
(defvar *for-mom* "/me/tplex/content/videos/for-mom/")
(defvar *tv-archive* "/me/tplex/content/videos/archive/")
(defvar *allowed-codecs* '(:x264 :h.264))
;;;;;;;;;;;;;;;;;;;
;; For --cleanup ;;
;;;;;;;;;;;;;;;;;;;
;; Minimum seed time in seconds: 3 days, for torrents that don't have a
;; tracker-specific rule that takes precedence over this value.
(setq *minimum-seed-seconds* #.(* 3600 24 3))
;; Ignore anything watched in the last 3 days:
(setq *ignore-watched-within* #.(* 24 3))
(setq *sqlite3-binary*
;; CentOS 6.6 doesn't have an up-to-date enough sqlite3 binary, so we use
;; one from sqlite.org (version 3.8.10.2).
"/usr/local/bin/sqlite3")
;; The directories to clean
(setq *watch-directories*
'(("/me/tplex/content/videos/kevin" "/data/videos/kevin" . "TV Shows")
("/me/tplex/content/videos/movies/kevin" "/data/videos/movies/kevin"
. "Movies")
("/me/tplex/content/videos/tmp" "/data/videos/tmp" . "temp")))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Trackers
(deftracker :freshon
;; The feed URL is
;; https://freshon.tv/rss.php?feed=dl&passkey=<passkey>"
;; Get the <passkey> from your profile page.
:url "https://freshon.tv/..."
:debug-feed :freshon
:download-delay 0
;; See if limiting the upload rate allows freshon to correctly count my
;; upload credits. Sick of uploading lots and not getting credit for it.
:upload-limit 100 ;; KB/s
:ratio 1.3
:re "freshon\\.tv"
:char "F"
:setter (lambda (torrent)
;; 1.2 ratio and 3 days should do it?
(setf (torrent-ratio-limit torrent) 1.2)
(setf (torrent-seed-min-time torrent) (* 3600 24 3))))
(deftracker :btn
;; RSS feed configured by creating a notification filter with these
;; options:
;; - categories: episode, season
;; - containers: avi, mkv, mp4
;; - codecs: x264, h.264
;; - resolutions: sd, 720p
;; All other checkboxs were unchecked.
;;
;; passkey is in DL links. Very hard to figure this out!
;;
:url "https://broadcasthe.net/..."
:debug-feed :btn
:disabled #-debug nil #+debug t
;; Can't make the delay too long or shows will fall off the RSS feed
:download-delay #-debug 1 #+debug 0
:ratio 1.5
:re "broadcasthe\\.net|landof\\.tv"
:char "B"
:setter (lambda (torrent)
(setf (torrent-ratio-limit torrent) 1.50)
(if* (torrent-seasonp torrent)
then ;; Seed for a week + slop
(setf (torrent-seed-min-time torrent)
(* 3600 24 12))
else ;; The rules state you need to seed to 1:1 or 24
;; hours. Give it some slop, to make sure I don't
;; get a H&R
;; NOTE: the BTN tracker is notorious for not
;; counting seed time, so use 3d as the minimum
;; time here to make sure we don't get a H&R
(setf (torrent-seed-min-time torrent) (* 3600 24 3)))))
(deftracker :mtv
:url "https://www.morethan.tv/..."
:disabled nil
:download-delay 0
:ratio 1.5
:re "morethantv"
:char "T"
:setter (lambda (torrent)
(setf (torrent-ratio-limit torrent) 1.5)
(setf (torrent-seed-min-time torrent) (* 3600 24 100))))
(deftracker :shazbat
:url "https://www.shazbat.tv/..."
:debug-feed :shazbat
:disabled t
:download-delay 0
:ratio 1.5
:re "shazbat|bttracker\\.tv"
:char "S"
:setter (lambda (torrent)
;; 1.2 ratio and 3 days should do it?
(setf (torrent-ratio-limit torrent) 1.2)
(setf (torrent-seed-min-time torrent) (* 3600 24 3))))
;;;;TODO: rename this to *auto-trackers* or something
(defvar *trackers*
;; Give the new guy first crack:
(list #+ignore :mtv ;; for now
:btn
#+ignore :shazbat ;; hasn't worked in forever???
))
;; for --cleanup, only manually downloaded
(deftracker :720pier
:re "720pier"
:char "7"
:setter (lambda (torrent)
;; 1.5 ratio and 3 days should do it?
(setf (torrent-ratio-limit torrent) 1.5)
(setf (torrent-seed-min-time torrent) (* 3600 24 3))))
;; for --cleanup, only manually downloaded
(deftracker :mma-tracker
:re "mma-tracker"
:char "M"
:setter (lambda (torrent)
;; Hard to seed stuff here, so seed longer.
(setf (torrent-seed-min-time torrent) (* 3600 24 4))
(setf (torrent-ratio-limit torrent) 1.5)))
(deftracker :ptn
:re "piratethenet"
:char "x"
:setter (lambda (torrent)
;; Hard to seed stuff here, so seed longer.
(setf (torrent-seed-min-time torrent) (* 3600 24 4))
(setf (torrent-ratio-limit torrent) 1.1)))
;; Public trackers, just in case
(deftracker :public-misc
:re "tracker\\.ccc\\.de|openbittorrent\\.com|thepiratebay"
:char "p"
:setter (lambda (torrent)
;; Public tracker, so just seed to 1:1
(setf (torrent-ratio-limit torrent) 1.0)))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Quality settings
(defquality :high-1080p
:priority 10
:codec *allowed-codecs*
:resolution :1080p)
(defquality :high-1080i
:priority 1
:codec *allowed-codecs*
:resolution :1080i)
(defquality :normal
;; The priority of a quality allows selection of episodes when more
;; than one quality is available at the same time, as is often the
;; case. Higher numerical priority is given precedence.
:priority 50
:source :hdtv
:codec *allowed-codecs*
:resolution :sd)
(defquality :high
:priority 40
:codec *allowed-codecs*
:resolution :720p)
(defquality :low
:priority 30
:source :hdtv
:codec :xvid
:resolution :sd)
(defquality :btn
:priority 90
:codec *allowed-codecs*
:resolution :720p)
;; The additional delay waiting to download a high quality ep while
;; waiting for a normal quality one to become available
(defvar *download-hq-delay* 0)
;; The additional delay waiting to download a low quality ep while
;; waiting for a normal or high quality one to become available
(defvar *download-lq-delay* #-debug 24 #+debug 24)
;; While I'm tuning the new delays, let's be verbose:
;;(pushnew :tget-config-debug *features*)
(defun my-quality (episode &aux (tracker (episode-tracker episode))
temp)
(flet
(#+tget-config-debug
(debug-msg (temp)
(when temp
;; we would have downloaded this if enough time had passed, so
;; let's say that
(format t "~
DEBUG: (tracker delay + quality delay) - hours avail = ~d hours for:
~a~%"
(- temp (hours-available episode)) episode))))
;; My defined quality, as a function. This allows me to download
;; different qualities based on different criteria.
;;
;; We also output progress reports for episodes not downloaded,
;; so we can monitor progress, for debugging purposes.
(when (and (null
;; See if there is an episode with :normal quality. The
;; :transient keyword is important, since it restricts the
;; search to episodes we have *not* downloaded yet.
(query-episode :episode episode :quality :normal :transient t))
(eq :high (episode-quality episode))
(and tracker
(numberp (tracker-download-delay tracker))
(if* (>= (hours-available episode)
(setq temp
(+ (tracker-download-delay tracker)
*download-hq-delay*)))
thenret
else (@log ">>>waiting for ~a more hours for this HQ ep"
(- temp (hours-available episode)))
nil)))
;; :normal quality is not available and the :high quality episode
;; has been available for a set amount of hours, then take this
;; one
(return-from my-quality :high))
#+tget-config-debug (debug-msg temp)
(when (=~ "broadcasthe.net" (episode-torrent-url episode))
(return-from my-quality :btn))
(setq temp nil)
(when (and (null
;; See if there is an episode with :normal or :high
;; quality.
(or
(query-episode :episode episode :quality :normal :transient t)
(query-episode :episode episode :quality :high :transient t)))
(eq :low (episode-quality episode))
(and tracker
(>= (hours-available episode)
(setq temp
(+ (tracker-download-delay tracker)
*download-lq-delay*)))))
(return-from my-quality :low))
#+tget-config-debug (debug-msg temp)
:normal))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Groups
(defgroup :kevin
:trackers '#.*trackers*
:quality 'my-quality
:download-path (merge-pathnames "kevin/" *download-root*))
(defgroup :temp
:trackers '#.*trackers*
:quality 'my-quality
:download-path (merge-pathnames "tmp/" *download-root*))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; TV shows
;; Use ... :catch-up "S01E02" ... to start a series after the 1st ep
;; Use ... :remove t ... to delete a series
;; Use ... :subdir "dirname" ... to put the episodes into a subdirectory of
;; the group download path -- this is a hack to make Plex Media
;; Server see episodes of The Daily Show and The Colbert Report.
(defseries "8 Out of 10 Cats" :kevin :archive #.*for-mom*
:subdir "8.out.of.10.cats")
(defseries "8 Out of 10 Cats Does Countdown" :kevin :archive #.*for-mom*
:subdir "8.out.of.10.cats.does.countdown")
(defseries "American Experience" :kevin :subdir "American.Experience")
(defseries "Atlanta" :kevin :subdir "Atlanta")
(defseries "Barry" :kevin :subdir "Barry" :private t)
(defseries "Better Call Saul" :kevin :archive #.*tv-archive*)
(defseries "Corporate" :kevin)
(defseries "Elementary" :kevin :subdir "Elementary")
(defseries "Fargo" :kevin :quality :high :archive #.*tv-archive*)
(defseries "Frontline (US)" :kevin)
(defseries "Future Man" :kevin :catch-up "S01" :archive #.*tv-archive*)
(defseries "Good Talk With Anthony Jeselnik" :kevin :subdir "Good.Talk")
(defseries "Jeff Ross Presents Roast Battle" :kevin)
(defseries "Luther" :kevin)
(defseries "Man Seeking Woman" :kevin :catch-up "S01")
(defseries "Not Safe with Nikki Glaser" :kevin)
(defseries "Naked and Afraid" :kevin :catch-up "S01"
:subdir "Naked.and.Afraid")
(defseries "Naked and Afraid XL" :kevin :catch-up "S02"
:subdir "Naked.and.Afraid.XL")
(defseries "Nathan for You" :kevin)
(defseries "Nature" :kevin :subdir "Nature")
(defseries "Nova" :kevin :subdir "Nova")
(defseries "Penn & Teller: Fool Us" :kevin :quality :high
:catch-up "S02E01"
:aliases ("Penn and Teller Fool Us"))
(defseries "Rick and Morty" :kevin :archive #.*tv-archive*)
(defseries "Sherlock" :kevin)
(defseries "Star Trek: Discovery" :kevin :archive #.*tv-archive*
:aliases ("Star Trek Discovery"))
(defseries "Star Trek: Picard" :kevin :archive #.*tv-archive*)
(defseries "Taskmaster" :kevin :catch-up "S03"
:aliases ("Taskmaster UK" "Taskmaster (UK)")
:subdir "Taskmaster")
(defseries "The Graham Norton Show" :kevin)
(defseries "The Righteous Gemstones" :kevin :subdir "The.Righteous.Gemstones"
:private t)
(defseries "The Terror" :kevin :catch-up "S01" :archive #.*tv-archive*)
(defseries "Tosh.0" :kevin)
(defseries "Would I Lie To You?" :kevin :catch-up "S08E01"
:aliases ("Would I Lie To You")
:archive #.*for-mom*)
(defseries "Hannibal" :kevin :delay 0 :quality :high)
(defseries "Killing Eve" :kevin :quality :high
:subdir "Killing.Eve")
(defseries "The Mandalorian" :kevin :quality :high :archive #.*tv-archive*
:subdir "The.Mandalorian")
(defseries "Avenue 5" :kevin :subdir "Avenue.5")
(defseries "Dispatches from Elsewhere" :kevin :catch-up "S01E02")
(defseries "The Last Dance" :kevin :catch-up "S01E02")
(defseries "Penny Dreadful City of Angels" :kevin
:subdir "Penny.Dreadful.City.of.Angels")
(defseries "I Know This Much is True" :kevin)
(defseries "Resident Alien" :kevin)
(defseries "Criminal UK" :kevin)
(defseries "Loki" :kevin :catch-up "S01E01")
(defseries "Kin" :kevin)
(defseries "Squid Game" :kevin :catch-up "S01" :archive #.*tv-archive*)