Commit graph

477 commits

Author SHA1 Message Date
Sergey M․
e2effb08a4 [YoutubeDL] Sanitize format_id (Closes #8494) 2016-02-10 21:16:58 +06:00
Yen Chi Hsuan
b286f201a8 [YoutubeDL] Do not override ie_key in url_transparent 2016-02-01 17:05:48 +08:00
Jaime Marquínez Ferrándiz
7d106a65ca Add --hls-use-mpegts option
When using the mpegts container hls vidoes can be played while being downloaded (useful if you are recording a live stream).
VLC and mpv play them file, but QuickTime doesn't.
2016-01-30 12:26:40 +01:00
dyn888
b0df5223be Update YoutubeDL.py 2016-01-28 12:07:15 +01:00
Sergey M․
b555942428 [YoutubeDL] Ensure protocol is always present 2016-01-16 10:10:28 +06:00
Yen Chi Hsuan
10d33b3473 [YoutubeDL] Introduce CSS3 like string operators 2016-01-16 09:53:12 +06:00
Sergey M․
33d2fc2f64 [YoutubeDL] Auto generate title fields corresponding to the *_number fields
Auto generate title fields corresponding to the *_number fields when missing in order to always have clean titles. This is very common for TV series.
2016-01-16 00:09:54 +06:00
Jaime Marquínez Ferrándiz
30e2f2d76f [YoutubeDL] use a more correct terminology in the error message for file:// URLs 2016-01-14 16:28:46 +01:00
Jaime Marquínez Ferrándiz
4240d50496 [YoutubeDL] improve error message for file:/// URLs 2016-01-14 14:07:54 +01:00
Jaime Marquínez Ferrándiz
6240b0a278 [YoutubeDL] urlopen: use build_opener again
Otherwise we would need to manually add handlers like HTTPRedirectHandler, instead we add a customized FileHandler instance that raises an error.
2016-01-14 08:16:39 +01:00
Jaime Marquínez Ferrándiz
e37afbe0b8 [YoutubeDL] urlopen: disable the 'file:' protocol (#8227)
If someone is running youtube-dl on a server to deliver files, the user could input 'file:///some/important/file' and youtube-dl would save that file as a video giving access to sensitive information to the user.
'file:' urls can be filtered, but the user can use an URL to a crafted m3u8 manifest like:

    #EXTM3U
    #EXT-X-MEDIA-SEQUENCE:0
    #EXTINF:10.0
    file:///etc/passwd
    #EXT-X-ENDLIST

With this patch 'file:' URLs raise URLError like for unknown protocols.
2016-01-14 00:24:04 +01:00
remitamine
6ec6cb4e95 Revert "fix typos"
This reverts commit 36a0e46c39.
2016-01-10 19:27:22 +01:00
remitamine
36a0e46c39 fix typos 2016-01-10 17:55:41 +01:00
Jakub Wilk
dfb1b1468c Fix typos
Closes #8200.
2016-01-10 17:24:28 +01:00
Philipp Hagemeister
32f9036447 [ccc] Add language information to formats 2016-01-01 13:28:45 +01:00
Sergey M․
9b9c5355e4 Rename error_to_str to error_to_compat_str 2015-12-20 07:00:39 +06:00
Sergey M․
2c74e6fa77 [YoutubeDL] Revert error_to_str for ExtractorError 2015-12-20 06:35:58 +06:00
Sergey M․
c0384f221e Use proper encoding on compat_str construction when necessary 2015-12-20 06:29:36 +06:00
Sergey M․
8900ab4d9b [YoutubeDL] More error_to_str 2015-12-20 06:22:01 +06:00
Sergey M․
fb043a6e4e [YoutubeDL] Use error_to_str 2015-12-20 06:16:19 +06:00
Sergey M․
7f8b271465 Properly convert errors to strings 2015-12-20 05:27:38 +06:00
Sergey M․
3d24bbfbe4 [YoutubeDL] Check formats for merge to be opposite (#7786) 2015-12-07 23:10:57 +06:00
Sergey M?
5c2266df4b Switch codebase to use sanitized_Request instead of
compat_urllib_request.Request

[downloader/dash] Use sanitized_Request

[downloader/http] Use sanitized_Request

[atresplayer] Use sanitized_Request

[bambuser] Use sanitized_Request

[bliptv] Use sanitized_Request

[brightcove] Use sanitized_Request

[cbs] Use sanitized_Request

[ceskatelevize] Use sanitized_Request

[collegerama] Use sanitized_Request

[extractor/common] Use sanitized_Request

[crunchyroll] Use sanitized_Request

[dailymotion] Use sanitized_Request

[dcn] Use sanitized_Request

[dramafever] Use sanitized_Request

[dumpert] Use sanitized_Request

[eitb] Use sanitized_Request

[escapist] Use sanitized_Request

[everyonesmixtape] Use sanitized_Request

[extremetube] Use sanitized_Request

[facebook] Use sanitized_Request

[fc2] Use sanitized_Request

[flickr] Use sanitized_Request

[4tube] Use sanitized_Request

[gdcvault] Use sanitized_Request

[extractor/generic] Use sanitized_Request

[hearthisat] Use sanitized_Request

[hotnewhiphop] Use sanitized_Request

[hypem] Use sanitized_Request

[iprima] Use sanitized_Request

[ivi] Use sanitized_Request

[keezmovies] Use sanitized_Request

[letv] Use sanitized_Request

[lynda] Use sanitized_Request

[metacafe] Use sanitized_Request

[minhateca] Use sanitized_Request

[miomio] Use sanitized_Request

[meovideo] Use sanitized_Request

[mofosex] Use sanitized_Request

[moniker] Use sanitized_Request

[mooshare] Use sanitized_Request

[movieclips] Use sanitized_Request

[mtv] Use sanitized_Request

[myvideo] Use sanitized_Request

[neteasemusic] Use sanitized_Request

[nfb] Use sanitized_Request

[niconico] Use sanitized_Request

[noco] Use sanitized_Request

[nosvideo] Use sanitized_Request

[novamov] Use sanitized_Request

[nowness] Use sanitized_Request

[nuvid] Use sanitized_Request

[played] Use sanitized_Request

[pluralsight] Use sanitized_Request

[pornhub] Use sanitized_Request

[pornotube] Use sanitized_Request

[primesharetv] Use sanitized_Request

[promptfile] Use sanitized_Request

[qqmusic] Use sanitized_Request

[rtve] Use sanitized_Request

[safari] Use sanitized_Request

[sandia] Use sanitized_Request

[shared] Use sanitized_Request

[sharesix] Use sanitized_Request

[sina] Use sanitized_Request

[smotri] Use sanitized_Request

[sohu] Use sanitized_Request

[spankwire] Use sanitized_Request

[sportdeutschland] Use sanitized_Request

[streamcloud] Use sanitized_Request

[streamcz] Use sanitized_Request

[tapely] Use sanitized_Request

[tube8] Use sanitized_Request

[tubitv] Use sanitized_Request

[twitch] Use sanitized_Request

[twitter] Use sanitized_Request

[udemy] Use sanitized_Request

[vbox7] Use sanitized_Request

[veoh] Use sanitized_Request

[vessel] Use sanitized_Request

[vevo] Use sanitized_Request

[viddler] Use sanitized_Request

[videomega] Use sanitized_Request

[viewvster] Use sanitized_Request

[viki] Use sanitized_Request

[vk] Use sanitized_Request

[vodlocker] Use sanitized_Request

[voicerepublic] Use sanitized_Request

[wistia] Use sanitized_Request

[xfileshare] Use sanitized_Request

[xtube] Use sanitized_Request

[xvideos] Use sanitized_Request

[yandexmusic] Use sanitized_Request

[youku] Use sanitized_Request

[youporn] Use sanitized_Request

[youtube] Use sanitized_Request

[patreon] Use sanitized_Request

[extractor/common] Remove unused import

[nfb] PEP 8
2015-11-23 21:56:23 +06:00
Sergey M․
67dda51722 Rename compat_urllib_request_Request to sanitized_Request and move to utils 2015-11-23 21:55:15 +06:00
Sergey M․
82d8a8b6e2 [YoutubeDL] Wrap plain-text URL requests in compat_urllib_request_Request 2015-11-23 21:55:13 +06:00
ashutosh-mishra
17cc153435 Typo fix, found while going through the code. 2015-11-20 22:51:46 +05:30
Sergey M
f270cf1a26 Merge pull request #7519 from barlik/master
Clarify that automatic subtitles are generated.
2015-11-19 22:44:08 +06:00
hedii
371c3b796c [YoutubeDL] Add playlist finished downloading message (Closes #7517)
Conflicts:
	youtube_dl/YoutubeDL.py
2015-11-19 22:39:02 +06:00
Rastislav Barlik
741dd8ea65 Clarify that automatic subtitles are generated.
It wasn't clear what automatic word mean.
2015-11-16 14:15:25 +00:00
Sergey M․
b3613d36da [YoutubeDL] Sanitize path after output template substitution (Closes #7367) 2015-11-05 04:39:21 +06:00
Yen Chi Hsuan
8b172c2e10 [YoutubeDL] Use DataHandler 2015-10-18 13:44:22 +08:00
Sergey M․
4bba371644 [YoutubeDL] Autocalculate ext for subtitles when missing 2015-10-04 20:42:26 +06:00
Sergey M․
6a3f4c3f82 [YoutubeDL] Use custom cookie processor 2015-09-06 06:21:33 +06:00
Sergey M․
4abf617b9c [YoutubeDL] Ensure bool params always present in downloader 2015-09-05 19:17:30 +06:00
Sergey M․
d3d89c3256 [YoutubeDL] Encode thumbnail filename (Closes #6714) 2015-08-31 02:01:13 +06:00
Sergey M․
671302b5c0 [YoutubeDL] Remove unused imports 2015-08-07 00:08:11 +06:00
Sergey M․
51f267d9d4 [YoutubeDL:utils] Move percent encode non-ASCII URLs workaround to http_request and simplify (Closes #6457) 2015-08-06 22:01:01 +06:00
Jaime Marquínez Ferrándiz
232541df44 [YoutubeDL] format spec: correctly handle dashes and other unused operators
'mp4-baseline-16x9' must be handled as a single string, but the '-' was treated as an operator.
2015-08-04 22:29:23 +02:00
Jaime Marquínez Ferrándiz
d96d604e53 YoutubeDL: format spec: don't accept a bare '/' (#6124) 2015-08-03 23:04:11 +02:00
Jaime Marquínez Ferrándiz
03950c90f7 Merge remote-tracking branch 'jaimemf/format_spec_groups' (closes #6124) 2015-08-03 15:22:51 +02:00
Jaime Marquínez Ferrándiz
8250c32f49 [YoutubeDL] don't default to 'bestvideo+bestaudio/best' for live videos
Doesn't work currently.
2015-07-20 20:25:53 +02:00
Sergey M․
f72b0a6032 Revert xvid to avi and make docs to be similar to existing external downloader option 2015-07-11 22:15:16 +06:00
Sergey M․
ac9ed061ec Merge branch 'master' of https://github.com/aurium/youtube-dl into aurium-master 2015-07-11 21:58:12 +06:00
Jaime Marquínez Ferrándiz
0a31a35098 [YoutubeDL] format spec: add additional checks for invalid syntax 2015-07-10 22:46:25 +02:00
Jaime Marquínez Ferrándiz
bb8e553662 [YoutubeDL] format spec: Do not fail when a filter gives an empty result
For example with 'best[height<40]' we ended getting a 'IndexError: list index out of range'.
2015-07-04 21:41:09 +02:00
Jaime Marquínez Ferrándiz
f5f4a27a96 [YoutubeDL] format spec: fix handling of '/' with ','
When using 'bestvideo/best,bestaudio', 'bestvideo/best' must be set as the current_selector (instead of appending it to the selectors), otherwise when it gets the ',' it would append 'None' to the selectors.
2015-07-04 21:30:26 +02:00
Aurélio A. Heckert
1866432db7 Rename --pp-params to --postprocessor-args and access value as super class attribute 2015-06-30 16:22:09 -03:00
Jaime Marquínez Ferrándiz
cf2ac6df68 [YoutubeDL] format spec: Fix handling of '+' with '/'
'bestvideo+bestaudio/best' was incorrectly interpreted as 'bestvideo+(bestaudio/best)', so it would fail if 'bestaudio' doesn't exist instead of falling back to 'best'.
2015-06-30 19:50:17 +02:00
Jaime Marquínez Ferrándiz
0130afb76e [YoutubeDL] format spec: allow grouping specifiers with parentheses 2015-06-29 12:46:02 +02:00
Jaime Marquínez Ferrándiz
5acfa126c8 [YoutubeDL] format spec: treat 'all' like a normal specifier
So you can use filters with it, for example 'all[width>=400][width<=600]'.
2015-06-28 22:48:02 +02:00
Jaime Marquínez Ferrándiz
67134eaba1 [YoutubeDL] rework how the format spec is processed
The spec string is processed using 'tokenize.tokenize' to split it in words and operators, the filters are still processed using regular expressions.
This should make easier to allow grouping operators with parens.
2015-06-28 22:31:35 +02:00
Sergey M․
9603e8a7d9 [YoutubeDL] Handle None width and height similarly to formats 2015-06-28 22:55:28 +06:00
Sergey M.
3c3b4176bd Merge pull request #5961 from dstftw/force-generic-extractor
Add --force-generic-extractor
2015-06-24 19:10:45 +05:00
Aurélio A. Heckert
14835de9fb Use shlex.split for --pp-params and update related docs. 2015-06-16 18:10:31 -03:00
Yen Chi Hsuan
0501bfa159 [YoutubeDL] Youku extractor now uses the standard format selection 2015-06-16 00:15:30 +08:00
Shrimadhav U K
180940e02d spelling mistake corrected
acces changed to accessing
2015-06-14 11:19:42 +05:30
Sergey M․
0072afca8e [YoutubeDL] Remove force_generic_extractor arg from process_ie_result 2015-06-13 02:21:29 +06:00
Sergey M․
61aa5ba36e [YoutubeDL] Remove global state for force_generic_extractor flag in favor of passing argument 2015-06-13 02:05:21 +06:00
Sergey M․
9f4323252a [YoutubeDL] Fix for multiple URLs 2015-06-12 21:56:50 +06:00
Sergey M․
d22dec74ff Add --force-generic-extractor
For some extractors that are hard to workout a good _VALID_URL we use very vague and unrestrictive ones,
e.g. just allowing anything after hostname and capturing part of URL as id.
If some of these extractors happen to have an video embed of some different hoster or platform
and this scenario was not handled in extractor itself we end up with inability to download this embed
until extractor is fixed to support embed of this kind.
Forcing downloader to use the generic extractor can be a neat temporary solution for this problem.
Example: FiveTV extractor with Tvigle embed - http://www.5-tv.ru/rabota/broadcasts/48/
2015-06-12 19:20:12 +06:00
Sergey M․
a55e36f48d [YoutubeDL] Handle out-of-range timestamps (#5826) 2015-06-08 21:05:17 +06:00
Sergey M․
df15ef8dab [YoutubeDL] Tweak select_format for video only media 2015-05-31 04:05:09 +06:00
Sergey M․
931bc3c3a7 [YoutubeDL] Do not loose request method information 2015-05-30 22:52:02 +06:00
Sergey M․
181c7053e3 [YoutubeDL] Make sure all formats have unique format_id 2015-05-30 16:04:44 +06:00
Sergey M․
71646e4653 [YoutubeDL] Initialize files_to_delete (Closes #5797) 2015-05-24 04:14:01 +06:00
Sergey M․
4a5a898a8f [YoutubeDL] Clarify incompatible formats merge message
When `-f` is not specified it's misleading to see `You have requested ...` as user did not actually request any formats.
2015-05-17 20:56:03 +06:00
Jaime Marquínez Ferrándiz
3884dcf313 YoutubeDL: ignore indexes from 'playlist_items' that are not in the list (fixes #5706)
We ignore them instead of failing to match the behaviour of the 'playliststart' parameter.
2015-05-15 14:08:26 +02:00
Sergey M․
82245a6de7 [YoutubeDL] Restore filename for thumbnails 2015-05-14 15:21:27 +06:00
Sergey M․
98b8ec8616 Merge branch 'best-fallback-on-outdated-avconv' of https://github.com/dstftw/youtube-dl into dstftw-best-fallback-on-outdated-avconv
Conflicts:
	youtube_dl/YoutubeDL.py
2015-05-14 15:18:58 +06:00
Yen Chi Hsuan
8e59539752 [postprocessor/embedthumbnail] Use thumbnails downloaded by YoutubeDL 2015-05-14 02:32:00 +08:00
Yen Chi Hsuan
3749e36e9f [YoutubeDL] Fix PEP8 W503 2015-05-13 21:16:45 +08:00
Sergey M․
97fcf1bbd0 [YoutubeDL] Check if merger can actually merge 2015-05-11 02:01:16 +06:00
Sergey M․
7fcb605b82 [YoutubeDL] Fallback to -f best when merger is outdated 2015-05-11 00:27:29 +06:00
Jaime Marquínez Ferrándiz
95c5534f8e ExecAfterDownloadPP, YoutubeDL: remove unused parameters 2015-05-10 17:41:11 +02:00
Sergey M․
3799834dcf [YoutubeDL] Do not force bestvideo+bestaudio when outtmpl is stdout (#5627) 2015-05-07 20:46:11 +06:00
Sergey M․
2699da8041 [YoutubeDL] Improve description file naming 2015-05-02 23:36:55 +06:00
Sergey M․
98727e123f [YoutubeDL] Improve annotations file naming 2015-05-02 23:35:18 +06:00
Sergey M․
b29e0000e6 [YoutubeDL] Improve JSON info file naming 2015-05-02 23:23:44 +06:00
Sergey M․
666a9a2b95 [YoutubeDL] Improve audio/video-only file naming 2015-05-02 23:11:34 +06:00
Sergey M․
38c6902b90 [YoutubeDL] Ensure correct extension is always present for a merged file (Closes #5535) 2015-05-02 22:52:21 +06:00
Sergey M․
cb202fd286 [YoutubeDL] Filter requested info fields on --load-info as well
In order to properly handle JSON info files generated by youtube-dl versions prior to 4070b458ec
2015-05-01 00:44:34 +06:00
Jaime Marquínez Ferrándiz
df8301fef5 [YoutubeDL] pep8: use 'k not in' instead of 'not k in' 2015-04-30 20:18:42 +02:00
Sergey M․
4070b458ec [YoutubeDL] Do not write requested info in info JSON file (Closes #5562, closes #5564) 2015-04-30 23:55:05 +06:00
Sergey M․
e01c56f9e1 [YoutubeDL] Generalize best/worst format match behavior 2015-04-30 21:06:51 +06:00
Jaime Marquínez Ferrándiz
8dd5418803 Make 'best' format only match non-DASH formats (closes #5554)
Otherwise it's impossible to only download non-DASH formats, for example `best[height=?480]/best` would download a DASH video if it's the only one with height=480, instead for falling back to the second format specifier.
For audio only urls (soundcloud, bandcamp ...), the best audio will be downloaded as before.
2015-04-29 22:53:18 +02:00
zouhair
5456d78f0c Typo "incompatible" instead of "uncompatible" 2015-04-29 10:07:49 -04:00
Jaime Marquínez Ferrándiz
9f3fa89f7c Remove the --max-quality option
It doesn't work well with 'bestvideo' and 'bestaudio' because they are usually before the max quality.
Format filters should be used instead, they are more flexible and don't require the requested quality to exist for each video.
2015-04-25 11:59:54 +02:00
Yen Chi Hsuan
8be2bdfabd [YoutubeDL] Remove the redundant assignment to old_filename
Caused by commmit 592e97e855
2015-04-22 15:05:35 +08:00
Sergey M․
6728187ac0 [YoutubeDL] mp3 is compatible with mp4 2015-04-20 21:58:46 +06:00
Sergey M․
17c8675853 [YoutubeDL] Allow bestvideo+bestaudio/best strategy for ted extractor 2015-04-20 21:58:29 +06:00
Sergey M․
c0dea0a782 [YoutubeDL] Respect explicit --merge-format-output for uncompatible formats as well 2015-04-19 22:33:52 +06:00
Sergey M․
81cd954a51 [YoutubeDL] Merge incompatible formats into mkv (#5456) 2015-04-19 17:55:42 +02:00
Sergey M․
feccf29c87 [YoutubeDL] Make bestvideo+bestaudio/best default format when merger is available 2015-04-19 17:51:56 +02:00
Jaime Marquínez Ferrándiz
5b5fbc0867 Detect already merged videos
Without the '--keep-video' option the two files would be downloaded again and even using the option, ffmpeg would be run again, which for some videos can take a long time.
We use a temporary file with ffmpeg so that the final file only exists if it success
2015-04-19 17:51:41 +02:00
Jaime Marquínez Ferrándiz
d47aeb2252 FFmpegMergerPP: use the new system for specifying which files can be delete 2015-04-18 11:52:36 +02:00
Jaime Marquínez Ferrándiz
592e97e855 Postprocessors: use a list for the files that can be deleted
We could only know if we had to delete the original file, but this system allows to specify us more files (like subtitles).
2015-04-18 11:36:42 +02:00
Jaime Marquínez Ferrándiz
ecc6bd1341 YoutubeDL.post_process: simplify keep_video handling
Since keep_video started as None we always set it to keep_video_wish unless it was None, so in the end keep_video == keep_video_wish. This should have been changed in f3ff1a3696, but I didn't notice it.
2015-04-17 22:38:14 +02:00
Sergey M․
c36a959549 [YoutubeDL] Try to download worst audio+video served by a single file first (Closes #5408) 2015-04-12 17:36:29 +06:00
Jaime Marquínez Ferrándiz
8e4b83b96b Remove check for ssl certs
When it uses a capath instead of a cafile, 'get_ca_certs' or 'cert_store_stats' only returns certificates already used in a connection.
(see #5364)
2015-04-06 22:18:08 +02:00
Sergey M․
a3c7019e06 [YoutubeDL] Check for get_ca_certs availability
`get_ca_certs` is not available in python <3.4
2015-04-02 22:50:10 +06:00