A script to download 4oD

Anyone living in the UK will be familiar with the superb video on demand service run by Channel4; while it does have years worth of programmes, it doesn’t have an option to view the content without an internet connection. This is particularly frustrating as my TV doesn’t have an internet connection, although it does have support for USB mass storage devices. To work around this, I’ve written a ruby script that downloads 4oD videos to local MPEG-4 files.

While there is a flash based player hosted by Channel4, most of the material is also shared on youtube. To simplify things, I decided to use youtube-dl to actually download these videos; what my script does is to provide a command line interface to find the appropriate video file. It operates in one of three modes: listing the available programmes, listing the available episodes of a programme and actually downloading a video.

The first thing the script does is attempt to find a programme id that matches the first argument passed from the command line; these codes are generally easy to guess, usually comprising of the title of the programme in lower case with all punctuation removed. If either no id is passed, or no match is found then all the potential titles and ids are printed, these can easily be searched by piping the output to grep. A similar process is followed with the second and third arguments referencing the series and episode to download; if either one is unmatched or omitted, a list of episodes is printed.

Once an episode has been identified, the youtube id is extracted and passed to youtube-dl; this does take a while regardless of connection speed as youtube servers will only transmit files at approximately the same bitrate as the video. After this is complete, ffmpeg converts the flv into an MPEG-4 video with AAC sound packaged in an avi file; this is the particular format that my TV requires, but the parameters can easily be changed for different requirements.

4od-dl.rb:

#Written by Mike Worth
#http://www.mike-worth.com/2012/11/19/a-script-to-download-4od/
#https://github.com/MikeWorth

require 'hpricot'
require 'open-uri'

#First lookup the programme, list available if not found
begin
  showid=ARGV.first#TODO sanitise
  doc = Hpricot(open('http://www.youtube.com/show/'+showid))
  serieslinks=doc.search("[@class~='playlists-wide']").search("[@class='yt-uix-tile-link']")
  series=Hash.new
  serieslinks.each do |serieslink|
    series[serieslink.inner_text.strip.match(/Season ([0-9]*) Episodes/)[1]]=serieslink.attributes['href']#TODO what if there are playlists that we don`t get a match for
  end
rescue
  puts 'Show not found, available shows:'
  begin
    doc = Hpricot(open('http://www.youtube.com/user/4oD'))
    shows=doc.search("[@class='channel-summary-list']").search("[@class~='yt-uix-tile-link']")
    shows.each do |showlink|
      puts showlink.inner_text.strip + '	' + showlink.attributes['href'].gsub('/show/','') + "\n"
    end
  rescue
    puts 'Error fetching listings'
  end
  Kernel::exit
end

#Then the series and episode, list available if not found
begin
  class NoSeries<StandardError; end
  class NoEpisode<StandardError; end#TODO are these the best way to handle this?
  raise NoSeries if ARGV[1].nil?
  seriesid=ARGV[1]#TODO sanitise
  seriespage=Hpricot(open('http://www.youtube.com'+series[seriesid]))
  episodelinks=seriespage.search("[@class~='playlist-landing']").search("[@class~='yt-uix-tile-link']")
  raise NoEpisode if ARGV[2].nil?
  episodeid=ARGV[2].to_i#TODO sanitise
  episodehref=episodelinks[episodeid-1].attributes['href']
  episodeyid=episodehref.to_s.match(/v=([a-zA-Z0-9\-_]{11})&/)[1]
rescue
  begin
    series.sort.each do |s|
      puts 'Series '+s[0]+':'
      seriespage=Hpricot(open('http://www.youtube.com'+s[1]))
      episodelinks=seriespage.search("[@class~='playlist-landing']").search("[@class~='yt-uix-tile-link']")
      i=1
      episodelinks.each do |es|
        puts '  '+i.to_s+': '+es.inner_text.strip
        i+=1
      end
    end
  rescue
    puts 'Error fetching listings'
  end  
  Kernel::exit
end

#Now actually download it:
puts 'Downloading: ' + episodelinks[episodeid-1].inner_text.strip + ': ' + episodeyid
filename=showid.to_s + '.' + seriesid.to_s + '.' + episodeid.to_s
system('youtube-dl -o ' + filename + '.flv.tmp http://www.youtube.com/watch?v=' +episodeyid)#give the full youtube path in case yid starts with a -
system('avconv -i ' + filename + '.flv.tmp  -acodec copy -b 1024k ' + filename + '.avi')
File.delete(filename + '.flv.tmp')

13 thoughts on “A script to download 4oD

  1. richthespoof

    Hi I encountered this on Homeland – the pid code for the programme was 3445496, I tried messing with the last number and lo and behold changing it to a 4 – giving me 3445494 – enabled me to download it. I tried it with a previous episode and though it was a different last number the trick still worked.

    I guess they have versions on there where the flash is not DRM’d.

    I was getting mighty pissed of at 4od because I was sitting through all their ad’s and then the programme would refuse to play – It’s OK for homegrown programmes you can watch them on Youtube 4od – but I was stuck for stuff like Homeland.

    I’m guessing my problem is to do with using Ubuntu and DRM flash. So I’m very grateful you and the original author came up with this – Thanks!

    Reply
    1. Mike Post author

      That should have worked, but youtube slightly changed their layout; I’ve tweaked the code above and on github to reflect this. That said, their update seems to have caused problems with youtube-dl; it looks like an issue they’re working on.

      Requiring rubygems is not required in 1.9 and not good practice in 1.8; see here for more details.

      Reply
      1. hadi

        one question, should I download youtube-dl into the same folder? not sure how that works since it’s not on my gem list either.

        Reply
        1. Mike Post author

          This script invokes it with ‘$ youtube-dl’; it either needs to be installed somewhere that your shell can find it, or you need to change line 64 to explicitly point to it.

          On Ubuntu it is as simple as:
          $ sudo apt-get install youtube-dl
          $ sudo youtube-dl -U

          Reply
    1. Mike Post author

      Requiring rubygems is not required in 1.9 and not good practice in 1.8; see here for more details.

      youtube-dl is a program that is able to download videos from youtube given their id; my script handles getting the id then re-encoding the video after youtube-dl downloads it. What operating system are you running? Most linux distributions have it in their repositories.

      Reply
  2. eeos

    Hi Mike, thanks for the script, but I do not understand how you invoke it. I have tried for example ruby 4oD-dl.rb Misfits, but nothing happens ….

    Reply
  3. Maurice

    Does not work with 4od-dl grand-designs
    .
    I get this message: Show not found, available shows:
    Error fetching listings

    Reply
    1. Mike Post author

      Unfortunately channel 4 have stopped uploading their content to youtube; this means that this script won’t work any more

      Reply

Leave a Reply