TL;DR Use selenium+firefox to download mp3 and make them accessible via RSS…

I wanted to listen to an audio book…

§1. Webpage player

Worked fine, but no player controls (fast forward and rewind).

§2. Inspector+wget

Using chrome inspector, I can get the url to mp3, but plain wget didn’t work. I guess some anti-crawling mechanism was implemented, so I resorted to selenium. Since I had some experience on browser automation in a Ruby on Rails project, it seemed natural to start from there…

§3. selenium ruby binding

Very solid documentation; hello-world worked smoothly on the first try.

§4. selenium+chrome

Somehow, chrome always played the mp3 using its built-in player, instead of downloading it. I experimented with all kinds of options, but none worked… Then, I switched to firefox.

§5. selenium+firefox

The key config was media.play-stand-alone, which controls if the built-in player should be used. One can experiment toggling this option in firefox manually.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
require 'selenium-webdriver'
require 'timeout'

# https://github.com/SeleniumHQ/selenium/wiki/Ruby-Bindings

# maybe firefox/networking is slow...
client = Selenium::WebDriver::Remote::Http::Default.new
client.read_timeout = 60 * 10 # seconds

profile = Selenium::WebDriver::Firefox::Profile.new
# disable embedded player
profile['media.play-stand-alone'] = false
profile['browser.download.dir'] = "/tmp/"
profile['browser.download.folderList'] = 2
profile["browser.helperApps.neverAsk.saveToDisk"] =
  'audio/mp3,audio/mpeg,audio/mpeg3,audio/x-mpeg-3,video/mpeg,video/x-mpeg'

options = Selenium::WebDriver::Firefox::Options.new(args: ['-headless'])
# options = Selenium::WebDriver::Firefox::Options.new
options.profile = profile
driver = Selenium::WebDriver.for(:firefox, options:options, http_client:client)

# using browser inspector to find the mp3 url
original = 'http://ysting.ysxs8.com:81/官场刑侦/《白夜行》/《白夜行》001--有声小说吧[www.ysxs8.com].mp3'
from = '001'
page_template = 'http://www.ysxs8.com/yousheng/4914_%d.html'

(1..61).each do |i|
  page = page_template % i
  puts "visiting #{i} episode"
  driver.get(page)

  to = "%03d" % i
  download_url = original.gsub(from, to)

  puts "Downloading #{to}.mp3"

  begin
    # 'get' triggers the download, but never receives the response (maybe this
    # is by design), so using explicit timeout to recover the control
    Timeout::timeout(10) {
      driver.get(download_url)
    }
  rescue
    # give some time to the downloading, and throttle the request rate
    puts "sleeping for 10 minutes..."
    sleep(10 * 60)
    if Dir.glob("/tmp/*#{to}*.mp3").any? && Dir.glob("/tmp/*#{to}*.mp3.part").empty?
      puts "Done #{to}.mp3"
    else
      puts "Fail #{to}.mp3"
    end
  end
end

driver.quit

Now, all mp3 were downloaded to my pc, but how to transfer them to my iphone?

§6. Dropbox

Dropdox was installed on both my pc and iphone, but there was no easy way to make the whole folder available offline, and no fast forward or rewind support in the built-in player. Since my experience with the Podcasts app on my iphone had been mostly positive, I contemplated distributing these mp3 using RSS…

§6. Anchor

I added Anchor to my bookmarks a while ago, but never had a chance to check it out. After some click-around, I realized that its support for batch uploading was poor. IOW, one can’t upload multiple episodes in one step. Further searching brought me to…

§7. dropcaster

Dropcaster can turn a folder of mp3 into something RSS accessible, which was exactly what I need. However, since dropbox dropped the support for relative path in public folder, I couldn’t use it for hosting. Well, all hopes were not lost yet…

§8. local http server

Since this was not meant to be accessible outside my LAN, a local http server should be enough:

1
ruby -run -ehttpd . -p8000

Then add podcast by url, http://192.168.1.10:8000/index.rss. Done.

§Summary

It was hell of a journey just to play some mp3 on my iphone. No one else but me to blame for choosing the unsupported workflow.