Recent Posts

Pages: 1 ... 8 9 [10]
91
Geek Speak / Download a 64 kbps Podcast at 384 kbps using FFMPEG
« Last Post by Ed on July 11, 2017, 04:47:12 PM »
Whilst listening to a "Listen Again" radio show on BBC iPlayer, I noticed that it had a button marked DOWNLOAD. This was the first time I'd seen such an option on iPlayer.

Turns out it's a fairly common feature on science programmes (I was listening to a broadcast about botany), albeit one which is completely unknown on Light Entertainment programmes. (This may tell you more about me than about the iPlayer!)

Investigating further, I discovered that it is what the BBC term a 'podcast'. They define this as being an mp3 file, recorded at 64 kbps, downloadable on demand. Probably, therefore, everything I'm about to mention will apply to all BBC podcasts.

You've opened this thread because of what you read in its title, so it won't surprise you to learn that I'm about to describe a method for violating the 64 kbps limitation, and instead downloading the podcast of interest at the standard on-demand bitrate of 384 kbps.


Software required -

FFMPEG.exe (Static build recommended)
https://ffmpeg.zeranoe.com/builds/



Procedure -

1.  Create a batch file (a plain-text file with the extension .bat or .cmd)
     (name it, say, download.bat).
2.  Put FFMPEG.exe in the same directory/folder as the batch file.
3.  Copy-and-paste the command line, below, into the batch file (for this,
     open the batch file in a text editor, e.g. Notepad, not a word processor).
4.  Then save the file (be sure to save it with the extension .bat not .txt).
5.  (Optional) Modify the file extension specified on the command line:
     change it from .mp2 to whatever you want (only mp2 gives 384 kbps output).
6.  Copy-and-paste into the batch file the url address of the mp3 podcast file
     (found by right-clicking on the podcast's "Download" button).
7.  Run the batch file.



Here is the command line for Step 3 (actually, 2 lines), which you copy-and-paste into the batch file:

Code: [Select]
SET url=

ffmpeg  -v 16  -stats  -i  %url%  output.mp2



In Step 6, when you copy-and-paste the url address into the batch file, the SET command should resemble this example (I've added colours: the part in silver is the part you insert) -

SET url=http://open.live.bbc.co.uk/mediaselector/5/redir/version/2.0/mediaset/audio-nondrm-download/proto/http/vpid/p02qjf77.mp3

Important:  Do not modify the url address. Although it is an mp3 address, and you aim to download an mp2 file, it is essential that the target url is EXACTLY the same as the iPlayer page specifies. My command line, above, handles the necessary technical jiggery-pokery.




The above method should be capable of downloading any podcast that does not use DRM (but I've never come across a DRM podcast on the BBC site). What you get for your trouble is a download of the podcast show, at 384 kbps, as a 44.1kHz mp2 file (instead of as 64 kbps 44.1kHz mp3).




Technical Note

On the 'Listen Again' service, FFMPEG fetches whichever audio stream you specify.

If the command line specifies .mp2 the program will fetch the .mp2 stream (which, on BBC iPlayer, is a 384 kbps stream); or if the command line specifies .m4a the program will fetch the .m4a stream (which, on BBC iPlayer, is a 128 kbps stream).

There are lots of different file types (e.g. .mp3, .mp2, .m4a, .mp4, .mpeg, .ts, .flac, .wav). FFmpeg will fetch a different stream (i.e. a different bitrate and/or sample rate) for each of the different file types.

The highest bitrate is for uncompressed .wav (approx 1,500 kbps). The highest bitrate I've yet come across for a compressed format is 384 kbps for the .mp2 file type.





92
Book at Bedtime, Summer Lies Episode 1 of 5

The Night in Baden-Baden,
Part 1
http://www.bbc.co.uk/programmes/b01n6263#play
329 days to listen

Bill Nighy reads from the new collection of short fiction by the author of The Reader, Bernhard Schlink.

He begins with a three-part story called The Night in Baden-Baden. An author goes to the first night of his first play, which is being performed in Baden-Baden. To celebrate the occasion he invites Therese to go with him. But Therese is not his girlfriend.

Abridged and produced by Jane Marshall
A Jane Marshall production for BBC Radio 4.

The man reads WICKEDLY well does the NIGHY man!
93
Geek Speak / Re: Screenscraping - A technique to find the vpid
« Last Post by Ed on July 09, 2017, 11:56:20 PM »
I'm running Firefox 39 as my browser on one computer, and Internet Explorer 8 on a laptop. Both computers are Windows 7 64-bit.

When I open an iplayer 'Listen Again' page, and try to play the radio show, if I'm using Firefox I'm blocked by the paywall. But if I'm using IE8, I simply don't see any paywall.

So I think that everyone will have a different experience, depending on which browser - perhaps even which version of that browser - they are using. And it seems to be the older browsers which fare best, probably because the paywall uses a modern, new type of javascript which some older browsers can't recognise.

As I say, I'm using Windows 7. It will be interesting to hear from people who are on Windows 8 or Windows 10, as to whether that makes any difference.

94
Geek Speak / Re: Screenscraping - A technique to find the vpid
« Last Post by Truthyness on July 08, 2017, 08:40:05 PM »
Gosh I haven't yet come up against a Beeb iPlayer paywall! Sounds hideous. Thanks to your posts at least now I am psyched up! 8)
95
Good Books & Readings / Re: AudioGO files for administration
« Last Post by Truthyness on July 08, 2017, 08:36:20 PM »
Timely information Ed, thank you. What of Amazon's Audible, I recall getting enmeshed. Mandatory CC, hyped up twaddle about free access, lies, deceit and entrapment ... my experience at least ... echoed by complaints across the "boards"

https://www.google.be/search?q=amazon%27s+audible+is+a+scam&ie=utf-8&oe=utf-8&client=firefox-b&gfe_rd=cr&ei=czNhWdaML4bH8AfO8bv4Dw

Sigh's to the power of infinity!
96
The Radio 4 Extra listing for the seven days beginning 08/07/17 has been posted at the following permanent link:

http://www.radio-lists.org.uk/4x

This contains the online listing and the pdf: hover the mouse over the 'pdf' button and right-click and 'save as' to download, or left-click to display and then possibly print through the pdf viewer.

The rtf version is no longer produced, but there is a printer-friendly html version under the 'print-frendly' button; download and viewing is as per the procedure above for the pdf.

An archive is being built up - see the 'archive' button. Archive versions are also available under the 'print-friendly' button.

Note that the online version now lands on the current programme; to read next week's list early, use the 'Archive' drop-down list.
97
I know this is a very old thread, but in view of the fact that in all the intervening time the BBC have been unable to get any replacement for AudioGo off the ground, perhaps we should wonder whether that may be significant. By which, of course, I mean significant for us.

It was not bad management which sank AudioGo.

That was a partnership between BBC and a private company, with all of BBC's resources behind it. Yet it failed to make a profit. Even though it had the entire BBC archive to draw on, and the very deep pockets of the Corporation.

The problem it couldn't crack is that the internet provides unlimited free access to the type of material AudioGo was selling on CD (and, to a degree, as downloads). This very website, Beebotron, is dedicated to a BBC station which provides free access to a large chunk of the archive material AudioGo was selling.

And out in the wild wild west of the www, there are inumerable websites offering free OTR, including BBC shows; plus innumerable Torrents offering file sharing of that material; plus inumerable pirate sites selling (far cheaper than AudioGo were) the same material - albeit not in as good quality, but much cheaper.

The BBC seem to have concluded from AudioGo's bankruptcy that the type of commercial operation AudioGo were involved in isn't viable.

But what are BBC doing about this problem?

All of a sudden, BBC iPlayer is vanishing behind a paywall. As yet - and I emphasise the word yet - we are not being asked for any money to access it. But that is the logical next step.

In 1971 the radio licence was abolished; and, ever since, BBC radio has been funded from the TV licence fee.  But the BBC now seem to be gearing up to seek new sources of revenue. Hiding the online access to their entire tv and radio output is a logical first step in moving to a new business model, in which online access is only available by paying a subscription - on top of the Licence Fee.

Their most recent step was to extend the law, so that watching BBC tv live on line now requires a TV Licence. Now they put up a paywall. Next, they encrypt.

It's a logical progression, triggered off by the failure of their other commercial ventures, as a means of making money out of us.



98
R4 Extra (The Station Formerly Known As R7) Matter / The simplest solution
« Last Post by Ed on July 05, 2017, 03:30:25 PM »
Having now had a couple of days to review all the download solutions, in my opinion the simplest is the following.


STEP 1: Get the vpid (an 8-digit text string)

  1. Open the iPlayer page for the radio show, in Firefox.
  2. Without playing that show, right-click on that page, and select 'View Page Source'.
  3. Search (on the toolbar: Edit > Find) the resulting text for the phrase: "vpid"

  NB: You might use Internet Explorer instead. Some versions ( e.g. Internet Explorer 8 )
        allow you to view a page's source (on the toolbar: View > Source).


STEP 2: Modify the following url (substitute the 8-digit vpid for the 8 zero's) -

http://open.live.bbc.co.uk/mediaselector/5/redir/version/2.0/vpid/00000000/mediaset/audio-syndication/proto/http


The media player Media Player Classic - Home Cinema (a.k.a "MPC-HC") can play this url, if the vpid is substituted for the 8 zero's.

  Get MPC-HC here:  https://mpc-hc.org  OR  https://mpc-hc.org/downloads/

  On Windows 7 64-bit I recommend this version:  MPC-HC.1.7.10.x64.exe


STEP 3: To download the radio show, create the following .bat file -

  a. Create a plain text file, and name it:  download.bat

  b. Open that file using NOTEPAD.EXE (or any simple text editor),
      then copy-and-paste the following 2 lines into it -

Code: [Select]
SET url=
ffmpeg  -v 16  -stats  -i %url%  "output.mp2"

  c. After url=, paste the MODIFIED url from STEP 2,
      containing the actual 8-digit vpid (in place of the 8 zero's)

  d. Save the file. Make certain you save it with the extension .bat
      (NOT .txt)

  e. Put ffmpeg.exe in the same folder as the .bat file

        Get it here:  https://ffmpeg.zeranoe.com/builds/

  f. Double-click the .bat file to run it


When you have copied the url address into the .bat file, in step 3c, that file should look like this example -

Code: [Select]
SET url=http://open.live.bbc.co.uk/mediaselector/5/redir/version/2.0/vpid/12345678/mediaset/audio-syndication/proto/http
ffmpeg  -v 16  -stats  -i %url%  "output.mp2"

Note that a HLS stream is an mpeg stream, therefore can ONLY be used to download a stream in the .mp3 or .mp2 or .mpeg or .ts format (i.e. the output file should be named output.mp3 or output.mp2 or output.mpeg or output.ts). Obviously, call it what you like; but (to avoid problems) it must have the .mp3 or .mp2 or .mpeg or .ts extension.


99
Here are some recent examples of the ffmpeg command line which downloads the audio stream (in these examples, downloading an .mp2 file from a HLS stream).

These are just examples of the correct syntax to use.

Obviously, none of these examples will actually download anything now, as the https url is only valid for a couple of hours, and all of these have expired. To use these examples again (within the 30 days that a show is available on the iplayer site), that url must be replaced with a current one.


Code: [Select]

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/3be86f-b08w933f/vf_b08w933f_036923dd-b21a-4507-bc02-6715dcf0c2ef.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498354121_b90a8c9bcae3a8263772a5fb16def086" -ss 0.7 -aframes 148900 Episode_1_The_Lost_Stories_Point_of_Entry_Doctor_Who_BBC_Radio_4_Extra[148900].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/bad0c8-b006gpbg/vf_b006gpbg_a20362d3-ed30-42cc-9d47-aac249886756.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498443493_8ed2ac2905d9fcd8db484a6f4d37754e" -ss 0.6 -aframes 71954 From_11_02_1954_Educating_Archie_BBC_Radio_4_Extra[71954].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/0f75f6-b062n92s/vf_b062n92s_25e2421c-c1c9-49d0-83e4-0b947c27a3fc.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498444824_0ed9dabe8681704da6ab5b9c1b86fae6" -ss 0.5 -aframes 67499 From_11_07_1961_Whack_O_BBC_Radio_4_Extra[67499].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/23f26f-b005dzm1/vf_b005dzm1_e021d01a-96ee-4f78-9661-038747d50dfb.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498499043_43f1483c9992577af6b7a18cdcda5fd2" -ss 0.015 -aframes 74903 The_Finger_of_Suspicion_John_Creasey_The_Toff_and_the_Runaway_Bride_BBC_Radio_4_Extra[74903].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/26c844-b00273cv/vf_b00273cv_82ef9e80-b97f-4a31-97f7-8b9a59dc64e9.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498499389_21ca7353e45b06cfca97c54193ee21cb" -ss 0.015 -aframes 69232 War_Dance_Series_2_Dad_s_Army_BBC_Radio_4_Extra[69232].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/2f2722-b005f4k3/vf_b005f4k3_b8cd9622-866a-4f97-a071-29c15bd37b90.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498579925_f6043548721e4bb6402f53ab14815b41" -ss 0.015 -aframes 74075 No_Sleep_for_the_Wicked_John_Creasey_The_Toff_and_the_Runaway_Bride_BBC_Radio_4_Extra[74075].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/a6292a-b0027b1s/vf_b0027b1s_f216dd8c-60cf-4039-a48d-3d5ed1534873.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498580914_e950bd6bfebe754945aa39e0a0036bbd" -ss 0.315 -aframes 68738 Live_Now_Paye_Later_Series_6_Steptoe_and_Son_BBC_Radio_4_Extra[68738].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/7055e0-b002427b/vf_b002427b_da6ffa73-36b1-4c70-a911-0bd031caff10.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498581705_f8cd98c500b488d006f6f8f1cfa67561" -ss 0.015 -aframes 72570 Bare_Necessities_The_Men_From_the_Ministry_BBC_Radio_4_Extra[72570].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/ee9a87-b00fy2hs/vf_b00fy2hs_f63d9701-f3cf-43f6-bb6b-d05d57a4ae4c.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498670284_9cf087e10821e9b907b10a88ba182f3d" -ss 0.1 -aframes 70073 Still_in_Trying_Times_Series_2_Revolting_People_BBC_Radio_4_Extra[70073].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/de754a-b005f9qf/vf_b005f9qf_3340920a-fab1-47bc-a948-a4f50ae0f3d1.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498671065_c474e270fcc076668b9de3fd62beca87" -ss 0.015 -aframes 73283 Death_Is_No_Alibi_John_Creasey_The_Toff_and_the_Runaway_Bride_BBC_Radio_4_Extra[73283].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/e1e2e1-b00bf7sm/vf_b00bf7sm_ae1c2d09-ea64-45a3-b7dc-a63d0df5325b.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498672200_b4138efaa0324a990b669149dfae982a" -ss 1 -aframes 77168 The_Missing_Jeep_Series_1_The_Navy_Lark_BBC_Radio_4_Extra[77168].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/a7eb2c-b0089k3l/vf_b0089k3l_618e7e3b-2511-4689-9006-6b03b6af0840.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498673234_b08c73ade0161392e857f9609830d317" -ss 0.8 -aframes 66304 The_Foreign_Legion_Series_5_Hancock_s_Half_Hour_BBC_Radio_4_Extra[66304].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/8d5ea3-b005fjj4/vf_b005fjj4_ec1a4da8-018b-42d3-b6b5-6141cddcaff9.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498755923_9240b09d998370c7cfd99a42b4a96501" -ss 0 -aframes 74260 Motive_for_Murder_John_Creasey_The_Toff_and_the_Runaway_Bride_BBC_Radio_4_Extra[74260].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/fe97fa-b04tvk93/vf_b04tvk93_a6921ada-558b-4450-8aa8-25fc546bbdab.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498757238_bbbe54ea4cdd1999c505bc94047b7076" -ss 0 -aframes 74890 The_Illness_Series_1_Marriage_Lines_BBC_Radio_4_Extra[74890].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/0cd29a-b05s7j4t/vf_b05s7j4t_17c6d9c3-b110-4394-88e8-ce928e4f4f52.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498758111_c538d9870a2c977b821267b4b6503bd1" -ss 0.015 -aframes 72783 Episode_5_Play_It_Cool_BBC_Radio_4_Extra[72783].mp2

::  Dr Who: The timeless appeal of the Time Lord’s theme tune  [www.bbc.co.uk/programmes/p01lmmj8]
ffmpeg -v 16 -stats -i "https://cp401489-vh.akamaihd.net/i/iplayerstream/secure_auth/ibroadcast_audio/2c97c59342137ec301425c3f95292c4c--audio--SHORT-MARK-AYERS-BOUNCE_flv_aac_med.mp4/master.m3u8?hdnea=st=1498740311~exp=1498761911~acl=/*2c97c59342137ec301425c3f95292c4c--audio--SHORT-MARK-AYERS-BOUNCE_flv_aac_med.mp4*~hmac=dcfb9fddce761bbb16a182d649e7e71be3e85d3395a403131b19f51d8619e7ef" Dr_Who_The_timeless_appeal_of_the_Time_Lord_s_theme_tune_Doctor_Who_BBC_Radio_4_Extra.mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/fef031-b005fsvh/vf_b005fsvh_30bf17f0-dec0-4cc0-a149-638bb8bbf663.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498845579_d972a3d872dc2d2bfa0d07f083f7e1f5" -ss 0 -aframes 73191 Confrontation_in_Paris_John_Creasey_The_Toff_and_the_Runaway_Bride_BBC_Radio_4_Extra[73191].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/8d0ec4-b00scvd8/vf_b00scvd8_bb0f16de-1089-49e5-a2cd-64dcb05a28f4.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498845729_0367660370a382568169ecb0ceb6517e" -ss 0.615 -aframes 74024 Episode_9_Series_3_I_m_Sorry_I_ll_Read_That_Again_BBC_Radio_4_Extra[74024].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/d9fb9a-b009xxqs/vf_b009xxqs_14c0eef4-6261-4539-a1ac-6c1bc07026d9.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498845897_1e8f2b3ededb62e93ba1cb722b1c4886" -ss 0.015 -aframes 73924 In_General_Practice_Doctor_at_Large_BBC_Radio_4_Extra[73924].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/dc7feb-b006bm2h/vf_b006bm2h_8c83614a-8af7-4054-a60c-13241d62d475.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498930981_04653b10d6ee167d8190829f569201ca" RL_Stevenson_Treasure_Island_BBC_Radio_4_Extra.mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/c6a440-b08wx026/vf_b08wx026_c8727fcc-f6a1-44bd-adcf-f5766667bf28.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498955654_1b3e124b9c3ca301769fb056cc05d51b" -ss 1.115 -aframes 149045 Episode_2_The_Lost_Stories_Point_of_Entry_Doctor_Who_BBC_Radio_4_Extra[149045].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/457513-b00g9h4n/vf_b00g9h4n_9da5503c-03e5-4d10-9dc9-35cb511d9c22.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1499044128_7d491eede9b7d87b6e24d60a3e663f28" -ss 1.315 -aframes 72347 From_30_01_1957_Educating_Archie_BBC_Radio_4_Extra[72347].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/f4b59b-b063j9bz/vf_b063j9bz_4e2e241c-f8f1-4a18-8f40-251febd05484.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1499045074_b5999663dfd57847fcac2f0bc058bc2b" -ss 0.515 -aframes 68141 From_18_07_1961_Whack_O_BBC_Radio_4_Extra[68141].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/a87a96-b00279s7/vf_b00279s7_bbd0cc49-3bf3-4385-be98-1d165cb92414.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1499101315_147c080b0c35b3d29cb9a64fd858b517" -ss 1.015 -aframes 68821 Mum_s_Army_Series_2_Dad_s_Army_BBC_Radio_4_Extra[68821].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/ecb1f6-b01f9t29/vf_b01f9t29_f603006f-9416-42bf-ba13-c10b202aa076.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1499102617_5950d087f40a0ccab3ac9b83c98e2678" -ss 1.015 -aframes 71458 Episode_1_North_East_of_Eden_BBC_Radio_4_Extra[71458].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/6009b8-b0027l55/vf_b0027l55_a4340db0-f8de-46d2-8c7e-340a17176bc7.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1499221714_851a7329436c04ca263c82e16ebb623f" -ss 0 -aframes 69073 Upstairs_Downstairs_Upstairs_Downstairs_Series_6_Steptoe_and_Son_BBC_Radio_4_Extra[69073].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/ad967a-b00zmxmc/vf_b00zmxmc_04a4eee4-bb3b-4b43-a953-4c99c567c072.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1499222502_d723e08c7e5b71817dee68cfc4fae1b5" -ss 0.2 -aframes 67524 Storm_in_a_Tea_Urn_The_Men_From_the_Ministry_BBC_Radio_4_Extra[67524].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/fdce3a-b005hc91/vf_b005hc91_9b4bd9b1-4f6a-4049-a447-772d3a71c223.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1499275466_6bd299693cfcec6e4a617a3c88117642" -ss 0.015 -aframes 74451 Operation_Fag_End_Series_1_The_Navy_Lark_BBC_Radio_4_Extra[74451].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/b16b8d-b005jy99/vf_b005jy99_6a28ecef-584b-4380-9856-fa7abf5786c7.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1499276437_3c5953396fbb94a876a61b121c3106f8" -ss 1.615 -aframes 74333 Sunday_Afternoon_at_Home_Series_5_Hancock_s_Half_Hour_BBC_Radio_4_Extra[74333].mp2



The easiest source of a replacement url for these is probably -

  http://www.iPlayerConverter.co.uk/convert.aspx


Looking at the first part of one of these example urls -

  https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/3be86f-b08w933f/ ...

the section highlighted in red is the vpid which the iPlayerConverter.co.uk page requires.


For instance, this form of url (from iPlayerConverter.co.uk which they call the Playlist URL) works fine as a substitute for the original https url shown in my above examples -

Code: [Select]
  http://open.live.bbc.co.uk/mediaselector/5/redir/version/2.0/vpid/b08w933f/mediaset/audio-syndication/proto/http

You can in fact simply take this url and put the substitute vpid in yourself (i.e. replacing b08w933f), and that url is then useable, without bothering to go to the iPlayerConverter.co.uk site.

One thing you need to know, though. This is a "HLS MP3 (UK only)" url, so it will probably only work if the download is using the mp3 format (i.e. the filename has either the .mp3 or .mp2 or .mpeg or .ts extension). And if you are in the UK (i.e. are accessing iplayer from an ip address physically in - or registered in - the UK, or access it from an anonymous ip address).

For non-UK listeners, try the Firefox add-on anonymoX 2.5.3.1 (compatible with Firefox 39). I can't test it, because I'm in the UK.




Slightly off-topic, for listeners outside the UK here (briefly) is a comparison of a UK stream with a non-UK stream, showing the differences between them. The differences which I think are significant I've highlighted in blue.


    Example of Akamai HLS url (UK stream) (a UK Stream for mobile phones) -

    http ://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/
    dc7feb-b006bm2h/vf_b006bm2h_8c83614a-8af7-4054-a60c-13241d62d475.ism/
    mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=
    1499036530_28808d5a86fb76d13d26039fc83f8cc5


    Example of Akamai DASH url (a non-UK stream) (designated a ww stream: worldwide) -

    http ://aod-dash-ww-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/
    03316c-b08vxtx4/vf_b08vxtx4_417a1117-ab79-43bb-bd4a-56d511bce726.ism/
    pc_hd_abr_v2_nonuk_dash_master.mpd?__gda__=
    1499120071_1e7c85c8b3a4d6fc439d8b41a92171a1




100
Geek Speak / Screenscraping - A technique to find the vpid
« Last Post by Ed on July 03, 2017, 09:29:06 PM »
When I am completely stuck, I use the following website to look up url's for the 'Listen Again' programmes on the radio iPlayer:

  http://www.iPlayerConverter.co.uk/convert.aspx

I can use ffmpeg.exe to download high quality streams from these url's (higher bitrates than I typically get from using programs such as get_iplayer, e.g. 320 kbps and 384 kbps streams).

It used to be trivial to use iPlayerConverter.co.uk because all it needed was the 8-digit PID (Programme ID) included in the Listen Again page's url.

But recently iPlayerConverter.co.uk changed over to using the vpid instead of the pid. I thought it was still trivial to get the vpid because, when you play the 'Listen Again' radio show, you can right-click on the playing window in the browser, and a pop-up window opens and displays both the PID and the VPID.

Now, however, the iPlayer has disappeared behind a pay-wall, and you can't play the show, so can't right-click on the playing window. (Yes, I know no money changes hands, but the technology being deployed against us is nonetheless what's known in the trade as a 'paywall'.)

Yes, it is still trivial to get the vpid. You can still play the show in older browsers: if you play it in Internet Explorer 8, the newer javascript on the iplayer website is ignored by the browser, which only understands older forms of javascript. So the show plays, and the player window appears, and can be right-clicked.

And in any browser, you can view the page source and do a 'find' search for "vpid" to find it.

I thought it would be better to write a short batch file routine to extract the 8-digit vpid from the page automatically, using just the ordinary PID. So here it is. The only contribution made by the user is to insert the PID (replacing the example PID shown as a row of 8 zero's).

This is a demo script, really, because it can readily be adapted to extract any desired information from a .html web page (that is, an ordinary webpage, one which does not use xml). The javascript runs in Internet Explorer, and downloads the target webpage specified, as a text string, which it then searches: using a javascript technique called Regular Expressions to specify the text to be extracted. It then displays any matching text found.


Click 'Select' to highlight the code, then copy-and-paste it into a plain text file with the file extension .bat instead of .txt (e.g. vpid.bat). Then open the .bat file (using, say, Notepad.exe), and replace the 8 zero's with the actual 8-digit PID of the target iplayer page. Then double click the .bat file to run it.



Code: [Select]
@echo off

::  *** Get the VPID : BBC Radio on-demand ***

::  ** Programme ID **
SET PID=00000000

::  -------------------------------------------------------------------------------------------------------------------------------------------  ::

::  *** User Variables ***
    SET temp_dir=C:\\Users\\dg\\AppData\\Local\\Temp

::  -------------------------------------------------------------------------------------------------------------------------------------------  ::

ECHO ^<html^>                                                                                                                    > %temp%\temp.htm
ECHO ^<head^>                                                                                                                   >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO ^<title^>Download BBC iPlayer HTML page and extract Text^</title^>                                                         >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO ^<SCRIPT language="javascript" type="text/javascript"^>                                                                    >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO // Target HTML file's URL address                                                                                          >> %temp%\temp.htm
ECHO    var url = "http://www.bbc.co.uk/programmes/%PID%" ;                                                                     >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO window.open('','_self'); // This prevents the browser window prompting before closing                                      >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO var http_request = false;                                                                                                  >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO function makeRequest(url) {  // URL = iPlayer page                                                                         >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO   http_request = false;                                                                                                    >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO   try {                                                                                                                    >> %temp%\temp.htm
ECHO     http_request = new ActiveXObject("Msxml2.XMLHTTP");                                                                    >> %temp%\temp.htm
ECHO   } catch (e) {                                                                                                            >> %temp%\temp.htm
ECHO     try {                                                                                                                  >> %temp%\temp.htm
ECHO       http_request = new ActiveXObject("Microsoft.XMLHTTP");                                                               >> %temp%\temp.htm
ECHO     } catch (e) {}                                                                                                         >> %temp%\temp.htm
ECHO   }                                                                                                                        >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO   if (!http_request) {                                                                                                     >> %temp%\temp.htm
ECHO     alert ('Giving up - Cannot create an XMLHTTP instance');                                                               >> %temp%\temp.htm
ECHO     return false;                                                                                                          >> %temp%\temp.htm
ECHO   }                                                                                                                        >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO   http_request.onreadystatechange = alertContents;                                                                         >> %temp%\temp.htm
ECHO   http_request.open('GET', url, true);                                                                                     >> %temp%\temp.htm
ECHO   http_request.send(null);                                                                                                 >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO }  // Close FUNCTION                                                                                                       >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO function alertContents() {                                                                                                 >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO   if ( http_request.readyState == 4 ) {                                                                                    >> %temp%\temp.htm
ECHO     if ( http_request.status == 200 ) {                                                                                    >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO        var fso  = new ActiveXObject("Scripting.FileSystemObject");                                                         >> %temp%\temp.htm
ECHO        var fh  = fso.CreateTextFile("%temp_dir%\\%PID%.html",true);         // Create Text File                            >> %temp%\temp.htm
ECHO        var fh1 = fso.CreateTextFile("%temp_dir%\\%PID%_vpid.txt",true);     // Create Text File                            >> %temp%\temp.htm
ECHO        var fh2 = fso.CreateTextFile("%temp_dir%\\%PID%_vpid_2.txt",true);   // Create Text File                            >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO        // Save contents of HTML page to text file as a String                                                   // line 50 >> %temp%\temp.htm
ECHO           fh.WriteLine( "<base href='http://www.bbc.co.uk'>" );             // Add BBC domain's href address               >> %temp%\temp.htm
ECHO           fh.WriteLine( http_request.responseText );                        // Add HTML contents of target webpage         >> %temp%\temp.htm
ECHO           // alert ( http_request.responseText );                                                                          >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO        // Use RegEx [Regular Expression] to find data in responseText (HTML)                                               >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO           // Input String (HTML from target URL)                                                                           >> %temp%\temp.htm
ECHO              var input = http_request.responseText;                          // Input                                      >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO           // Define the Pattern                                              // Pattern                                    >> %temp%\temp.htm
ECHO           // 8 literals + 8 chrs (mixed letters and digits) + 1 literal      // Example: "vpid":"b006bm2h"                 >> %temp%\temp.htm
ECHO              var pattern = /"vpid":"[A-Z0-9]{8}"/gi;                                                                       >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO           // Create OUTPUT string holding ALL matches                                                                      >> %temp%\temp.htm
ECHO              var str1 = input.match(pattern);                                // Match String with Pattern                  >> %temp%\temp.htm
ECHO              // alert ( str1 );                                                                                            >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO           // Save the data to a text file -                                                                                >> %temp%\temp.htm
ECHO              fh1.WriteLine( str1 );                                          // %PID%_vpid.txt                  // line 70 >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO           // Adjust the data -                                                                                             >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO              // Replace some text                                                                                          >> %temp%\temp.htm
ECHO                 var str2 = fso.OpenTextFile("%temp_dir%\\%PID%_vpid.txt");   // Open a text file                           >> %temp%\temp.htm
ECHO                 var str3 = str2.ReadAll();                                   // Read file's contents into a variable       >> %temp%\temp.htm
ECHO                 var str4 = str3.replace(/^"/g, "").replace(/^:/g, ": ");     // Replace all quotes and semicolons          >> %temp%\temp.htm
ECHO              // var str4 = str3.replace(/^"/g, "").replace(/^,/g, "\n");     // Replace all quotes and all commas          >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO              // Split the data into sections seperated by the specified symbol                                             >> %temp%\temp.htm
ECHO                 var str5 = str4.split(",");                                                                                >> %temp%\temp.htm
ECHO                 // alert ( str5[0] ) ;                                                                                     >> %temp%\temp.htm
ECHO                 // alert ( str5[1] ) ;                                                                                     >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO              // Save the adjusted data to a text file -                      // Save to %PID%_vpid_2.txt                   >> %temp%\temp.htm
ECHO                 fh2.WriteLine( "pid:  %PID%" );                                                                            >> %temp%\temp.htm
ECHO                 fh2.WriteLine( str5[0] );                                    // Save only the FIRST (discard str5[1])      >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO     } else { alert ('There was a problem with the request') };                                                             >> %temp%\temp.htm
ECHO   }                                                                                                                        >> %temp%\temp.htm
ECHO }     // Close FUNCTION                                                                                                    >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO ^</SCRIPT^>                                                                                                                >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO ^</head^>                                                                                                                  >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO ^<body onload="makeRequest(url);setTimeout('window.close()',4000)"^> ^</body^>                                             >> %temp%\temp.htm
ECHO ^</html^>                                                                                                                  >> %temp%\temp.htm

::  The function setTimeout gives TEMP.HTM enough time to access the Internet
::  before the scripts in it run (which need the data drawn from the Internet)

::  -------------------------------------------------------------------------------------------------------------------------------------------  ::

::  *** Run temp.htm ***
"C:\Program Files (x86)\Internet Explorer\IEXPLORE.EXE" %temp%\temp.htm

::  *** Open file : VPID ***
START ""  "C:\Windows\SYSTEM32\Notepad.exe" "%temp_dir%\\%PID%_vpid_2.txt"





Pages: 1 ... 8 9 [10]