Show Posts

You can view here all posts made by this member. Note that you can only see posts made in areas to which you currently have access.


Messages - Ed

Pages: [1] 2 3 ... 8
1
This is a brief note intended to help listeners to the BBC iPlayer service, following recent changes to it that have hidden many of the audio streams that run at more than 128 kbps.



BBC iPlayer DASH audio streams

I have, with regret, abandoned the FFMPEG program entirely.

The BBC now use two types of audio stream, HLS and DASH. FFMPEG can only download HLS, but I cannot find any HLS on-demand streams that are genuinely running at more than 128 kbps (i.e. the entire 'Listen-again' service is only running at 128 kbps, even streams that are advertised as having a higher bitrate).

Solutions I've suggested in the past on Beebotron for HLS streams will still work. And the live streams can still be accessed at the 320 kbps bitrate. Even the On-demand streams can be accessed, but only at 128 kbps, not at 320 kbps.

But I would now recommend Livestreamer.exe as the best program for accessing all of those streams, not ffmpeg.

Livestreamer is here:

https://github.com/chrippa/livestreamer/releases/download/v1.12.2/livestreamer-v1.12.2-win32-setup.exe

And for anyone who wants the 320 kbps bitrate, only the DASH streams now provide that, for the On-demand streams (and the BBC seem to be moving gradually towards using only DASH). So welcome to the world of YouTube-dl.



YouTube-dl

This is a downloader program which downloads DASH streams. It runs on Windows (and other platforms), and was originally developed to download videos from YouTube.com (hence the fact that it's called YouTube Downloader), but it also works with other sites that use DASH.

Get YouTube-dl here:

  https://rg3.github.io/youtube-dl/

The Firefox script by Andrea has been rewritten from scratch for the new version (version 4), to cope with the latest changes to iPlayer. The version 4 script doesn't use FFMPEG, instead it takes advantage of YouTube-dl : a program which has DASH recording capability.

Get Andrea's script (currently v4.0.5), for Firefox, here:

  https://greasyfork.org/en/scripts/8161-bbc-iplayer-video-download


The audio streams are downloaded in their original formats. Usually this results in an .MP4 file for tv shows, and .M4A for radio shows. The file plays in GOM media player (and, in my tests, in most media players that run on Windows - except, sadly, Winamp).

The script enables you to download (i.e. record) video or audio in the DASH format, by providing the necessary command line. You also need to use youtube-dl to actually record the streams (https://rg3.github.io/youtube-dl/).


So all you need is the YouTube-dl.exe program, in order to download a DASH audio file using the new type of command line, which Andrea's script will give you (currently the script is version 4.0.5).

I use Firefox 39, with the Greasemonkey add-on. Andrea's script installs in Firefox as a Greasemonkey script, as described in my other posts on Beebotron about the iPlayer 320 kbps streams.

Here's an example of the command line that appears on the iPlayer page when you open it in Firefox. This is an episode of 'To the Manor Born' (from the radio series) -

Code: [Select]
youtube-dl -f bestaudio "http://aod-dash-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/648c91-b0059prl/vf_b0059prl_3e1c3c39-8dd1-4b21-9b3b-a3a7656fc519.ism/pc_hd_abr_v2_uk_dash_master.mpd?__gda__=1501970164_f704cf749be1d3bcabe59ff62e003d1a" -o To_the_Manor_Born_The_Rhythms_of_the_Earth
Copy-and-paste the command line from Firefox into a batch file (a .bat or .cmd file in Windows), put the batch file in the same folder as YouTube-dl.exe, then run the batch file. The file which downloads must then be given the extension .mp4 or .m4a by you.



MPEG-DASH

Dynamic Adaptive Streaming over HTTP (DASH), also known as MPEG-DASH, works by breaking the content into a sequence of small HTTP-based file segments, each segment containing a short interval of playback time of content.

MPEG-DASH should not be confused with a transport protocol — the transport protocol that MPEG-DASH uses is TCP.

It can use content encoded with any codec. The BBC currently use both HLS and MPEG-DASH.




2
I used the link which you posted, and the programme played without any problem.

I'm in the UK, and my experience might be different to listeners overseas, as a consequence of recent BBC changes to iPlayer. (Has there ever been a time when the phrase 'due to recent BBC changes to iPlayer' did not end my every sentence?)


Listeners to iPlayer should note my recent thread, here:

  https://forum.beebotron.org/index.php?topic=66093.msg66363

Your listening experience may be worse currently, due to you being intentionally blocked by some of the recent changes. My post explains some counter-measures you can take to resolve the problem.



3
Geek Speak / Download 384 kbps LIVE stream
« on: July 31, 2017, 05:03:12 PM »
Your comments are interesting, but I believe you are wrong.

I have the ability to monitor the audio stream itself, and to measure its bitrate. There are various software tools which include this capability. My tests establish that ffmpeg is actually fetching a 384 kbps bitrate stream.

Let's try the easiest test. I'm running Windows 7: for anyone using it, and interested in carrying out a test, all you need do is right-click on the Desktop task bar, then click 'start task manager'.

The 'Networking' tab in Task Manager will show the bitrate of the incoming stream: both as a visual graph, and (in the column 'Bytes Received per Interval') as a number. It's important to only have a single stream download running when you're doing this test (otherwise you'll see the combined data for all running streams).

It's quite good fun to take one of my .mp2 download methods, and run it with Task Manager open so you can watch the results. Then change the .mp2 extension to .wav and re-run the test: watch the incoming rate shoot up from 384 kbps to 1,500 kbps !

(Whatever figures you're seeing will quadruple, as will the average height of the visual display graph. Because these are 'chunked' streams, the audio is delivered in discontinuous chunks, for reliability, not as a continuous steady stream at 384 kb/sec, so what you see is an average, rising and falling, not a steady line.)

Not that I recommend downloading the uncompressed .wav stream, because it gives you a gigantic file size. But it makes a powerful demonstration of my point.


@ilainftw : I'd be interested to know what software you are using to measure the bitrate. Perhaps it's something I can run in Windows 7 ?

4
Geek Speak / Re: Download 384 kbps LIVE stream
« on: July 25, 2017, 02:25:09 PM »
Although the above described method does work, I was never happy with it, because the audio stream which it accesses is unstable. The connection rarely lasts more than 15 minutes at a stretch.

So although you can reconnect when that happens, I decided to look around for a method which gives better results. And I eventually found the url's detailed below, each of which gives a stream which, in my tests, is usually stable over several hours.


Here, then, is the new method for forcing the live stream to run at 384 kbps, instead of at the standard bitrate of 320 kbps.


Use any of these four command lines (each is actually a 2-line command line) -

Code: [Select]
SET url=http://a.files.bbci.co.uk/media/live/manifesto/audio/simulcast/hls/uk/sbr_med/ak/bbc_radio_four_extra.m3u8
ffmpeg  -v 16  -stats  -i %url%  output.mp2

Code: [Select]
SET url=http://a.files.bbci.co.uk/media/live/manifesto/audio/simulcast/hls/uk/sbr_high/ak/bbc_radio_four_extra.m3u8
ffmpeg  -v 16  -stats  -i %url%  output.mp2

Code: [Select]
SET url=http://as-hls-uk-live.akamaized.net/pool_6/live/bbc_radio_four_extra/bbc_radio_four_extra.isml/bbc_radio_four_extra-audio=320000.m3u8
ffmpeg  -v 16  -stats  -i %url%  output.mp2

Code: [Select]
SET url=http://as-hls-uk-live.akamaized.net/pool_6/live/bbc_radio_four_extra/bbc_radio_four_extra.isml/bbc_radio_four_extra-audio=320000.norewind.m3u8
ffmpeg  -v 16  -stats  -i %url%  output.mp2


To play the command line, copy-and-paste the url into, for example, VLC Media Player.

To download the live stream, copy-and-paste one of the 2-line command lines into a batch file ( e.g. download.bat ), put the batch file in the same directory as the ffmpeg.exe program, then run the batch file.

Notes on creating a batch fileCreate a Batch file






5
Geek Speak / How to force the Live Stream to use 384kbps
« on: July 25, 2017, 02:00:26 PM »
Here is a method for forcing the live stream to run at 384 kbps, instead of at the standard bitrate of 320 kbps.


Use any of these four command lines -

Code: [Select]
SET url=http://a.files.bbci.co.uk/media/live/manifesto/audio/simulcast/hls/uk/sbr_med/ak/bbc_radio_four_extra.m3u8
ffmpeg  -v 16  -stats  -i %url%  output.mp2

Code: [Select]
SET url=http://a.files.bbci.co.uk/media/live/manifesto/audio/simulcast/hls/uk/sbr_high/ak/bbc_radio_four_extra.m3u8
ffmpeg  -v 16  -stats  -i %url%  output.mp2

Code: [Select]
SET url=http://as-hls-uk-live.akamaized.net/pool_6/live/bbc_radio_four_extra/bbc_radio_four_extra.isml/bbc_radio_four_extra-audio=320000.m3u8
ffmpeg  -v 16  -stats  -i %url%  output.mp2

Code: [Select]
SET url=http://as-hls-uk-live.akamaized.net/pool_6/live/bbc_radio_four_extra/bbc_radio_four_extra.isml/bbc_radio_four_extra-audio=320000.norewind.m3u8
ffmpeg  -v 16  -stats  -i %url%  output.mp2


To play the command line, copy-and-paste the url and use it in, for example, VLC Media Player.


To download the live stream, copy-and-paste one of the 2-line command lines into a batch file ( e.g. download.bat ), put the batch file in the same directory as the ffmpeg.exe program, then run the batch file.

Notes on creating a batch fileCreate a Batch file






6
Here is a method for forcing the live stream to run at 384 kbps, instead of at the standard bitrate of 320 kbps.


Use any of these four command lines -

Code: [Select]
SET url=http://a.files.bbci.co.uk/media/live/manifesto/audio/simulcast/hls/uk/sbr_med/ak/bbc_radio_four_extra.m3u8
ffmpeg  -v 16  -stats  -i %url%  output.mp2

Code: [Select]
SET url=http://a.files.bbci.co.uk/media/live/manifesto/audio/simulcast/hls/uk/sbr_high/ak/bbc_radio_four_extra.m3u8
ffmpeg  -v 16  -stats  -i %url%  output.mp2

Code: [Select]
SET url=http://as-hls-uk-live.akamaized.net/pool_6/live/bbc_radio_four_extra/bbc_radio_four_extra.isml/bbc_radio_four_extra-audio=320000.m3u8
ffmpeg  -v 16  -stats  -i %url%  output.mp2

Code: [Select]
SET url=http://as-hls-uk-live.akamaized.net/pool_6/live/bbc_radio_four_extra/bbc_radio_four_extra.isml/bbc_radio_four_extra-audio=320000.norewind.m3u8
ffmpeg  -v 16  -stats  -i %url%  output.mp2


To play the command line, copy-and-paste the url into, for example, VLC Media Player.

To download the live stream, copy-and-paste one of the 2-line command lines into a batch file ( e.g. download.bat ), put the batch file in the same directory as the ffmpeg.exe program, then run the batch file.

Notes on creating a batch fileCreate a Batch file






7
Today I listened to The Lives of Harry Lime starring Orson Welles: an episode entitled Blackmail is a Nasty Word.

Or perhaps I should say, 'Today I listened to The Half-life of Harry Lime'.

Thanks are due to the BBC cretin who arranged to broadcast this 28 minute episode in a 19 minute timeslot.

8
I used to see this quite a lot in the old days.

Whenever the automatics find a gap in the station's schedules, the website's software thinks the station is off-air, and displays a message accordingly, until someone fills in the gap with programme details.

You'll notice that this filling-in has now happened for July 28th, but if you look at some of the dates for the following week you'll see that where no broadcast information has yet been posted the software defaults to announcing the station is off-air: naturally, because its machine-logic doesn't include artificial intelligence.

We, being real(?), "know" that the Beeb won't broadcast silence, because licence payers would object. But the machine, not being real, doesn't understand that concept: no schedule means no broadcast during that period, in terms of its machine logic.


This is actually rather an interesting exercise in the sport of what was formerly termed 'Kremlin-watching'. A practice indulged in by bored journalists during the days of the Cold War.

We observe some external indications, which we use to deduce things about (in this case) the Beeb's internal activities.

For example, I've noticed over the years - and today you noticed it too - that the station's schedules are only posted about a week in advance, even though Jon Pertwee explained in various interviews over the years that the BBC schedules are nowadays finalised about 18 months ahead of time. He used to complain bitterly about it.

Now, we "know" (i.e. we presume) that because 4 Extra is not a normal station - since it is not a production department - that its schedules are actually much easier to fix than, say, those of Radio 4 - as the latter actually has to make the programmes it airs. So we can deduce that being the Scheduler for 4 Extra is easy - all you do is nominate a list of already existing shows held in the Archives.

Thus we can suppose that the station's schedules are already finalised for the next 18 months at least.

However, nothing is posted on the website schedule until 7 days prior to its air-date. I could actually do nearly as well by consulting the Radio Times, which traditionally goes to print about a week ahead of its on-sale date. The RT's staff are given the schedules for printing (or used to be) about ten days before I pick up the magazine off the newsstand in WH Smith.

Presumably, someone releases the schedules to RT for compositing about 7 days before the on-sale day, which is about ten days before the first air-date for that week's issue; so, presumably, that same someone also delivers those schedules to the BBC's website staff for posting at that same time.

Presumably, the schedules are not posted anywhere until RT is sent to the printers, so that late breaking changes can be incorporated: big schedule changes might be made at the last minute, e.g. the Queen Mother dies suddenly, or smaller changes can be made when a celeb or BBC presenter dies, e.g. when Terry Wogan died they were able to insert a tribute programme very quickly.

It's only speculation, but that seems to be what is behind the oddly phrased announcement you saw on the schedules for next week: otherwise they could post these schedules months, maybe years, in advance.









9
Geek Speak / Convert TS files (320kbps) to MP2 (320kbps)
« on: July 11, 2017, 05:46:50 PM »
This suggestion supplements the method I originally described in this thread, which downloaded a bunch of .TS files from the iPlayer's LIVE stream (containing audio at 320 kbps), and then concatenated them into a single .TS file.

Here is a method for converting the obscure .ts format file into the popular .m4a format, by EXTRACTING the audio stream from the .ts file (which, on iPlayer's live stream, is AAC format audio), and creating a (new) .m4a file - or, optionally, an .mp4 file - containing that audio.

Doing this has one big advantage: files in .m4a (mp4 audio) format are playable in a very wide range of media players/programs (including Winamp, Media Player Classic, GOM Player, VLC Player, etc).

Also, you can add an ID3 tag (well, the iTunes equivalent at least) to an .m4a file, as that is the popular iTunes audio format. The .ts format does not allow ID3 tags to be stored in the file.



Software required -

FFMPEG.exe (Static build recommended)
https://ffmpeg.zeranoe.com/builds/



Procedure -

1.  Create a batch file (a plain-text file with the extension .bat or .cmd)
     (name it, say, convert.bat).
2.  Put FFMPEG.exe in the same directory/folder as the batch file.
3.  Copy-and-paste the command line, below, into the batch file (for this,
     open the batch file in a text editor, e.g. Notepad, not a word processor).
4.  Then save the file (be sure to save it with the extension .bat not .txt).
5.  Run the batch file (a new approach: just drag the .ts file onto the batch file).



Here is the command line for Step 3 (actually, 3 lines), which you copy-and-paste into the batch file:

Code: [Select]
SET file=%1
SET options=-hide_banner -bsf:a aac_adtstoasc -flags global_header
ffmpeg  -i %file%  -vn  -acodec copy  %options%  %file%.m4a



Put the .TS file in the same folder/directory as this batch file, then drag the .ts file onto the batch file. In the batch file, change .m4a to .mp4 if you prefer that.

The procedure performs what is known as a stream copy: i.e. it copies the audio stream in file A into file B, without modifying that stream. Because no re-encoding occurs, the copying procedure is very fast (typically 5 seconds, for a 30 MB half-hour radio show).




10
Geek Speak / Download a 64 kbps Podcast at 384 kbps using FFMPEG
« on: July 11, 2017, 04:47:12 PM »
Whilst listening to a "Listen Again" radio show on BBC iPlayer, I noticed that it had a button marked DOWNLOAD. This was the first time I'd seen such an option on iPlayer.

Turns out it's a fairly common feature on science programmes (I was listening to a broadcast about botany), albeit one which is completely unknown on Light Entertainment programmes. (This may tell you more about me than about the iPlayer!)

Investigating further, I discovered that it is what the BBC term a 'podcast'. They define this as being an mp3 file, recorded at 64 kbps, downloadable on demand. Probably, therefore, everything I'm about to mention will apply to all BBC podcasts.

You've opened this thread because of what you read in its title, so it won't surprise you to learn that I'm about to describe a method for violating the 64 kbps limitation, and instead downloading the podcast of interest at the standard on-demand bitrate of 384 kbps.


Software required -

FFMPEG.exe (Static build recommended)
https://ffmpeg.zeranoe.com/builds/



Procedure -

1.  Create a batch file (a plain-text file with the extension .bat or .cmd)
     (name it, say, download.bat).
2.  Put FFMPEG.exe in the same directory/folder as the batch file.
3.  Copy-and-paste the command line, below, into the batch file (for this,
     open the batch file in a text editor, e.g. Notepad, not a word processor).
4.  Then save the file (be sure to save it with the extension .bat not .txt).
5.  (Optional) Modify the file extension specified on the command line:
     change it from .mp2 to whatever you want (only mp2 gives 384 kbps output).
6.  Copy-and-paste into the batch file the url address of the mp3 podcast file
     (found by right-clicking on the podcast's "Download" button).
7.  Run the batch file.



Here is the command line for Step 3 (actually, 2 lines), which you copy-and-paste into the batch file:

Code: [Select]
SET url=

ffmpeg  -v 16  -stats  -i  %url%  output.mp2



In Step 6, when you copy-and-paste the url address into the batch file, the SET command should resemble this example (I've added colours: the part in silver is the part you insert) -

SET url=http://open.live.bbc.co.uk/mediaselector/5/redir/version/2.0/mediaset/audio-nondrm-download/proto/http/vpid/p02qjf77.mp3

Important:  Do not modify the url address. Although it is an mp3 address, and you aim to download an mp2 file, it is essential that the target url is EXACTLY the same as the iPlayer page specifies. My command line, above, handles the necessary technical jiggery-pokery.




The above method should be capable of downloading any podcast that does not use DRM (but I've never come across a DRM podcast on the BBC site). What you get for your trouble is a download of the podcast show, at 384 kbps, as a 44.1kHz mp2 file (instead of as 64 kbps 44.1kHz mp3).




Technical Note

On the 'Listen Again' service, FFMPEG fetches whichever audio stream you specify.

If the command line specifies .mp2 the program will fetch the .mp2 stream (which, on BBC iPlayer, is a 384 kbps stream); or if the command line specifies .m4a the program will fetch the .m4a stream (which, on BBC iPlayer, is a 128 kbps stream).

There are lots of different file types (e.g. .mp3, .mp2, .m4a, .mp4, .mpeg, .ts, .flac, .wav). FFmpeg will fetch a different stream (i.e. a different bitrate and/or sample rate) for each of the different file types.

The highest bitrate is for uncompressed .wav (approx 1,500 kbps). The highest bitrate I've yet come across for a compressed format is 384 kbps for the .mp2 file type.






11
Geek Speak / Re: Screenscraping - A technique to find the vpid
« on: July 09, 2017, 11:56:20 PM »
I'm running Firefox 39 as my browser on one computer, and Internet Explorer 8 on a laptop. Both computers are Windows 7 64-bit.

When I open an iplayer 'Listen Again' page, and try to play the radio show, if I'm using Firefox I'm blocked by the paywall. But if I'm using IE8, I simply don't see any paywall.

So I think that everyone will have a different experience, depending on which browser - perhaps even which version of that browser - they are using. And it seems to be the older browsers which fare best, probably because the paywall uses a modern, new type of javascript which some older browsers can't recognise.

As I say, I'm using Windows 7. It will be interesting to hear from people who are on Windows 8 or Windows 10, as to whether that makes any difference.


12
I know this is a very old thread, but in view of the fact that in all the intervening time the BBC have been unable to get any replacement for AudioGo off the ground, perhaps we should wonder whether that may be significant. By which, of course, I mean significant for us.

It was not bad management which sank AudioGo.

That was a partnership between BBC and a private company, with all of BBC's resources behind it. Yet it failed to make a profit. Even though it had the entire BBC archive to draw on, and the very deep pockets of the Corporation.

The problem it couldn't crack is that the internet provides unlimited free access to the type of material AudioGo was selling on CD (and, to a degree, as downloads). This very website, Beebotron, is dedicated to a BBC station which provides free access to a large chunk of the archive material AudioGo was selling.

And out in the wild wild west of the www, there are inumerable websites offering free OTR, including BBC shows; plus innumerable Torrents offering file sharing of that material; plus inumerable pirate sites selling (far cheaper than AudioGo were) the same material - albeit not in as good quality, but much cheaper.

The BBC seem to have concluded from AudioGo's bankruptcy that the type of commercial operation AudioGo were involved in isn't viable.

But what are BBC doing about this problem?

All of a sudden, BBC iPlayer is vanishing behind a paywall. As yet - and I emphasise the word yet - we are not being asked for any money to access it. But that is the logical next step.

In 1971 the radio licence was abolished; and, ever since, BBC radio has been funded from the TV licence fee.  But the BBC now seem to be gearing up to seek new sources of revenue. Hiding the online access to their entire tv and radio output is a logical first step in moving to a new business model, in which online access is only available by paying a subscription - on top of the Licence Fee.

Their most recent step was to extend the law, so that watching BBC tv live on line now requires a TV Licence. Now they put up a paywall. Next, they encrypt.

It's a logical progression, triggered off by the failure of their other commercial ventures, as a means of making money out of us.




13
Having now had a couple of days to review all the download solutions, in my opinion the simplest is the following.


STEP 1: Get the vpid (an 8-digit text string)

  1. Open the iPlayer page for the radio show, in Firefox.
  2. Without playing that show, right-click on that page, and select 'View Page Source'.
  3. Search (on the toolbar: Edit > Find) the resulting text for the phrase: "vpid"

  NB: You might use Internet Explorer instead. Some versions ( e.g. Internet Explorer 8 )
        allow you to view a page's source (on the toolbar: View > Source).


STEP 2: Modify the following url (substitute the 8-digit vpid for the 8 zero's) -

http://open.live.bbc.co.uk/mediaselector/5/redir/version/2.0/vpid/00000000/mediaset/audio-syndication/proto/http


The media player Media Player Classic - Home Cinema (a.k.a "MPC-HC") can play this url, if the vpid is substituted for the 8 zero's.

  Get MPC-HC here:  https://mpc-hc.org  OR  https://mpc-hc.org/downloads/

  On Windows 7 64-bit I recommend this version:  MPC-HC.1.7.10.x64.exe


STEP 3: To download the radio show, create the following .bat file -

  a. Create a plain text file, and name it:  download.bat

  b. Open that file using NOTEPAD.EXE (or any simple text editor),
      then copy-and-paste the following 2 lines into it -

Code: [Select]
SET url=
ffmpeg  -v 16  -stats  -i %url%  "output.mp2"

  c. After url=, paste the MODIFIED url from STEP 2,
      containing the actual 8-digit vpid (in place of the 8 zero's)

  d. Save the file. Make certain you save it with the extension .bat
      (NOT .txt)

  e. Put ffmpeg.exe in the same folder as the .bat file

        Get it here:  https://ffmpeg.zeranoe.com/builds/

  f. Double-click the .bat file to run it


When you have copied the url address into the .bat file, in step 3c, that file should look like this example -

Code: [Select]
SET url=http://open.live.bbc.co.uk/mediaselector/5/redir/version/2.0/vpid/12345678/mediaset/audio-syndication/proto/http
ffmpeg  -v 16  -stats  -i %url%  "output.mp2"

Note that a HLS stream is an mpeg stream, therefore can ONLY be used to download a stream in the .mp3 or .mp2 or .mpeg or .ts format (i.e. the output file should be named output.mp3 or output.mp2 or output.mpeg or output.ts). Obviously, call it what you like; but (to avoid problems) it must have the .mp3 or .mp2 or .mpeg or .ts extension.



14
Here are some recent examples of the ffmpeg command line which downloads the audio stream (in these examples, downloading an .mp2 file from a HLS stream).

These are just examples of the correct syntax to use.

Obviously, none of these examples will actually download anything now, as the https url is only valid for a couple of hours, and all of these have expired. To use these examples again (within the 30 days that a show is available on the iplayer site), that url must be replaced with a current one.


Code: [Select]

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/3be86f-b08w933f/vf_b08w933f_036923dd-b21a-4507-bc02-6715dcf0c2ef.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498354121_b90a8c9bcae3a8263772a5fb16def086" -ss 0.7 -aframes 148900 Episode_1_The_Lost_Stories_Point_of_Entry_Doctor_Who_BBC_Radio_4_Extra[148900].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/bad0c8-b006gpbg/vf_b006gpbg_a20362d3-ed30-42cc-9d47-aac249886756.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498443493_8ed2ac2905d9fcd8db484a6f4d37754e" -ss 0.6 -aframes 71954 From_11_02_1954_Educating_Archie_BBC_Radio_4_Extra[71954].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/0f75f6-b062n92s/vf_b062n92s_25e2421c-c1c9-49d0-83e4-0b947c27a3fc.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498444824_0ed9dabe8681704da6ab5b9c1b86fae6" -ss 0.5 -aframes 67499 From_11_07_1961_Whack_O_BBC_Radio_4_Extra[67499].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/23f26f-b005dzm1/vf_b005dzm1_e021d01a-96ee-4f78-9661-038747d50dfb.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498499043_43f1483c9992577af6b7a18cdcda5fd2" -ss 0.015 -aframes 74903 The_Finger_of_Suspicion_John_Creasey_The_Toff_and_the_Runaway_Bride_BBC_Radio_4_Extra[74903].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/26c844-b00273cv/vf_b00273cv_82ef9e80-b97f-4a31-97f7-8b9a59dc64e9.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498499389_21ca7353e45b06cfca97c54193ee21cb" -ss 0.015 -aframes 69232 War_Dance_Series_2_Dad_s_Army_BBC_Radio_4_Extra[69232].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/2f2722-b005f4k3/vf_b005f4k3_b8cd9622-866a-4f97-a071-29c15bd37b90.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498579925_f6043548721e4bb6402f53ab14815b41" -ss 0.015 -aframes 74075 No_Sleep_for_the_Wicked_John_Creasey_The_Toff_and_the_Runaway_Bride_BBC_Radio_4_Extra[74075].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/a6292a-b0027b1s/vf_b0027b1s_f216dd8c-60cf-4039-a48d-3d5ed1534873.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498580914_e950bd6bfebe754945aa39e0a0036bbd" -ss 0.315 -aframes 68738 Live_Now_Paye_Later_Series_6_Steptoe_and_Son_BBC_Radio_4_Extra[68738].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/7055e0-b002427b/vf_b002427b_da6ffa73-36b1-4c70-a911-0bd031caff10.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498581705_f8cd98c500b488d006f6f8f1cfa67561" -ss 0.015 -aframes 72570 Bare_Necessities_The_Men_From_the_Ministry_BBC_Radio_4_Extra[72570].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/ee9a87-b00fy2hs/vf_b00fy2hs_f63d9701-f3cf-43f6-bb6b-d05d57a4ae4c.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498670284_9cf087e10821e9b907b10a88ba182f3d" -ss 0.1 -aframes 70073 Still_in_Trying_Times_Series_2_Revolting_People_BBC_Radio_4_Extra[70073].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/de754a-b005f9qf/vf_b005f9qf_3340920a-fab1-47bc-a948-a4f50ae0f3d1.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498671065_c474e270fcc076668b9de3fd62beca87" -ss 0.015 -aframes 73283 Death_Is_No_Alibi_John_Creasey_The_Toff_and_the_Runaway_Bride_BBC_Radio_4_Extra[73283].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/e1e2e1-b00bf7sm/vf_b00bf7sm_ae1c2d09-ea64-45a3-b7dc-a63d0df5325b.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498672200_b4138efaa0324a990b669149dfae982a" -ss 1 -aframes 77168 The_Missing_Jeep_Series_1_The_Navy_Lark_BBC_Radio_4_Extra[77168].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/a7eb2c-b0089k3l/vf_b0089k3l_618e7e3b-2511-4689-9006-6b03b6af0840.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498673234_b08c73ade0161392e857f9609830d317" -ss 0.8 -aframes 66304 The_Foreign_Legion_Series_5_Hancock_s_Half_Hour_BBC_Radio_4_Extra[66304].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/8d5ea3-b005fjj4/vf_b005fjj4_ec1a4da8-018b-42d3-b6b5-6141cddcaff9.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498755923_9240b09d998370c7cfd99a42b4a96501" -ss 0 -aframes 74260 Motive_for_Murder_John_Creasey_The_Toff_and_the_Runaway_Bride_BBC_Radio_4_Extra[74260].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/fe97fa-b04tvk93/vf_b04tvk93_a6921ada-558b-4450-8aa8-25fc546bbdab.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498757238_bbbe54ea4cdd1999c505bc94047b7076" -ss 0 -aframes 74890 The_Illness_Series_1_Marriage_Lines_BBC_Radio_4_Extra[74890].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/0cd29a-b05s7j4t/vf_b05s7j4t_17c6d9c3-b110-4394-88e8-ce928e4f4f52.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498758111_c538d9870a2c977b821267b4b6503bd1" -ss 0.015 -aframes 72783 Episode_5_Play_It_Cool_BBC_Radio_4_Extra[72783].mp2

::  Dr Who: The timeless appeal of the Time Lord’s theme tune  [www.bbc.co.uk/programmes/p01lmmj8]
ffmpeg -v 16 -stats -i "https://cp401489-vh.akamaihd.net/i/iplayerstream/secure_auth/ibroadcast_audio/2c97c59342137ec301425c3f95292c4c--audio--SHORT-MARK-AYERS-BOUNCE_flv_aac_med.mp4/master.m3u8?hdnea=st=1498740311~exp=1498761911~acl=/*2c97c59342137ec301425c3f95292c4c--audio--SHORT-MARK-AYERS-BOUNCE_flv_aac_med.mp4*~hmac=dcfb9fddce761bbb16a182d649e7e71be3e85d3395a403131b19f51d8619e7ef" Dr_Who_The_timeless_appeal_of_the_Time_Lord_s_theme_tune_Doctor_Who_BBC_Radio_4_Extra.mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/fef031-b005fsvh/vf_b005fsvh_30bf17f0-dec0-4cc0-a149-638bb8bbf663.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498845579_d972a3d872dc2d2bfa0d07f083f7e1f5" -ss 0 -aframes 73191 Confrontation_in_Paris_John_Creasey_The_Toff_and_the_Runaway_Bride_BBC_Radio_4_Extra[73191].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/8d0ec4-b00scvd8/vf_b00scvd8_bb0f16de-1089-49e5-a2cd-64dcb05a28f4.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498845729_0367660370a382568169ecb0ceb6517e" -ss 0.615 -aframes 74024 Episode_9_Series_3_I_m_Sorry_I_ll_Read_That_Again_BBC_Radio_4_Extra[74024].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/d9fb9a-b009xxqs/vf_b009xxqs_14c0eef4-6261-4539-a1ac-6c1bc07026d9.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498845897_1e8f2b3ededb62e93ba1cb722b1c4886" -ss 0.015 -aframes 73924 In_General_Practice_Doctor_at_Large_BBC_Radio_4_Extra[73924].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/dc7feb-b006bm2h/vf_b006bm2h_8c83614a-8af7-4054-a60c-13241d62d475.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498930981_04653b10d6ee167d8190829f569201ca" RL_Stevenson_Treasure_Island_BBC_Radio_4_Extra.mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/c6a440-b08wx026/vf_b08wx026_c8727fcc-f6a1-44bd-adcf-f5766667bf28.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1498955654_1b3e124b9c3ca301769fb056cc05d51b" -ss 1.115 -aframes 149045 Episode_2_The_Lost_Stories_Point_of_Entry_Doctor_Who_BBC_Radio_4_Extra[149045].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/457513-b00g9h4n/vf_b00g9h4n_9da5503c-03e5-4d10-9dc9-35cb511d9c22.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1499044128_7d491eede9b7d87b6e24d60a3e663f28" -ss 1.315 -aframes 72347 From_30_01_1957_Educating_Archie_BBC_Radio_4_Extra[72347].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/f4b59b-b063j9bz/vf_b063j9bz_4e2e241c-f8f1-4a18-8f40-251febd05484.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1499045074_b5999663dfd57847fcac2f0bc058bc2b" -ss 0.515 -aframes 68141 From_18_07_1961_Whack_O_BBC_Radio_4_Extra[68141].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/a87a96-b00279s7/vf_b00279s7_bbd0cc49-3bf3-4385-be98-1d165cb92414.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1499101315_147c080b0c35b3d29cb9a64fd858b517" -ss 1.015 -aframes 68821 Mum_s_Army_Series_2_Dad_s_Army_BBC_Radio_4_Extra[68821].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/ecb1f6-b01f9t29/vf_b01f9t29_f603006f-9416-42bf-ba13-c10b202aa076.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1499102617_5950d087f40a0ccab3ac9b83c98e2678" -ss 1.015 -aframes 71458 Episode_1_North_East_of_Eden_BBC_Radio_4_Extra[71458].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/6009b8-b0027l55/vf_b0027l55_a4340db0-f8de-46d2-8c7e-340a17176bc7.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1499221714_851a7329436c04ca263c82e16ebb623f" -ss 0 -aframes 69073 Upstairs_Downstairs_Upstairs_Downstairs_Series_6_Steptoe_and_Son_BBC_Radio_4_Extra[69073].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/ad967a-b00zmxmc/vf_b00zmxmc_04a4eee4-bb3b-4b43-a953-4c99c567c072.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1499222502_d723e08c7e5b71817dee68cfc4fae1b5" -ss 0.2 -aframes 67524 Storm_in_a_Tea_Urn_The_Men_From_the_Ministry_BBC_Radio_4_Extra[67524].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/fdce3a-b005hc91/vf_b005hc91_9b4bd9b1-4f6a-4049-a447-772d3a71c223.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1499275466_6bd299693cfcec6e4a617a3c88117642" -ss 0.015 -aframes 74451 Operation_Fag_End_Series_1_The_Navy_Lark_BBC_Radio_4_Extra[74451].mp2

ffmpeg -v 16 -stats -i "https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/b16b8d-b005jy99/vf_b005jy99_6a28ecef-584b-4380-9856-fa7abf5786c7.ism/mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=1499276437_3c5953396fbb94a876a61b121c3106f8" -ss 1.615 -aframes 74333 Sunday_Afternoon_at_Home_Series_5_Hancock_s_Half_Hour_BBC_Radio_4_Extra[74333].mp2



The easiest source of a replacement url for these is probably -

  http://www.iPlayerConverter.co.uk/convert.aspx


Looking at the first part of one of these example urls -

  https://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/3be86f-b08w933f/ ...

the section highlighted in red is the vpid which the iPlayerConverter.co.uk page requires.


For instance, this form of url (from iPlayerConverter.co.uk which they call the Playlist URL) works fine as a substitute for the original https url shown in my above examples -

Code: [Select]
  http://open.live.bbc.co.uk/mediaselector/5/redir/version/2.0/vpid/b08w933f/mediaset/audio-syndication/proto/http

You can in fact simply take this url and put the substitute vpid in yourself (i.e. replacing b08w933f), and that url is then useable, without bothering to go to the iPlayerConverter.co.uk site.

One thing you need to know, though. This is a "HLS MP3 (UK only)" url, so it will probably only work if the download is using the mp3 format (i.e. the filename has either the .mp3 or .mp2 or .mpeg or .ts extension). And if you are in the UK (i.e. are accessing iplayer from an ip address physically in - or registered in - the UK, or access it from an anonymous ip address).

For non-UK listeners, try the Firefox add-on anonymoX 2.5.3.1 (compatible with Firefox 39). I can't test it, because I'm in the UK.




Slightly off-topic, for listeners outside the UK here (briefly) is a comparison of a UK stream with a non-UK stream, showing the differences between them. The differences which I think are significant I've highlighted in blue.


    Example of Akamai HLS url (UK stream) (a UK Stream for mobile phones) -

    http ://aod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/
    dc7feb-b006bm2h/vf_b006bm2h_8c83614a-8af7-4054-a60c-13241d62d475.ism/
    mobile_wifi_main_sd_abr_v2_uk_hls_master.m3u8?__gda__=
    1499036530_28808d5a86fb76d13d26039fc83f8cc5


    Example of Akamai DASH url (a non-UK stream) (designated a ww stream: worldwide) -

    http ://aod-dash-ww-live.akamaized.net/usp/auth/vod/piff_abr_full_audio/
    03316c-b08vxtx4/vf_b08vxtx4_417a1117-ab79-43bb-bd4a-56d511bce726.ism/
    pc_hd_abr_v2_nonuk_dash_master.mpd?__gda__=
    1499120071_1e7c85c8b3a4d6fc439d8b41a92171a1





15
Geek Speak / Screenscraping - A technique to find the vpid
« on: July 03, 2017, 09:29:06 PM »
When I am completely stuck, I use the following website to look up url's for the 'Listen Again' programmes on the radio iPlayer:

  http://www.iPlayerConverter.co.uk/convert.aspx

I can use ffmpeg.exe to download high quality streams from these url's (higher bitrates than I typically get from using programs such as get_iplayer, e.g. 320 kbps and 384 kbps streams).

It used to be trivial to use iPlayerConverter.co.uk because all it needed was the 8-digit PID (Programme ID) included in the Listen Again page's url.

But recently iPlayerConverter.co.uk changed over to using the vpid instead of the pid. I thought it was still trivial to get the vpid because, when you play the 'Listen Again' radio show, you can right-click on the playing window in the browser, and a pop-up window opens and displays both the PID and the VPID.

Now, however, the iPlayer has disappeared behind a pay-wall, and you can't play the show, so can't right-click on the playing window. (Yes, I know no money changes hands, but the technology being deployed against us is nonetheless what's known in the trade as a 'paywall'.)

Yes, it is still trivial to get the vpid. You can still play the show in older browsers: if you play it in Internet Explorer 8, the newer javascript on the iplayer website is ignored by the browser, which only understands older forms of javascript. So the show plays, and the player window appears, and can be right-clicked.

And in any browser, you can view the page source and do a 'find' search for "vpid" to find it.

I thought it would be better to write a short batch file routine to extract the 8-digit vpid from the page automatically, using just the ordinary PID. So here it is. The only contribution made by the user is to insert the PID (replacing the example PID shown as a row of 8 zero's).

This is a demo script, really, because it can readily be adapted to extract any desired information from a .html web page (that is, an ordinary webpage, one which does not use xml). The javascript runs in Internet Explorer, and downloads the target webpage specified, as a text string, which it then searches: using a javascript technique called Regular Expressions to specify the text to be extracted. It then displays any matching text found.


Click 'Select' to highlight the code, then copy-and-paste it into a plain text file with the file extension .bat instead of .txt (e.g. vpid.bat). Then open the .bat file (using, say, Notepad.exe), and replace the 8 zero's with the actual 8-digit PID of the target iplayer page. Then double click the .bat file to run it.



Code: [Select]
@echo off

::  *** Get the VPID : BBC Radio on-demand ***

::  ** Programme ID **
SET PID=00000000

::  -------------------------------------------------------------------------------------------------------------------------------------------  ::

::  *** User Variables ***
    SET temp_dir=C:\\Users\\dg\\AppData\\Local\\Temp

::  -------------------------------------------------------------------------------------------------------------------------------------------  ::

ECHO ^<html^>                                                                                                                    > %temp%\temp.htm
ECHO ^<head^>                                                                                                                   >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO ^<title^>Download BBC iPlayer HTML page and extract Text^</title^>                                                         >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO ^<SCRIPT language="javascript" type="text/javascript"^>                                                                    >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO // Target HTML file's URL address                                                                                          >> %temp%\temp.htm
ECHO    var url = "http://www.bbc.co.uk/programmes/%PID%" ;                                                                     >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO window.open('','_self'); // This prevents the browser window prompting before closing                                      >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO var http_request = false;                                                                                                  >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO function makeRequest(url) {  // URL = iPlayer page                                                                         >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO   http_request = false;                                                                                                    >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO   try {                                                                                                                    >> %temp%\temp.htm
ECHO     http_request = new ActiveXObject("Msxml2.XMLHTTP");                                                                    >> %temp%\temp.htm
ECHO   } catch (e) {                                                                                                            >> %temp%\temp.htm
ECHO     try {                                                                                                                  >> %temp%\temp.htm
ECHO       http_request = new ActiveXObject("Microsoft.XMLHTTP");                                                               >> %temp%\temp.htm
ECHO     } catch (e) {}                                                                                                         >> %temp%\temp.htm
ECHO   }                                                                                                                        >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO   if (!http_request) {                                                                                                     >> %temp%\temp.htm
ECHO     alert ('Giving up - Cannot create an XMLHTTP instance');                                                               >> %temp%\temp.htm
ECHO     return false;                                                                                                          >> %temp%\temp.htm
ECHO   }                                                                                                                        >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO   http_request.onreadystatechange = alertContents;                                                                         >> %temp%\temp.htm
ECHO   http_request.open('GET', url, true);                                                                                     >> %temp%\temp.htm
ECHO   http_request.send(null);                                                                                                 >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO }  // Close FUNCTION                                                                                                       >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO function alertContents() {                                                                                                 >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO   if ( http_request.readyState == 4 ) {                                                                                    >> %temp%\temp.htm
ECHO     if ( http_request.status == 200 ) {                                                                                    >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO        var fso  = new ActiveXObject("Scripting.FileSystemObject");                                                         >> %temp%\temp.htm
ECHO        var fh  = fso.CreateTextFile("%temp_dir%\\%PID%.html",true);         // Create Text File                            >> %temp%\temp.htm
ECHO        var fh1 = fso.CreateTextFile("%temp_dir%\\%PID%_vpid.txt",true);     // Create Text File                            >> %temp%\temp.htm
ECHO        var fh2 = fso.CreateTextFile("%temp_dir%\\%PID%_vpid_2.txt",true);   // Create Text File                            >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO        // Save contents of HTML page to text file as a String                                                   // line 50 >> %temp%\temp.htm
ECHO           fh.WriteLine( "<base href='http://www.bbc.co.uk'>" );             // Add BBC domain's href address               >> %temp%\temp.htm
ECHO           fh.WriteLine( http_request.responseText );                        // Add HTML contents of target webpage         >> %temp%\temp.htm
ECHO           // alert ( http_request.responseText );                                                                          >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO        // Use RegEx [Regular Expression] to find data in responseText (HTML)                                               >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO           // Input String (HTML from target URL)                                                                           >> %temp%\temp.htm
ECHO              var input = http_request.responseText;                          // Input                                      >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO           // Define the Pattern                                              // Pattern                                    >> %temp%\temp.htm
ECHO           // 8 literals + 8 chrs (mixed letters and digits) + 1 literal      // Example: "vpid":"b006bm2h"                 >> %temp%\temp.htm
ECHO              var pattern = /"vpid":"[A-Z0-9]{8}"/gi;                                                                       >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO           // Create OUTPUT string holding ALL matches                                                                      >> %temp%\temp.htm
ECHO              var str1 = input.match(pattern);                                // Match String with Pattern                  >> %temp%\temp.htm
ECHO              // alert ( str1 );                                                                                            >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO           // Save the data to a text file -                                                                                >> %temp%\temp.htm
ECHO              fh1.WriteLine( str1 );                                          // %PID%_vpid.txt                  // line 70 >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO           // Adjust the data -                                                                                             >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO              // Replace some text                                                                                          >> %temp%\temp.htm
ECHO                 var str2 = fso.OpenTextFile("%temp_dir%\\%PID%_vpid.txt");   // Open a text file                           >> %temp%\temp.htm
ECHO                 var str3 = str2.ReadAll();                                   // Read file's contents into a variable       >> %temp%\temp.htm
ECHO                 var str4 = str3.replace(/^"/g, "").replace(/^:/g, ": ");     // Replace all quotes and semicolons          >> %temp%\temp.htm
ECHO              // var str4 = str3.replace(/^"/g, "").replace(/^,/g, "\n");     // Replace all quotes and all commas          >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO              // Split the data into sections seperated by the specified symbol                                             >> %temp%\temp.htm
ECHO                 var str5 = str4.split(",");                                                                                >> %temp%\temp.htm
ECHO                 // alert ( str5[0] ) ;                                                                                     >> %temp%\temp.htm
ECHO                 // alert ( str5[1] ) ;                                                                                     >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO              // Save the adjusted data to a text file -                      // Save to %PID%_vpid_2.txt                   >> %temp%\temp.htm
ECHO                 fh2.WriteLine( "pid:  %PID%" );                                                                            >> %temp%\temp.htm
ECHO                 fh2.WriteLine( str5[0] );                                    // Save only the FIRST (discard str5[1])      >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO     } else { alert ('There was a problem with the request') };                                                             >> %temp%\temp.htm
ECHO   }                                                                                                                        >> %temp%\temp.htm
ECHO }     // Close FUNCTION                                                                                                    >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO ^</SCRIPT^>                                                                                                                >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO ^</head^>                                                                                                                  >> %temp%\temp.htm
ECHO.                                                                                                                           >> %temp%\temp.htm
ECHO ^<body onload="makeRequest(url);setTimeout('window.close()',4000)"^> ^</body^>                                             >> %temp%\temp.htm
ECHO ^</html^>                                                                                                                  >> %temp%\temp.htm

::  The function setTimeout gives TEMP.HTM enough time to access the Internet
::  before the scripts in it run (which need the data drawn from the Internet)

::  -------------------------------------------------------------------------------------------------------------------------------------------  ::

::  *** Run temp.htm ***
"C:\Program Files (x86)\Internet Explorer\IEXPLORE.EXE" %temp%\temp.htm

::  *** Open file : VPID ***
START ""  "C:\Windows\SYSTEM32\Notepad.exe" "%temp_dir%\\%PID%_vpid_2.txt"






Pages: [1] 2 3 ... 8