Jump to content


Photo

Pakistani Hum TV Player;Tutorial and plugin

Started by mfaraj57, 26 Jul 2013 09:45

  • Please log in to reply
6 replies to this topic

#1 mfaraj57

  • Senior Member
  • 1,605 posts

+286
Excellent

Posted 26 July 2013 - 09:45

HowTo:scraping technique for Pakistani hum Tv

Upon request from many members how to extract dynamic video link i put this tutorial,but to mention that every website has different approach so i will mention general steps.

The tutorial include three parts
1-General guide how to extract sreaming url from website
2-programing the parsing method and building scraping script by windows python
3- coding HumTvplayer plugin

requirement
-hidownload software or any other url sniffing softare
-python GUI http://www.python.org/getit/
-google chrome

to do this you should have basic knowledge in html stucture and python
i will take pakistani hum tv as example http://www.hum.tv/

1- first we should know the structure and format of the streaming video url and w can catch the url by hidwonload software

http://lwx006.tunefi...4389651ab7b.flv
better to test the stream on enigma2 box before proceeding to next step because the format may be not acceptable by the box
,the simplist method to test the stream open the site http://www.vlc.eu.pn/index.php and put the link mentioned above and submit and open the TSmedia/user streams/user stream tester you will find the stream in the top of the list ,play it and see if working or not

2- now our target to get the stream url mentioned above for the site www.hum.tv and first we open the site and look for the source by right mouse click and search for http://lwx006.tunefi...4389651ab7b.flv but nothing found and this is expected as kind of url protection
so we start to think the stream url is stored in external file executed by the site and by help of google chrome(f12) we find url
http://tune.pk/playe...240&autoplay=no

 <iframe width="320" height="240" src="http://tune.pk/player/embed_player.php?vid=134121&folder=2013/07/10/&width=320&height=240&autoplay=no" frameborder="0" allowfullscreen scrolling="no"></iframe>


open the file and search for extension .flv but also nothing and that,s mean that the stream url is not stored in this url and by help of google chrome network(f12) we find the url containg the stream url
http://embed.tune.pk...121?autoplay=no
 <html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Embedded Player</title>
</head>

<body style="margin:0px; padding:0px">
<iframe src="http://embed.tune.pk/play/134121?autoplay=no"
width="320" height="240"
frameborder="0" allowfullscreen scrolling="no"></iframe>
</body>
</html>


now we open the url source of http://embed.tune.pk...121?autoplay=no and by search for extension .flv we find this

 sources: [

{
file: "http://lwx006.tunefiles.com/files/videos/2013/07/10/13734389651ab7b.flv",
width: "100%",
height: "100%",
label : "SD",
type : "flv",
default : true


our final stream url catched by hidownload presents in the code
now we take the stream url and test it by the vlc tester mentioned above
as we the extracted stream url by above method is not constant and will change may be after minute,hour ... so everytime you need to play the stream you should do this procedure and sure this is unpractical and by aid of python we make this is practical and easy


Re: Pakistani Hum TV Player;Tutorial and plugin #2 mfaraj57

  • Senior Member
  • 1,605 posts

+286
Excellent

Posted 26 July 2013 - 09:46

Now we have to program what we did above and this part the reader should have basic knowledge of python
i assume the reader already installed python for the mentioned links in above post now by running python gui we paste this code







import urllib,urllib2,re


from httplib import HTTP

from urlparse import urlparse

import StringIO

import httplib






baseurl='http://www.hum.tv/'


def gethtmlcontent(url):### to get the souce of website

data=None

try:

req = urllib2.Request(url)

req.add_header('User-Agent', 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3')

response = urllib2.urlopen(req)


data=response.read()


except:

return 'Download error',None


return 'none',data



def getvideopage(url,regx):

list1=[]

err='none'


try:

error,data=gethtmlcontent(url)

if not error=='none' or data is None:

return error,[]


match = re.findall(regx,data, re.M|re.I)



for href in match:



list1.append((href))




except:

err='downnload or parsing error'

list1=[]



return err,list1


mode=2


if mode==2: ##we start for here to get the final stream url

regx='<iframe width="320" height="240" src="(.*?)" frameborder="0" allowfullscreen scrolling="no"></iframe>' ##regular expression to extract first file http://tune.pk/player/embed_player.php?vid=134121&folder=2013/07/10/&width=320&height=240&autoplay=no


url='http://www.hum.tv/'


error,videopage=getvideopage(url,regx)


url=videopage[0]

print "*************************"

print 'videopage1:',url ### we get here the first which contain the second file

print "*************************"

regx='''<iframe src="(.*?)"

width="320" height="240"

frameborder="0" allowfullscreen scrolling="no"></iframe>'''## regular expression to get the second file which contain the final stream url


error,videopage=getvideopage(url,regx)


url=videopage[0]

print "*************************"

print 'videopage2',url ## second file

print "*************************"

regx=regx='file: "(.*?).flv"'### regular expression to get the final streaming url

error,videopage=getvideopage(url,regx)


url=videopage[0]

print "*************************"

print 'final video url:',url+".flv" ## we get the stream url here

print "*************************"


in executing the above code in python we get this
Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] on win32

Type "copyright", "credits" or "license()" for more information.

>>> ================================ RESTART ================================

>>>

*************************

first file: http://tune.pk/player/embed_player.php?vid=134121&folder=2013/07/10/&width=320&height=240&autoplay=no

*************************

*************************

second file: http://embed.tune.pk/play/134121?autoplay=no

*************************

*************************

final video url: http://lwx006.tunefiles.com/files/videos/2013/07/10/13734389651ab7b.flv

*************************

>>>


Re: Pakistani Hum TV Player;Tutorial and plugin #3 mfaraj57

  • Senior Member
  • 1,605 posts

+286
Excellent

Posted 26 July 2013 - 09:49

The third part how to apply this in enigma as when user press play the code above executed and get the final stream which played by movieplayer

###enigma plugin py

# source by mfaraj57

from Components.ActionMap import ActionMap
from enigma import eTimer,eServiceReference
from MoviePlayer import MoviePlayer
from Components.Label import Label
from Components.MenuList import MenuList
from Plugins.Plugin import PluginDescriptor
from Screens.Screen import Screen

import os
#####
import urllib,urllib2,re
from httplib import HTTP
import StringIO
import httplib





baseurl='http://www.hum.tv/'

def gethtmlcontent(url):
data=None
try:
req = urllib2.Request(url)
req.add_header('User-Agent', 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3')
response = urllib2.urlopen(req)

data=response.read()

except:
return 'Download error',None

return 'none',data


def getvideopage(url,regx):
list1=[]
err='none'

try:
error,data=gethtmlcontent(url)
if not error=='none' or data is None:
return error,[]

match = re.findall(regx,data, re.M|re.I)


for href in match:


list1.append((href))



except:
err='downnload or parsing error'
list1=[]


return err,list1


def getstreamurl():
regx='<iframe width="320" height="240" src="(.*?)" frameborder="0" allowfullscreen scrolling="no"></iframe>'

url='http://www.hum.tv/'

error,videopage=getvideopage(url,regx)

url=videopage[0]
print "*************************"
print 'videopage1:',url
print "*************************"
regx='''<iframe src="(.*?)"
width="320" height="240"
frameborder="0" allowfullscreen scrolling="no"></iframe>'''

error,videopage=getvideopage(url,regx)

url=videopage[0]
print "*************************"
print 'videopage2',url
print "*************************"
regx=regx='file: "(.*?).flv"'
error,videopage=getvideopage(url,regx)
url=None
url=videopage[0]
print "*************************"
print 'final video url:',url+".flv"
print "************"
if error=='none'and url is not None:
return url+".flv"
else:
return None
##########
class humstreamplayer(Screen):

skin = """
<screen name ="humstreamplayer" position="center,center" size="1280,720" backgroundColor="#080000" flags="wfNoBorder" title="Hum TV Player" >
<widget name="info" position="0,0" zPosition="4" size="600,500" font="Regular;22" foregroundColor="yellow" transparent="1" halign="center" valign="center" />


</screen>"""
def __init__(self, session, args = 0):
Screen.__init__(self, session)
self['actions'] = ActionMap([
'ColorActions',
'OkCancelActions'], {
'ok': self.close,
'cancel': self.close,
'red': self.close }, -1)

self["info"] = Label("Loading hum TV,please wait...")
self.timer = eTimer()
self.timer.callback.append(self.playvideourl)
self.timer.start(50, 1)

def playvideourl(self):
url=getstreamurl()
if url:

self["info"].setText("Press exit to return to TV channels")
sref = eServiceReference(0x1001, 0, url)
sref.setName("Live hum TV")
self.session.openWithCallback(self.close,MoviePlayer, sref,False)
else:

self["info"].setText("Sorry,unable to run hum TV")
def main(session, **kwargs):
session.open(humstreamplayer)

def Plugins(**kwargs):
return [
PluginDescriptor(name = "HumTVPlayer", description="watch live hum TV", where =PluginDescriptor.WHERE_EXTENSIONSMENU, fnc = main),
PluginDescriptor(name = "HumTVPlayer", description="watch live hum TV", where = PluginDescriptor.WHERE_PLUGINMENU, fnc=main, icon="plugin.png")
]



rro.png
5f3.png
3h6o.png

Attached Files


Re: Pakistani Hum TV Player;Tutorial and plugin #4 athoik

  • PLi® Core member
  • 8,458 posts

+327
Excellent

Posted 12 July 2014 - 18:46

Do you have a simpler method to share?
Wavefield T90: 0.8W - 1.9E - 4.8E - 13E - 16E - 19.2E - 23.5E - 26E - 33E - 39E - 42E - 45E on EMP Centauri DiseqC 16/1
Unamed: 13E Quattro - 9E Quattro on IKUSI MS-0916

Re: Pakistani Hum TV Player;Tutorial and plugin #5 Pedro_Newbie

  • Senior Member
  • 4,631 posts

+225
Excellent

Posted 12 July 2014 - 18:49

No but I thinks his only goal is increasing his postcount considering the age of the topics where he is reacting on ;)


Edited by Pedro_Newbie, 12 July 2014 - 18:51.

Re: Pakistani Hum TV Player;Tutorial and plugin #6 rizwanawan

  • Senior Member
  • 70 posts

0
Neutral

Posted 14 July 2014 - 18:00

This is for creating a Plugin for Box, How can only get a link and put in IPTV ?


DM 800 HD

VU+ Solo2

Wetek Play2

Octagon SF 8008

Re: Pakistani Hum TV Player;Tutorial and plugin #7 mfaraj57

  • Senior Member
  • 1,605 posts

+286
Excellent

Posted 15 July 2014 - 07:24

This is for creating a Plugin for Box, How can only get a link and put in IPTV ?

use one of the sniffer softwares as hidownload(google)


Edited by mfaraj57, 15 July 2014 - 07:25.

Back to [EN] Third-Party Development

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users

  1. Forums
  2. → PLi® Development
  3. → [EN] Third-Party Development