pyTivo Discussion Forum Forum Index pyTivo Discussion Forum
Answers and the development of pyTivo a TiVo transcoding server
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Offset exception with compatible file

 
Post new topic   Reply to topic    pyTivo Discussion Forum Forum Index -> pyTivo
 View previous topic :: View next topic  
Author Message
Iluvatar



Joined: 29 Feb 2008
Posts: 335

PostPosted: Tue Feb 28, 2012 9:18 pm    Post subject: Offset exception with compatible file Reply with quote

I was looking into adding WMV remuxing w/ audio transcoding for VC1 files and the first sample I ran into is causing an exception with 'Offset' returning 'NoneType'. It looks like the 'Range' header does not exist for the request causing a type error during the 'size - offset' operation.

handler is only returning:
Code:

2/28/12 4:07:09.646 PM pyTivo: Host: 192.168.1.100:9032
2/28/12 4:07:09.646 PM pyTivo: User-Agent: TvHttpClient
2/28/12 4:07:09.646 PM pyTivo: tsn: 746xxxxxxxx
2/28/12 4:07:09.646 PM pyTivo: Connection: close
2/28/12 4:07:09.646 PM pyTivo: TiVo_SW_VER: 20.xxxxxxxx


Anyone have insight into this? I can get other samples of course but I wanted to learn what why this is happening first.

The sample has video track only. I adjusted the tivo_compatible_audio function to pass through files where aCodec is None (which seems like it should be okay in all cases.....TiVo has handled other files just fine in testing it (corner case I know)).

Code:

2/28/12 2:11:04.156 PM pyTivo: DEBUG:pyTivo.video.transcode:CONTAINER IS: asf and MIME IS: video/bif
2/28/12 2:11:04.156 PM pyTivo: DEBUG:pyTivo.video.transcode:TRANSCODE=NO, all compatible, /Volumes/Media/TiVo/Test Vids/VC1-NOAUDIO.wmv
2/28/12 2:11:04.156 PM pyTivo: DEBUG:pyTivo.video.video:OFFSET IN VIDEO.PY IS: None
2/28/12 2:11:04.156 PM pyTivo: DEBUG:pyTivo.video.video:SIZE IN VIDEO.PY IS: 43685583
2/28/12 2:11:04.156 PM pyTivo: INFO:pyTivo:192.168.1.110 [28/Feb/2012 14:11:04] "GET /Macbook/Test%20Vids/VC1-NOAUDIO.wmv?Format=video%2Fbif HTTP/1.1" 200 -
2/28/12 2:11:04.157 PM pyTivo: ERROR:pyTivo:Exception during request from ('192.168.1.110', 40672)
2/28/12 2:11:04.157 PM pyTivo: Traceback (most recent call last):
2/28/12 2:11:04.157 PM pyTivo:   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 582, in process_request_thread
2/28/12 2:11:04.157 PM pyTivo:     self.finish_request(request, client_address)
2/28/12 2:11:04.157 PM pyTivo:   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 323, in finish_request
2/28/12 2:11:04.157 PM pyTivo:     self.RequestHandlerClass(request, client_address, self)
2/28/12 2:11:04.157 PM pyTivo:   File "/Applications/pyTivo/httpserver.py", line 75, in __init__
2/28/12 2:11:04.157 PM pyTivo:     client_address, server)
2/28/12 2:11:04.157 PM pyTivo:   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 639, in __init__
2/28/12 2:11:04.157 PM pyTivo:     self.handle()
2/28/12 2:11:04.157 PM pyTivo:   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/BaseHTTPServer.py", line 337, in handle
2/28/12 2:11:04.157 PM pyTivo:     self.handle_one_request()
2/28/12 2:11:04.157 PM pyTivo:   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/BaseHTTPServer.py", line 325, in handle_one_request
2/28/12 2:11:04.157 PM pyTivo:     method()
2/28/12 2:11:04.157 PM pyTivo:   File "/Applications/pyTivo/httpserver.py", line 106, in do_GET
2/28/12 2:11:04.157 PM pyTivo:     self.handle_file(query, splitpath)
2/28/12 2:11:04.157 PM pyTivo:   File "/Applications/pyTivo/httpserver.py", line 181, in handle_file
2/28/12 2:11:04.157 PM pyTivo:     plugin.send_file(self, path, query)
2/28/12 2:11:04.157 PM pyTivo:   File "/Applications/pyTivo/plugins/video/video.py", line 128, in send_file
2/28/12 2:11:04.157 PM pyTivo:     handler.send_header('Content-Length', size - offset)
2/28/12 2:11:04.157 PM pyTivo: TypeError: unsupported operand type(s) for -: 'int' and 'NoneType'


extra bits is just me trying to debug it some.

_________________
My pyTivo fork - Read link for changes
FFmpeg for OS X
Back to top
View user's profile Send private message
wmcbrine



Joined: 04 Jan 2008
Posts: 2009
Location: Maryland

PostPosted: Tue Feb 28, 2012 10:46 pm    Post subject: Reply with quote

Well you'd have to ask TiVo that, but if you want to get around it, something like this would be in order:

offset = handler.headers.getheader('Range', 'bytes 0-')

I'll put this change in my repo.

When I first added VC1 support, the request would fail (i.e. the TiVo would drop the connection) about every other time, on the same file. Fortunately the TiVo would repeat the request automatically, and on the second or third try, it would work, as mysteriously as it had failed.

The TiVo has always been much pickier about VC1 than h264, and I don't have anywhere near as much VC1 material to test with, so this feature hasn't received a lot of attention from me. I'm not surprised it could break without my noticing. Smile Maybe in 20.2? Earlier? Who knows.

_________________
My pyTivo fork . My page
Back to top
View user's profile Send private message Visit poster's website
Iluvatar



Joined: 29 Feb 2008
Posts: 335

PostPosted: Wed Feb 29, 2012 12:25 am    Post subject: Reply with quote

wmcbrine wrote:
Well you'd have to ask TiVo that, but if you want to get around it, something like this would be in order:

offset = handler.headers.getheader('Range', 'bytes 0-')

I'll put this change in my repo.

When I first added VC1 support, the request would fail (i.e. the TiVo would drop the connection) about every other time, on the same file. Fortunately the TiVo would repeat the request automatically, and on the second or third try, it would work, as mysteriously as it had failed.

The TiVo has always been much pickier about VC1 than h264, and I don't have anywhere near as much VC1 material to test with, so this feature hasn't received a lot of attention from me. I'm not surprised it could break without my noticing. Smile Maybe in 20.2? Earlier? Who knows.


Thanks. The HTTP code is def not something I even remotely understand. Think I'll steer clear Smile

I am beginning to think this is just a futile waste of effort. I can't even find a fully proper VC1 sample and FFmpeg can't encode one, only stream copy. I had hoped eventually to create a generic remux function based on codec/mime and the appropriate native containers.

Being as there are so many variations of VC1 and TiVo is so picky it would seem easier just to fully drop support.

_________________
My pyTivo fork - Read link for changes
FFmpeg for OS X
Back to top
View user's profile Send private message
wmcbrine



Joined: 04 Jan 2008
Posts: 2009
Location: Maryland

PostPosted: Wed Feb 29, 2012 12:29 am    Post subject: Reply with quote

BTW, as I look at it now, the pyTivo code was already supposed to deal with an absent Range header, but at some point, there was a change where it started assuming that "offset" would always be a numeric value -- albeit, usually 0 -- instead of None, like it would be in the absence of the Range header. I've fixed the setting of offset to match this assumption.
_________________
My pyTivo fork . My page
Back to top
View user's profile Send private message Visit poster's website
Iluvatar



Joined: 29 Feb 2008
Posts: 335

PostPosted: Wed Feb 29, 2012 12:36 am    Post subject: Reply with quote

wmcbrine wrote:
BTW, as I look at it now, the pyTivo code was already supposed to deal with an absent Range header, but at some point, there was a change where it started assuming that "offset" would always be a numeric value -- albeit, usually 0 -- instead of None, like it would be in the absence of the Range header. I've fixed the setting of offset to match this assumption.


I see. Your second edit is actually how I bypassed it but lacking the understanding I felt it was just a hack. Glad it was the proper way. Thanks for your help.

_________________
My pyTivo fork - Read link for changes
FFmpeg for OS X
Back to top
View user's profile Send private message
txporter



Joined: 11 Dec 2008
Posts: 263
Location: Austin, TX

PostPosted: Wed Feb 29, 2012 5:47 pm    Post subject: Reply with quote

Iluvatar wrote:
I am beginning to think this is just a futile waste of effort. I can't even find a fully proper VC1 sample and FFmpeg can't encode one, only stream copy. I had hoped eventually to create a generic remux function based on codec/mime and the appropriate native containers.

Being as there are so many variations of VC1 and TiVo is so picky it would seem easier just to fully drop support.


I did some work a year or so trying to find a workable method of transcoding to VC-1 or WMV3 using Windows Media Encoder or Windows Expression Encoder. It was just too difficult to figure out how to get good output video, especially compared with the solutions available using x264 (and it was much slower than x264 encoding). Good output, that is, that played well with Tivo. If I remember correctly, Tivo accepted (or play back well) an even narrower range of framerate than it did H.264. Maybe only 29.97fps and 59.94fps. I came back to it twice and ended up dropping it every time.
Back to top
View user's profile Send private message
Iluvatar



Joined: 29 Feb 2008
Posts: 335

PostPosted: Wed Feb 29, 2012 6:20 pm    Post subject: Reply with quote

txporter wrote:
Iluvatar wrote:
I am beginning to think this is just a futile waste of effort. I can't even find a fully proper VC1 sample and FFmpeg can't encode one, only stream copy. I had hoped eventually to create a generic remux function based on codec/mime and the appropriate native containers.

Being as there are so many variations of VC1 and TiVo is so picky it would seem easier just to fully drop support.


I did some work a year or so trying to find a workable method of transcoding to VC-1 or WMV3 using Windows Media Encoder or Windows Expression Encoder. It was just too difficult to figure out how to get good output video, especially compared with the solutions available using x264 (and it was much slower than x264 encoding). Good output, that is, that played well with Tivo. If I remember correctly, Tivo accepted (or play back well) an even narrower range of framerate than it did H.264. Maybe only 29.97fps and 59.94fps. I came back to it twice and ended up dropping it every time.


Wonder why they even bothered. Was this for Netflix or Amazon originally?

For right now it's gone on the back burner. I can't find a sample that works even with proper codecs/container. It may be the framerate like you say. Are you aware of any decent VC1 encoders on OS X? If I can narrow it down I can at least improve the compat tests.

_________________
My pyTivo fork - Read link for changes
FFmpeg for OS X
Back to top
View user's profile Send private message
txporter



Joined: 11 Dec 2008
Posts: 263
Location: Austin, TX

PostPosted: Wed Feb 29, 2012 6:30 pm    Post subject: Reply with quote

Unfortunately, those are the only encoders that I am aware of.
Back to top
View user's profile Send private message
wmcbrine



Joined: 04 Jan 2008
Posts: 2009
Location: Maryland

PostPosted: Wed Feb 29, 2012 6:45 pm    Post subject: Reply with quote

Yes, I think both Netflix AND Amazon use it. Definitely Amazon does (or did).
_________________
My pyTivo fork . My page
Back to top
View user's profile Send private message Visit poster's website
txporter



Joined: 11 Dec 2008
Posts: 263
Location: Austin, TX

PostPosted: Wed Feb 29, 2012 6:57 pm    Post subject: Reply with quote

According to the wiki on Amazon Instant Video, Amazon does indeed use VC-1. I thought they had moved to H.264 but apparently not.

On the quick google search that I just did, it appears that Netflix also uses VC-1 (albeit many of the hits were from 2007-2009 time frame). I wonder why both companies decided to go with VC-1 rather than H.264?
Back to top
View user's profile Send private message
Iluvatar



Joined: 29 Feb 2008
Posts: 335

PostPosted: Thu Mar 01, 2012 5:16 am    Post subject: Reply with quote

txporter wrote:
According to the wiki on Amazon Instant Video, Amazon does indeed use VC-1. I thought they had moved to H.264 but apparently not.

On the quick google search that I just did, it appears that Netflix also uses VC-1 (albeit many of the hits were from 2007-2009 time frame). I wonder why both companies decided to go with VC-1 rather than H.264?


I don't know about Amazon (I thought they used MPEG2 for TiVo) but Netflix was using VC1 originally because of their Silverlight platform for desktop browser use. After looking seems the TiVo uses the VC1 encodings Netflix made for Silverlight. I think this is why you can see variations on the streams between different devices.

Probably DRM capability was a driving factor on use.

_________________
My pyTivo fork - Read link for changes
FFmpeg for OS X
Back to top
View user's profile Send private message
wmcbrine



Joined: 04 Jan 2008
Posts: 2009
Location: Maryland

PostPosted: Thu Mar 01, 2012 5:42 am    Post subject: Reply with quote

Amazon uses MPEG2 for SD but VC1 for HD, as I recall it.
_________________
My pyTivo fork . My page
Back to top
View user's profile Send private message Visit poster's website
txporter



Joined: 11 Dec 2008
Posts: 263
Location: Austin, TX

PostPosted: Thu Mar 01, 2012 2:58 pm    Post subject: Reply with quote

wmcbrine wrote:
Amazon uses MPEG2 for SD but VC1 for HD, as I recall it.


Do you know if that is still true? It would be very easy to have an SD video with higher bitrate than the HD video if that were true. I think Amazon uses 1280x720 for their HD video, correct?
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    pyTivo Discussion Forum Forum Index -> pyTivo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum
Site is in NO WAY affiliated with TiVo Inc

Powered by phpBB © 2001, 2005 phpBB Group
phpBB SEO

Get pytivo at SourceForge.net. Fast, secure and Free Open Source software downloads
[ Time: 0.2481s ][ Queries: 12 (0.0217s) ][ GZIP on - Debug on ]