 |
pyTivo Discussion Forum Answers and the development of pyTivo a TiVo transcoding server
|
|
| Author |
Message |
Iluvatar
Joined: 29 Feb 2008 Posts: 335
|
Posted: Mon May 30, 2011 7:53 pm Post subject: |
|
|
so here are the changes I have made to allow pyTivo to remux any video with incompatible audio into mp4 for streaming as long as it has a compatible h264 video stream. This should detect the incompatible audio and re-encode as necessary...ex: DTS, vorbis, etc... You may still run into the non-monotone timestamp error which will still cause a full transcode. FFmpeg 0.6.3 doesn't seem to have this issue but later and earlier revisions do.
Most of this code already existed in pyTivo but I wasn't sure how to create a diff so I'll just post the affected sections.
transcode.py - I have split the mp4_remuxable into separate audio video tests and edited mp4_remux function
| Code: |
# this was previously just a single function that would fail on audio check if incompatible
def mp4_remuxable_video(inFile, tsn=''):
vInfo = video_info(inFile)
return (tivo_compatible_video(vInfo, tsn, 'video/mp4')[0])
def mp4_remuxable_audio(inFile, tsn=''):
vInfo = video_info(inFile)
return (tivo_compatible_audio(vInfo, inFile, tsn, 'video/mp4')[0])
# most of this is wmcbrine's code.
def mp4_remux(inFile, basename, tsn=''):
outFile = inFile + '.pyTivo-temp'
newname = basename + '.pyTivo-temp'
if os.path.exists(outFile):
return None # ugh!
ffmpeg_path = config.get_bin('ffmpeg')
fname = unicode(inFile, 'utf-8')
oname = unicode(outFile, 'utf-8')
if mswindows:
fname = fname.encode('iso8859-1')
oname = oname.encode('iso8859-1')
# I have only added this audio check call with ffmpeg cmd to re-encode
if mp4_remuxable_audio(inFile, tsn):
debug('remuxing ' + inFile + ' to ' + outFile)
cmd = [ffmpeg_path, '-i', fname, '-vcodec', 'copy', '-acodec',
'copy', '-f', 'mp4', oname]
else:
debug('remuxing with re-encoded audio ' + inFile + ' to ' + outFile)
cmd = [ffmpeg_path, '-i', fname, '-vcodec', 'copy', '-ab', '448k',
'-ar', '48000', '-acodec', 'ac3', '-copyts', '-f', 'mp4', oname]
ffmpeg = subprocess.Popen(cmd)
if ffmpeg.wait():
debug('error during remuxing')
os.remove(outFile)
return None
return newname
|
video.py - I have adjusted the mp4_remux(able) functions accordingly to work with the new code.
| Code: |
if (mime == 'video/mpeg' and
transcode.mp4_remuxable_video(f['path'], f['tsn'])):
new_path = transcode.mp4_remux(f['path'], f['name'], f['tsn'])
if new_path:
mime = 'video/mp4'
f['name'] = new_path
|
I have attached the full .py files and they should be able to be dropped in on wmcbrines git from 6/25/11.
I have tested this with many of the different types of files I have and they have all worked. However this is not to say you would have different results. I am sure someone will have issues with the hard coded ffmpeg command but I really didn't have time to hash that one out. Maybe another time.
edit: i have updated my the post to add comments in the code to show exactly what I changed. Most of what you see there already existed.
edit: updated files to match newer revision on wmcbrine's branch.
edit: deleted attachment. lucasnz's post below seems to have much better solution than what I had worked out.
Last edited by Iluvatar on Thu Jul 07, 2011 11:37 am; edited 9 times in total |
|
| Back to top |
|
 |
jcthorne
Joined: 30 Jan 2008 Posts: 243 Location: Houston, TX
|
Posted: Tue May 31, 2011 9:21 am Post subject: |
|
|
Your additions seem to assume that any h264 video stream is playable by the tivo as is. This is not the case. The AVC profile must be 4.1 or less, for 1080 resolution, must be 4 ref frames or less. Also, for a series 3 unit, 720p video must be full frame, ie 1280x720.
Also, audio bitrate need not be limited to 448kbps any longer. 640 seems to work fine now on both TivoHD and Premiere.
Further, the resultant files that ffmpeg creates from this command line are not IMSA standard streamable and the tivo will not accept them as is. The MOV atom must be at the front of the file. |
|
| Back to top |
|
 |
Iluvatar
Joined: 29 Feb 2008 Posts: 335
|
Posted: Tue May 31, 2011 1:23 pm Post subject: |
|
|
I understand that Tivo's will not accept higher than 4.1 AVC profile however I have done no additional checks other than what wmcbrine has programmed pyTivo to check for. If the video passed before then it will pass now (whether TiVo accepts the file or not). I am aware of the 1:1 PAR issue with S3/HD units as I frequently run up against it but pyTivo never fixed this anyways (on pushes). All I have done is moved the audio checks into the remux function in order to prevent the entire file from being transcoded which is a waste of time. Most of what you see there is from wmcbrine. He programmed the original MP4 remux function...I just made it slightly more compatible....to fit my needs.
As to the mov atom, pyTivo already checks for this once it exits the remux function and calls qtfaststart.py if necessary. I have had zero issues (except the h264 level 5 video, but pyTivo doesnt check this anyways as ffmpeg does not return this info) with streaming MKV files with DTS or other audio. I have thrown at least 30 files at my premiere and they work perfectly.
As to the audio, I have hardcoded the ffmpeg command but only because I was lazy. I'm certain it can be changed to use the user defined ffmpeg overrides in the pytivo.conf. Feel free to change it to whatever you want. To be honest it works for me and I just assumed that since I am not a programmer these are probably just ugly hacks that wouldn't ever make it into wmcbrine's GIT unless somebody took it and brought it up to standards. |
|
| Back to top |
|
 |
wmcbrine

Joined: 04 Jan 2008 Posts: 2009 Location: Maryland
|
Posted: Tue May 31, 2011 4:14 pm Post subject: |
|
|
| jcthorne wrote: | Further, the resultant files that ffmpeg creates from this command line are not IMSA standard streamable and the tivo will not accept them as is. The MOV atom must be at the front of the file. |
Yes, that's why this is written out to a temp file: so we can run qtfaststart on it during the transfer. Unfortunately I know of no way to create a streamable MP4 in one pass. _________________ My pyTivo fork . My page |
|
| Back to top |
|
 |
ercdvs
Joined: 11 Nov 2008 Posts: 8
|
Posted: Tue Jun 07, 2011 3:31 pm Post subject: |
|
|
I had not seen this mentioned, but does this feature respect the settings on a global or per tivo basis ?
ie will it always attempt a straight copy if I have a _tivo_ setting of audio_ch = 2 set?
Im getting mixed results during my tests. it seems no matter what my settings, pytivo will do the .pytivo-temp version, and if i kill ffmpeg, it will restart itself using my config settings while trying to stream directly.
basically i am finding that a copy takes just a few minutes, then transfers @ 20+ Mb/sec.. while my 'lower quality' transcode stream takes a lot longer, and seems to transfer @ 6 Mb/sec max (but strangely seems to give both -ac 2 AND -acodec copy:
/usr/local/bin/ffmpeg -i /home/master/video/TV/vidfile.mkv -vcodec mpeg2video -b 8388k -maxrate 30000k -bufsize 4096k -ab 448k -ar 48000 -ac 2 -acodec copy -copyts -threads 2 -f vob - |
|
| Back to top |
|
 |
Iluvatar
Joined: 29 Feb 2008 Posts: 335
|
Posted: Tue Jun 07, 2011 5:29 pm Post subject: |
|
|
| ercdvs wrote: | I had not seen this mentioned, but does this feature respect the settings on a global or per tivo basis ?
ie will it always attempt a straight copy if I have a _tivo_ setting of audio_ch = 2 set?
Im getting mixed results during my tests. it seems no matter what my settings, pytivo will do the .pytivo-temp version, and if i kill ffmpeg, it will restart itself using my config settings while trying to stream directly.
basically i am finding that a copy takes just a few minutes, then transfers @ 20+ Mb/sec.. while my 'lower quality' transcode stream takes a lot longer, and seems to transfer @ 6 Mb/sec max (but strangely seems to give both -ac 2 AND -acodec copy:
/usr/local/bin/ffmpeg -i /home/master/video/TV/vidfile.mkv -vcodec mpeg2video -b 8388k -maxrate 30000k -bufsize 4096k -ab 448k -ar 48000 -ac 2 -acodec copy -copyts -threads 2 -f vob - |
No the code does not change the audio stream to user defined settings it only copies what is in the container as long as it is compatible. The edited version I posted above transcodes incompatible audio but still does not change the stream to the user defined settings but only changes the codec to AC3 at a bitrate of 448kbps.
Why are you killing ffmpeg? It is remuxing the streams into a compatible container. When you kill ffmpeg pyTivo assumes it errored out and then does a normal full transcode on the file (along with your user defined audio settings)
The way the code is now you can either have a fast copy remux with audio the same as it is in the original file or you can do a full slow transcode to get your custom audio settings.
I have a feeling it wouldn't be hard to do but when I posted my edits I did not have time to figure out how to use pyTivo's existing functions to do this. Maybe wmcbrine will take a stab at it if he is interested. |
|
| Back to top |
|
 |
ercdvs
Joined: 11 Nov 2008 Posts: 8
|
Posted: Tue Jun 07, 2011 7:50 pm Post subject: |
|
|
I was killing ffmpeg because i was testing various options out, starting a pull, then killing it after i saw what it was sending. it was just curious to see a 'backup' transcode when the main process failed.
Yeah, 99% of the time sending the full file over in a different container is much better, even though with this particular tv HD video and 5.1 audio is wasted.
All my settings and test are still valid for transcoded streams though |
|
| Back to top |
|
 |
lucasnz
Joined: 13 Sep 2010 Posts: 251
|
|
| Back to top |
|
 |
Iluvatar
Joined: 29 Feb 2008 Posts: 335
|
Posted: Thu Jul 07, 2011 11:35 am Post subject: |
|
|
This is great. Thank you for your efforts. Exactly what I wanted but didn't know how to code. I'll throw lots of videos at it and let you know how it worked out. |
|
| Back to top |
|
 |
wmcbrine

Joined: 04 Jan 2008 Posts: 2009 Location: Maryland
|
Posted: Thu Jul 07, 2011 11:54 am Post subject: |
|
|
Note that, by using select_audiocodec(), your patch will transcode otherwise-compatible AAC tracks. _________________ My pyTivo fork . My page |
|
| Back to top |
|
 |
Iluvatar
Joined: 29 Feb 2008 Posts: 335
|
Posted: Thu Jul 07, 2011 12:42 pm Post subject: |
|
|
So far works great. DTS audio and >448k AC3 is transcoded.
I would guess though that you could remove the following code which seems redundant given the function:
| Code: |
'video_br': select_videobr(inFile, tsn),
'video_fps': select_videofps(inFile, tsn),
'max_video_br': select_maxvideobr(tsn),
'buff_size': select_buffsize(tsn),
'aspect_ratio': ' '.join(select_aspect(inFile, tsn)),
|
To get to this point the file must have already passed video checks so the video at this point is always going to be '-vcodec copy' and the above settings are never ever used. If the remux errors out then the file still gets returned back to the normal pyTivo transcode function anyways.
edit: nevermind, it looks like if 'getFFmpegTemplate(tsn)' doesn't get all variables it errors out. |
|
| Back to top |
|
 |
txporter
Joined: 11 Dec 2008 Posts: 263 Location: Austin, TX
|
Posted: Thu Jul 07, 2011 7:06 pm Post subject: |
|
|
| wmcbrine wrote: | Note that, by using select_audiocodec(), your patch will transcode otherwise-compatible AAC tracks. |
You are saying that H.264 video with AAC audio will transcode with this latest patch? Not good. I transcode everything to AAC so that I don't have to re-encode audio for my idevices. |
|
| Back to top |
|
 |
lucasnz
Joined: 13 Sep 2010 Posts: 251
|
Posted: Thu Jul 07, 2011 8:04 pm Post subject: |
|
|
| txporter wrote: | | wmcbrine wrote: | Note that, by using select_audiocodec(), your patch will transcode otherwise-compatible AAC tracks. |
You are saying that H.264 video with AAC audio will transcode with this latest patch? Not good. I transcode everything to AAC so that I don't have to re-encode audio for my idevices. |
Ths is by design. I find the tivo gets the channels mixed up with 6 channel aac audio.... So up to now my process has been to manually transcode aac audio to ac3. |
|
| Back to top |
|
 |
txporter
Joined: 11 Dec 2008 Posts: 263 Location: Austin, TX
|
Posted: Thu Jul 07, 2011 8:25 pm Post subject: |
|
|
| Is there a way to not get that by flag or something? I run my pytivo instances off of a tonidoplug, and I don't think it is really up to transcoding video on the fly. |
|
| Back to top |
|
 |
lucasnz
Joined: 13 Sep 2010 Posts: 251
|
Posted: Thu Jul 07, 2011 8:44 pm Post subject: |
|
|
| txporter wrote: | Is there a way to not get that by flag or something? I run my pytivo instances off of a tonidoplug, and I don't think it is really up to transcoding video on the fly. |
No just the audio will be transcoded the video is copied as per the -vcodec copy command.
If you have debuging enabled the cmd that it run is output to the log file.... |
|
| Back to top |
|
 |
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum You cannot attach files in this forum You can download files in this forum
|
|