Forgive the generality of this post - I am new to GPAC but would like to know before I start whether it can satisfy my use case.
I have an online interactive dialogue where users can change lines based on the alternatives provided throughout and play back the new video conversation. I want to deliver this via DASH to the dash.js player in Chrome.
So instead of having one file split into different segments either internally or as external files, I will start with very many external files in different directories as hopefully dashable segments which can be strung together into different sequences. For this, I will I think need unique mpds and some in-fragment on-the-fly alterations to the segments the mpd points to.
Assuming I can generate an mpd with the correct order of files, is it possible with GPAC to create from scratch the initial moov fragment header or mp4 init file, that pieces together with the correct metadata all the files the user or program has chosen to string together?
I will then presumably have to change the sidx of each named fragment, or mfhd so they obey the correct order now given to them, and perhaps the segment timing information too (?).
Important to my use case is the fact that at any given time the user can change a line or camera angle within their dialogue, and then play it back with the new material seamless inserted into it replacing what was there before. In this case I will also need to flush the current mpd file from the player, recreate it or modify it, and revise the sidx etc from the fragments now being used.
Is this theoretically and practically possible using GPAC? Are they any other gotchas I have not accounted for?
Thanking you in advance,
ian
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Yes you can. 1) You could demux the MP4 before passing the chuncks to MSE, like MP4Box.js does (https://github.com/gpac/mp4box.js/tree/master). Or 2) you could simply set the different cameras in different DASH representations and let the user choose (instead of relying on adaptation algorithms based on bitrate concerns).
Let me know if it helps.
Romain
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Great to get a quick answer, and even better to know it is possible.
Not sure about setting the cameras to different representations would work for me but the js library is really really interesting.
The player then, if I understand, will get the mpd I make and then via MP4Box.js make the ISOFile and the initializing header boxes with moov and the metadata etc. It could then could fetch the pre-segmented files (a better choice then the client doing it to ease the workload esp. if we need adaptive streaming?) and preprocess them via MP4Box.js setting all the sidx etc box values I need and then pass it off to the MSE in Chrome which will then play it without a hitch? That would be fabulous.
You mean to say there was no need for me to get into C again when I can just more or less manipulate everything in client side javascript?
best
ian
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
You can then push chuncks of data, from different cameras. MP4Box.js is not finished yet, but it can extract data from an input MP4 (why not already fragmented and ready for DASH) and push segments to your player. There is an example in the repo to play with.
You mean to say there was no need for me to get into C again when I can just more or less manipulate everything in client side javascript?
What does 'C' mean? MP4Box.js allows you to manipulate MP4 files (even non-framgmented ones) from javascript.
Romain
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Any news on your use case? Have you been able to solve it? Have you used GPAC, MP4Box.js? Feel free to report bugs or to ask for features at https://github.com/gpac/mp4box.js/issues
Cyril
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi All,
Forgive the generality of this post - I am new to GPAC but would like to know before I start whether it can satisfy my use case.
I have an online interactive dialogue where users can change lines based on the alternatives provided throughout and play back the new video conversation. I want to deliver this via DASH to the dash.js player in Chrome.
So instead of having one file split into different segments either internally or as external files, I will start with very many external files in different directories as hopefully dashable segments which can be strung together into different sequences. For this, I will I think need unique mpds and some in-fragment on-the-fly alterations to the segments the mpd points to.
Assuming I can generate an mpd with the correct order of files, is it possible with GPAC to create from scratch the initial moov fragment header or mp4 init file, that pieces together with the correct metadata all the files the user or program has chosen to string together?
I will then presumably have to change the sidx of each named fragment, or mfhd so they obey the correct order now given to them, and perhaps the segment timing information too (?).
Important to my use case is the fact that at any given time the user can change a line or camera angle within their dialogue, and then play it back with the new material seamless inserted into it replacing what was there before. In this case I will also need to flush the current mpd file from the player, recreate it or modify it, and revise the sidx etc from the fragments now being used.
Is this theoretically and practically possible using GPAC? Are they any other gotchas I have not accounted for?
Thanking you in advance,
ian
Yes you can. 1) You could demux the MP4 before passing the chuncks to MSE, like MP4Box.js does (https://github.com/gpac/mp4box.js/tree/master). Or 2) you could simply set the different cameras in different DASH representations and let the user choose (instead of relying on adaptation algorithms based on bitrate concerns).
Let me know if it helps.
Romain
Thank you, Romain.
Great to get a quick answer, and even better to know it is possible.
Not sure about setting the cameras to different representations would work for me but the js library is really really interesting.
The player then, if I understand, will get the mpd I make and then via MP4Box.js make the ISOFile and the initializing header boxes with moov and the metadata etc. It could then could fetch the pre-segmented files (a better choice then the client doing it to ease the workload esp. if we need adaptive streaming?) and preprocess them via MP4Box.js setting all the sidx etc box values I need and then pass it off to the MSE in Chrome which will then play it without a hitch? That would be fabulous.
You mean to say there was no need for me to get into C again when I can just more or less manipulate everything in client side javascript?
best
ian
You can even bypass the MPD as a first step. MSE is not DASH specific at all. The MSE spec says what it expects for ISOBMF/MP4: https://dvcs.w3.org/hg/html-media/raw-file/tip/media-source/isobmff-byte-stream-format.html
You can then push chuncks of data, from different cameras. MP4Box.js is not finished yet, but it can extract data from an input MP4 (why not already fragmented and ready for DASH) and push segments to your player. There is an example in the repo to play with.
What does 'C' mean? MP4Box.js allows you to manipulate MP4 files (even non-framgmented ones) from javascript.
Romain
Hi ian,
Any news on your use case? Have you been able to solve it? Have you used GPAC, MP4Box.js? Feel free to report bugs or to ask for features at https://github.com/gpac/mp4box.js/issues
Cyril