Content-type: multipart/alternative; Boundary="1__=4EBBFE04DFAB3F518f9e8a93df938690918c4EBBFE04DFAB3F51" --1__=4EBBFE04DFAB3F518f9e8a93df938690918c4EBBFE04DFAB3F51 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: quoted-printable Hi, Felipe is itegrating the changes ... you need to set up GIT and clone = git://github.com/felipec/gst-openmax.git = = BR, bruno = = = = To "Felipe Contreras" = , = "Ling Shi" "Bruno Smets" = cc gstreamer-openmax@lists.sourcefo= rge 2008-07-30 03:15 PM .net, = gstreamer-embedded@lists.sourcef= org e.net = Subj= ect Re: [Gstreamer-openmax] Discussi= on on the hardware accelerator = solution in GstOpenMAX project. = = = = = = = Felipe and Bruno, Thank for your reply. We have a very similar idea on how to use OMX in = gst. Now, I got more confidence to use, involve, and contribute into this project. Brouno, Could you tell me where can find your change? Is it possible for me to study your code before release? Felipe, TI OMAP 3xxx is just one of our target platform. I just got an TI OMAP = 3xxx board in test. I will investigate the OMX IL code from omap zoom. Please check other comments in line. On Wed, Jul 30, 2008 at 4:39 PM, Felipe Contreras < felipe.contreras@nokia.com> wrote: Hi, On Wed, 2008-07-30 at 15:00 +0800, ext Ling Shi wrote: > Hi, all > I'm in a research project to port gstreamer into embedded system. N= ow, > we encounter the issue on how to integrate hardware accelerators > (DSP/GPU) into gst. After evaluating different solution, we think > GstOpenMAX project may be the best one for us, because > > 1) OpenMAX is an industry standard > 2) more and more DSP/GPU vendor support OpenMAX > > But, I still have several questions on this project. > 1. Does Nokia N8xx serials use GstOpenMAX? > I know Nokia engineers lead this project. I also know N8xx serials = use > gstreamer as default playback engine, and it uses TI OMAP 2420, whi= ch > has DSP. Can anyone tell me if N8xx use GstOpenMAX? If no, does N8x= x > plan to use it in the future? The current Maemo Products don't use OpenMAX IL, they use TI's DSP directly through the open source version of the DSP bridge (DSP gateway). The plan is to use OpenMAX IL so we can choose between different implementations without much effort. TI has started to provide their OpenMAX IL source code: http://omapzoom.org/gf/project/openmax/wiki/ > 2. What's GstOpenMAX plan to support DSP/GPU in the future? > I review several plugins in GstOpenMAX and find current design can > only support none-tunnel communication. It's not the best solution= in > hardware, because of bad performance. So, we plan to improve it by > adding tunneled or proprietary communication. Do you have such plan= ? > If yes, can we involve in design? Indeed, as Bruno mentions, NXP has contributed code that adds support= for tunneled communication. It is maintained in a separate branch and= will soon be merged to the master branch. > In addition, most accelerators work as a decoder and a render. It > means, the encoded data sent to it will directly be decoded and > rendered, and will not be retrieved back again. How the gst or omx > organize its pipeline in this situation? We are evaluating two > solutions. > > =3D=3D=3DSolution 1=3D=3D=3D > We can design a super omx sink component to cover decoder and rende= r. > This is the solution is used by N8xx. > src ! demux ! sink > | > super omx sink > | > +--------------------------------------------+ > | hardware accelerator | > +--------------------------------------------+ The disadvantage of this solution is that it requires the creation of= many elements to cover all the possible combination of elements. This= becomes specially a problem when you add for example some filtering, like a volume control, etc. [Shi Ling] You idea is exaclty same with me. I also think it's not a good solution= . > =3D=3D=3DSolution 2=3D=3D=3D > We can separate omx decoder, omx post processer, and omx sink > elements. We enhance decoder, post processor, and sink plugin in > GstOpenMAX. If GstOpenMAX plugin found its neighborhood are GstOpen= MAX > plugin, it will try to establish tunneled communication or propriet= ary > communication firstly. It means, although we have 3 OMX plugin in g= st > pipeline, there is no data in gst pad and omx port. The last two > gst/omx plugin only provide control function, but not support proce= ss > data flow. Of cause, If the connection is failed, it will use > none-tunnel communication. > > src ! demux ! decoder ! post processor ! sink > | | = | > omx dec omx pp omx sink > | | = | > +--------------------------------------------+ > | hardware accelerator | > +--------------------------------------------+ > > It seems solution 2 is more flexible. How about your suggestion on = the > 2 solutions? Which one is feasible? Do you have other solutions? This is exactly how NXP implemented it and seems to be working fine. The problem I see with both solutions is that there will be A/V sync issues when using OMX sinks in tunneling mode. The idea is to solve these issues by mapping the OMX clock to the GST clock. However, this= hasn't been implemented yet. [Shi Ling] Yes, so many things need to be improved in the future. Best regards. -- Felipe Contreras = --1__=4EBBFE04DFAB3F518f9e8a93df938690918c4EBBFE04DFAB3F51 Content-type: text/html; charset=US-ASCII Content-Disposition: inline Content-transfer-encoding: quoted-printable

Hi,

Felipe is itegrating the changes ... you need to set up GIT and clone =
git://github.com/felipec/gst-openmax.git


BR, bruno
3D"Inactive"Ling Shi" <shil66@gmail.com&g= t;


=
    "Ling Shi" <shil66@gmail.com>

    2008-07-30 03:15 PM

=
3D""
To
3D""
"Felipe Contreras" <felipe.contreras@noki= a.com>, "Bruno Smets" <bruno.smets@nxp.com>
3D""
cc
3D""
gstreamer-openmax@lists.sourceforge.net, gstreamer-emb= edded@lists.sourceforge.net
3D""
Subject
3D""
Re: [Gstreamer-openmax] Discussion on the hardware acc= elerator solution in GstOpenMAX project.
=3D""3D""<= /td>

Felipe and Bruno,
Thank for your reply. We have a very similar idea on how to use OMX in = gst. Now, I got more confidence to use, involve, and contribute into th= is project.

Brouno,
Could you tell me where can find your change? Is it possible for me to = study your code before release?

Felipe,
TI OMAP 3xxx is just one of our target platform. I just got an TI OMAP = 3xxx board in test. I will investigate the OMX IL code from omap zoom. =

Please check other comments in line.



On Wed, Jul 30, 2008 at 4:39 PM, Felipe Contreras <= felipe.contreras@nokia.com> wrote:

    Hi,

    On Wed, 2008-07-30 at 15:00 +0800, ext Ling Shi wrote:
    > Hi, all
    > I'm in a research project to port gstreamer into embedded system. = Now,
    > we encounter the issue on how to integrate hardware accelerators > (DSP/GPU) into gst. After evaluating different solution, we think<= br> > GstOpenMAX project may be the best one for us, because
    >
    > 1) OpenMAX is an industry standard
    > 2) more and more DSP/GPU vendor support OpenMAX
    >
    > But, I still have several questions on this project.
    > 1. Does Nokia N8xx serials use GstOpenMAX?
    > I know Nokia engineers lead this project. I also know N8xx serials= use
    > gstreamer as default playback engine, and it uses TI OMAP 2420, wh= ich
    > has DSP. Can anyone tell me if N8xx use GstOpenMAX? If no, does N8= xx
    > plan to use it in the future?

    The current Maemo Products don't use OpenMAX IL, they = use TI's DSP
    directly through the open source version of the DSP bridge (DSP
    gateway).

    The plan is to use OpenMAX IL so we can choose between different
    implementations without much effort.

    TI has started to provide their OpenMAX IL source code:

    http://omapzoom.or= g/gf/project/openmax/wiki/

    > 2. What's GstOpenMAX plan to support DSP/GPU in the future?
    > I review several plugins in GstOpenMAX and find current design can=
    > only support none-tunnel communication. It's not the best solutio= n in
    > hardware, because of bad performance. So, we plan to improve it by=
    > adding tunneled or proprietary communication. Do you have such pla= n?
    > If yes, can we involve in design?

    Indeed, as Bruno mentions, NXP has contributed code th= at adds support
    for tunneled communication. It is maintained in a separate branch and will soon be merged to the master branch.


    > In addition, most accelerators work as a decoder and a render. It<= br> > means, the encoded data sent to it will directly be decoded and > rendered, and will not be retrieved back again. How the gst or omx=
    > organize its pipeline in this situation? We are evaluating two
    = > solutions.
    >
    > =3D=3D=3DSolution 1=3D=3D=3D
    > We can design a super omx sink component to cover decoder and rend= er.
    > This is the solution is used by N8xx.
    > src ! demux ! sink
    > |
    > super omx sink
    > |
    > +--------------------------------------------+<= br> > | hardware accelerator | > +--------------------------------------------+<= br>

    The disadvantage of this solution is that it requires = the creation of
    many elements to cover all the possible combination of elements. This becomes specially a problem when you add for example some filtering, like a volume control, etc.

[Shi Ling]
You idea is exaclty same with me. I also think it's not a good solution= .




    > =3D=3D=3DSolution 2=3D=3D=3D
    > We can separate omx decoder, omx post processer, and omx sink
    > elements. We enhance decoder, post processor, and sink plugin in > GstOpenMAX. If GstOpenMAX plugin found its neighborhood are GstOpe= nMAX
    > plugin, it will try to establish tunneled communication or proprie= tary
    > communication firstly. It means, although we have 3 OMX plugin in = gst
    > pipeline, there is no data in gst pad and omx port. The last two > gst/omx plugin only provide control function, but not support proc= ess
    > data flow. Of cause, If the connection is failed, it will use
    > none-tunnel communication.
    >
    > src ! demux ! decoder ! post processor ! sink
    > | | = |
    > omx dec omx pp omx sink
    > | | = |
    > +--------------------------------------------+<= br> > | hardware accelerator | > +--------------------------------------------+<= br> >
    > It seems solution 2 is more flexible. How about your suggestion on= the
    > 2 solutions? Which one is feasible? Do you have other solutions?

    This is exactly how NXP implemented it and seems to be= working fine.

    The problem I see with both solutions is that there will be A/V sync issues when using OMX sinks in tunneling mode. The idea is to solve
    = these issues by mapping the OMX clock to the GST clock. However, this hasn't been implemented yet.
[Shi Ling]
Yes, so many things need to be improved in the future.


    Best regards.

    --

    Felipe Contreras

= --1__=4EBBFE04DFAB3F518f9e8a93df938690918c4EBBFE04DFAB3F51--