From: David S. <ds...@sc...> - 2004-12-15 01:17:13
|
On Tue, Dec 14, 2004 at 05:48:23PM +0100, Thomas Vander Stichele wrote: > Hi, > > > No, Wim proposes to make lots of internal GStreamer function calls > > threadsafe by adding locking around ie gst_pad_(un)link. This requires > > lots of work - especially deadlock prevention - inside the core. > > My proposal is to make the whole API not threadsafe and require that each > > function be called in its context. That's a lot easier. > > Ok, I've tried to piece together from some IRC logs and some mails what > you *do* propose wrt. threads in general and I can't seem to piece > together a coherent pictures. Could you please explain exactly what you > want to do, where there are threads and where there aren't, and so on ? > > A mail higher up in the thread has you saying > > >>> > David and I agreed on the fact that we want to get rid of threads. Do > not > support threads in Gstreamer anymore, it only causes races. This of > course > only works with a non-blocking API. But if you want a threadsafe model > you're in for quite a ride, especially with dynamic pipelines. > <<< > > which Colin said he assumed meant "the GstThread element". Is that the > case ? Essentially, yes. AFAICT, Benjamin (in this context) wants to remove the concept of pipelines having explicit threads in them, i.e., gst-launch lines with {}'s. I've advocated this change for a long time, but for a different reason -- explicitly stating threading is not necessary, since a scheduler like opt can figure out correct threading without the hints. > Also, I assume everyone agrees that for all approaches, there could be > more than one thread involved ? Even if that thread is private to the > element ? Yes. It's quite clear that at the bare minimum, we need to allow elements to start up threads and execute code. There should be a GstElement subclass or a helper library for this. > I am not sure either how you can for example queue a network stream > inside the pipeline without using threads somehow. This is actually dirt simple. 'socketsrc ! queue ! sink', socketsrc select()s on its socket (or, more likely, requests the core to perform the same action) and passes buffers to queue. queue accepts any buffers and saves them. When requested, queue pushes a buffer. sink (assuming it works like osssink, say) waits for the output device to be ready (again, using select() functionality in the core), pulls a buffer, and writes it to the device. Note that this is all done without any waiting done while in the stack frame of any of the elements. That means interruptions, events, state changes, property changes, object desctruction, etc., all happen when the element is not executing, automagically. > I'm getting various different interpretations from different mails/irc > snippets/other people's interpretation, and it would be nice to see a > coherent picture of exactly what it is you want to do wrt. thread- > safety. It's impossible to evaluate if your ideas can work for actual > applications without having a good picture of what your ideas are. > > Specifics that are important is "how do apps react to signals/events > from the pipeline", "how can apps manipulate the pipeline", "which part > of gstreamer uses threads and for what reason", "how do they not leak", > and so on. - How do apps react to signals/events from the pipeline: exactly like gtk -- you create an object (a pipeline) from the main thread, call it from the main thread, manipulate it from the main, and get signals from the object (or its subobjects) in the main thread, marshalled from the main loop in the main thread. This is the very simple case; in a single threaded app, gstreamer should look and act like a single threaded library, and not ever make the app think about threading or locking. Advanced usage: replace 'main thread' above with 'thread B', and it works exactly the same. Key point is that a particular gst object is tied to a particular application thread. (Marlin and rhythmbox use this.) Independent pipelines do not affect each other in any way. Really advanced usage: (don't know if I really want to support this) As long as the application serializes all accesses to a particular GstPipeline and its children, gstreamer should not care which thread it is called from. It's unclear, in this case, to which thread signals should be sent, which is why I don't want to support it. I also don't know any application that needs this, and can't think of a sane design that would. Usage I'm not supporting: libgstreamer is completely reentrant, and you can access any object from any thread with no concern about concurrency, and gstreamer will handle all serialization and locking. - How can apps manipulate the pipeline: exactly like gtk, see above. - Which part of gstreamer uses threads and for what reason: In the cleanest, simplest case, _no part_ of gstreamer needs to use threads. Obviously, a limited number of elements must be written with a helper thread, since they can't properly work with a device and/or library without blocking. [Note that this is nearly identical in spirit to 0.8, except that the core forces lots of elements to be blocking since it doesn't provide appropriate API to allow them to be non-blocking, and it complicates helper threads by tying them into the scheduler. So, a simple application will run in the main thread, call gtk and gstreamer from the main thread, gstreamer schedules all elements in the main thread, and since every element is non-blocking, no portion is CPU starved (e.g., gtk, gst, or any of the elements.) This model is really simple and requires nothing special in terms of threading or locking in gstreamer -- signals are emitted and marshalled with no special thread-changing code, and you can even call gui code directly from a GstElement. There are a few difficulties with the above model. First, you can't take advantage of multiple processors. Second, certain elements would have trouble providing glitch-free output, e.g., ximagesink, since it depends on precise timing of code execution to display an image without timing glitches. Arguably, you could have ximagesink create a helper thread, which is what I would recommend if we were to strictly stick to the model in the previous paragraph. However, I think it's generally wise to separate gstreamer execution from GUI glitches. To fix this, the core starts up a thread for each CPU (roughly) for the purpose of executing element code. The core also might use the main thread for executing element code, particularly at the request of the element, which may want to run GUI code or manipulate the pipeline (which, remember, can only be done from the main thread). Most elements, however, are completely happy executing in another thread. There are a number of complications executing element code in a non-main thread: 1, signals from the elements need to be sent to the main thread to be marshalled, 2, the main thread needs to make sure that none of the execution threads are using the element when it wants to negotiate pads, link/unlink, change properties, etc., and 3, the API generally used by elements (which doesn't overlap significantly with the application API) needs to be thread-safe, since it can be called from a non-main thread. - How do they not leak? As you can see from the above paragraphs, non-main threads are very limited in what they can actually do, including manipulate objects, so leaking is not a problem. dave... |