GStreamer, Python, Qt, Mac OSX

I would like to develop a cross platform streaming-media application. We use:

• Gstreamer, for its advanced pipeline features and active community;
• Python, for its agility;
• Qt, for its native feel on different platforms and
• Mac OSX, to start with, because of its usability.

This seems to be a non-trivial combination. Acknowleding the call by Jono Bacon, I try to put my findings online so anyone using Google should be able to find it.

To start, I tried to show test video as a QWidget on Mac OS X. First I tried to adapt the example PyGST video player to receive an NSView. Unfortunately, using playbin, this is currently impossible. So, I had to just connect a videotestsrc to an osxvideosink. An extension to PyQt had to be made, to enable retrieving the NSView reference. Phil Thompson was so kind as to add this on 2009/10/10, which is now included in PyQt4 4.6.1. I submitted a patch to MacPorts to incorporate this upgrade, so you can now just install port py26-pyqt4@4.6.1.

A simple demonstration:

'''

Author: Sjoerd Op 't Land

Date: 30 October 2009

Sources:

http://ooboontoo.blogspot.com/2008/03/gstreamer-with-python-in-qt-learning-by.html

http://faq.pygtk.org/index.py?req=show&file=faq23.038.htp

http://doc.trolltech.com/4.5/qmaccocoaviewcontainer.html#details

Prerequisites (MacPort ports):

gstreamer

gst-plugins-base

gst-plugins-good

python_select (use to select Python 2.6 with "sudo python_select python26")

py26-pyqt4

py26-gst-python

gst-plugins-ugly

py26-pyobjc2-cocoa

'''

from PyQt4 import QtCore, QtGui # Qt bindings

import sys

import gst # gstreamer bindings

import Foundation # PyObjC bindings for Foundations framework

class TestMonitor(QtGui.QDialog):

def __init__(self, parent=None):

super(TestMonitor, self).__init__(parent)

layout = QtGui.QVBoxLayout(self)

self.label = QtGui.QLabel()

self.label.setText("SMPTE Color Bars with snow")

layout.addWidget(self.label)

self.player = gst.Pipeline("player")

self.source = gst.element_factory_make("videotestsrc","testsrc")

self.sink = gst.element_factory_make("osxvideosink","videosink")

self.player.add(self.source,self.sink)

gst.element_link_many(self.source,self.sink)

bus = gst.Pipeline.get_bus(self.player)

self.player.set_state(gst.STATE_PLAYING)

self.nsview = None

while self.nsview == None:

messagePoll = bus.poll(gst.MESSAGE_ANY,1000000000)

if messagePoll:

if messagePoll.structure and messagePoll.structure.has_key("nsview"):

print messagePoll.structure["nsview"]

# use a dirty hack to obtain the address of this gpointer

#TODO: make sure this also works on 64-bit systems

pointerAddress = int(str(messagePoll.structure["nsview"])[13:-1],16)

self.nsview = pointerAddress

# Pushing an NSAutoreleasePool on the stack is necessary to suppress

# leakage error messages. However, I'm not convinces that leakage is

# really avoided, this way.

#TODO: check for memory leaks

pool = Foundation.NSAutoreleasePool.alloc().init()

self.viewer = QtGui.QMacCocoaViewContainer(self.nsview,self)

del pool

self.viewer.setMinimumSize(320,240)

layout.addWidget(self.viewer)

if __name__ == "__main__":

app = QtGui.QApplication(sys.argv)

testmonitor = TestMonitor()

testmonitor.show()

app.exec_()

Some important things still are to be done:

• Make this cross-platform
• Check for memory leaks
• Do this in a threaded way

Suggestions, comments and insults are welcome!