Win32 Python: Getting all window titles

This post shows how you can retrieve all window titles in Microsoft Windows using Python’s ctypes module. Moreover, it also acts as a ctypes tutorial, showing how to create and use callback functions.

The following is the full code. Keep reading if you want to understand how it works. (Note: If you are reading this as a ctypes tutorial and are having trouble following the explanation, you may want to go through my previous tutorial first.)

import ctypes

EnumWindows = ctypes.windll.user32.EnumWindows
EnumWindowsProc = ctypes.CFUNCTYPE(ctypes.c_bool, ctypes.POINTER(ctypes.c_int), ctypes.POINTER(ctypes.c_int))
GetWindowText = ctypes.windll.user32.GetWindowTextW
GetWindowTextLength = ctypes.windll.user32.GetWindowTextLengthW
IsWindowVisible = ctypes.windll.user32.IsWindowVisible

titles = []
def foreach_window(hwnd, lParam):
	if IsWindowVisible(hwnd):
		length = GetWindowTextLength(hwnd)
		buff = ctypes.create_unicode_buffer(length + 1)
		GetWindowText(hwnd, buff, length + 1)
		titles.append(buff.value)
	return True
EnumWindows(EnumWindowsProc(foreach_window), 0)

print(titles)

Pango: Determine if a font is monospaced

If you have a GtkFontButton, finding out whether the chosen font is monospaced is quite a complicated process. Here is a complete walk-through.

(By the way, I will be using PyGTK’s Pango documentation because the C version is a mess.)

FontButton.get_font_name returns the font family (a.k.a. “font name”), style, and size; for example, “Liberation Serif Italic 14″. The first thing we need to do is pick just the family name. We do this by going through a PangoFontDescription.

desc_str = font_button.get_font_name()  # Liberation Serif Italic 14
desc = pango.FontDescription(desc_str)
family_name = desc.get_family()  # Liberation Serif

Next, check whether the font family describes a monospaced font. Here is where it gets dodgy. We need an arbitrary PangoContext, which can be obtained from a GtkWidget using Widget.get_pango_context. We then list all available font families and find the one with the appropriate name. Call FontFamily.is_monospace to finish the job.

(By the way, this is also a good place to show off Python’s for-else construct.)

context = widget.get_pango_context()  # widget can be any GtkWidget.
for family in context.list_families():
	if family.get_name() == family_name:
		break
else:  # Should not happen.
	assert False
family.is_monospace()  # False -- Liberation Serif is proportional.

Win32 Python: getting user’s display name using ctypes

This post explains how you can obtain the user’s display name (a.k.a. “real name” or “full name”) in Windows, using Python’s ctypes module. However, it also serves as a mini tutorial/demonstration of ctypes.

First, a bit of background. I researched this while working on a patch for Jokosher. When you create a new project in Jokosher, it will prompt you with a dialog asking for the name of the project and so on. One of the fields in this dialog is the Author field, which by default should be filled with the logged-in user’s real name. While there are several ways to get the user’s login name (a.k.a. “username”), there is no easy way to get their real name in Windows.

This is where ctypes and GetUserNameEx come in. ctypes is a Python library that lets you call C functions. GetUserNameEx is the C function in Win32 API that we want to call.

For the impatient, here is the full code. Continue reading if you want to know how it works and maybe learn a bit about ctypes. Otherwise, copy away. Note, however, that it does not have any error checking whatsoever.

import ctypes

def get_display_name():
	GetUserNameEx = ctypes.windll.secur32.GetUserNameExW
	NameDisplay = 3

	size = ctypes.pointer(ctypes.c_ulong(0))
	GetUserNameEx(NameDisplay, None, size)

	nameBuffer = ctypes.create_unicode_buffer(size.contents.value)
	GetUserNameEx(NameDisplay, nameBuffer, size)
	return nameBuffer.value

Using matplotlib in a Web application

matplotlib‘s FAQ has a section dealing with the exact topic of this post: using matplotlib in a web application server. The problem is that I couldn’t find it easily from my Web searches. What I’m doing here is adding my own twist to the answer, and hopefully making it slightly more search-friendly.

While testing a Web app that I was working on, I noticed that it would often hang. At first I dismissed it as a server problem, but it kept occuring on one particular page. A few hours and many head scratches later, I narrowed the problem down to matplotlib.

# Negative example; do not use.

import matplotlib.pyplot as plt

def callback():
	# ... (process data)
	fig = plt.figure()
	# ... (draw stuff)
	fig.savefile(path)

I used matplotlib to draw a plot and save it to a file. The code was quite long, but it involved steps similar to the above listing. The first time it ran, everything went OK. The second time, it always hung at the pyplot.figure call. This smelled like a threading / deadlock problem, so I tried to put a lock on the pyplot calls (which I should have done anyway, considering pyplot operates on a single plot at a time). Still, it didn’t work.

After some Web searches and more head scratching, I accidentally arrived at the FAQ entry mentioned earlier.

Here’s the gist of the available solutions.

First option: configure matplotlib to use the Anti-Grain Geometry backend. Continue using pyplot, carefully grouping its commands together and surrounding them with a lock.

from threading import Lock
lock = Lock()

import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt

def callback():
	# ... (process data)
	with lock:
		fig = pyplot.figure()
		# ... (draw stuff)
		fig.savefile(path)

Second, better option: dump pyplot and use matplotlib’s object-oriented API instead. For this one, you don’t need to care about threads or locking or whatever.

from matplotlib.backends.backend_agg import FigureCanvasAgg as FigureCanvas
from matplotlib.figure import Figure

def callback():
	# ... (process data)
	fig = Figure()
	canvas = FigureCanvas(fig)
	# ... (draw stuff)
	canvas.print_figure(path)

EBML/Matroska parser for Python

This post explains an EBML parser that I wrote in Python. (EBML is Matroska‘s binary “markup language”.) It is implemented as a single-file library and is available under a free software licence.

Background

I’ve been working to implement Matroska (mka, mkv, webm) tag-reading support in Exaile. Mutagen—the tag library that we use—currently doesn’t have this feature, so I looked elsewhere.

[Update (2010-06-16): News about Exaile 0.3.2, location of the Matroska parser in Exaile's source tree, and discussion on WebM support.]

Choices

Previously, I had a working solution using hachoir-metadata, but it doesn’t really make sense to depend on another large tagging library when we’re already using Mutagen. To make matters worse, I accidentally deleted the branch during our recent Bazaar upgrade problem.

I started shopping around for other possible solutions and found videoparser, which seemed quite nice and compact. It’s still a different library, though, and it doesn’t seem to be packaged in Debian.

I was considering just using it anyway for yet another temporary hack when I chanced on MatroskaParser.pm, a Perl library written by “Omion (on HA)”. It’s only 816 lines of Perl; discounting the README and the Matroska elements table, we’re looking at than less than 450.

Solution

I took the crazy decision of translating MatroskaParser.pm into Python. Despite the horror stories out there about Perl, this particular code is written in a style that is extremely readable if you’re somewhat familiar with the language.

Well, I’ve finished the porting: 250 lines of EBML parser written in Python. Parts of MatroskaParser.pm that are not relevant—mainly the validity checker and the Block parser—have been removed, and the output data structure has been simplified. The next job is to actually extract tags out of the structure.

Matroska tags

Matroska tags are quite different from MP3 and Vorbis tags, in that they’re not just a flat list of key-value pairs. Consider the following snippet.

[{'SimpleTag': [{'TagDefault': [1],
                 'TagLanguage': ['und'],
                 'TagName': ['TITLE'],
                 'TagString': ['Light + Shade']},
                {'TagDefault': [1],
                 'TagLanguage': ['und'],
                 'TagName': ['ARTIST'],
                 'TagString': ['Mike Oldfield']}],
  'Targets': [{'TargetTypevalue': [50]}]},
 {'SimpleTag': [{'TagDefault': [1],
                 'TagLanguage': ['und'],
                 'TagName': ['TITLE'],
                 'TagString': ['Surfing']}],
  'Targets': [{'TargetTypevalue': [30]}]}]

There are two types of tags in this example. The first (target type: 50) explains the album (title: Light + Shade, artist: Mike Oldfield), while the second (target type: 30) explains the track (title: Surfing). Translating this structure into tags that Exaile can understand is not hard, just needs a bit of planning.

(By the way, notice that Matroska makes implementing album artists / compilation albums very intuitive: you can have an artist tag at album level, and another at track level. There are even other levels specified.)

Another tricky part is getting the track length out of the structure. Under /Segment/Info, you’ll find something like

[{'Duration': [14821615.0],
  'TimecodeScale': [22674]}]

At first I randomly assumed the duration is specified in seconds, and got around 171 days as output, which is obviously wrong. Apparently you need to apply this formula to get the length in seconds:

Length = Duration * TimecodeScale / 10^9

Code

The code is now available at Exaile’s repository. It’s licensed under GPL 2+ with the standard Exaile exception, although I will consider relicensing it if there is interest.

Notice that the last 100-or-so lines make up the Matroska tagging part. Depending on your needs, you may need to expand the list of elements based on either MatroskaParser.pm or the Matroska specification.

Future

Matroska read-only tag support will be in Exaile 0.3.2. Maybe one day I’ll add write support and integrate the whole thing into Mutagen, but don’t count on it. If anyone wants to do it, I’m more than happy to help.

My next goal is to create a subclass of the EBML parser that uses GIO. It probably won’t be relevant to most people, so just be aware.

What about WebM?

Funny how I made this post shortly before WebM was announced. Coincidence? Yes, unfortunately; I’m not as cool as the Mozilla and Opera people, who were let in on Google’s secret.

At this point, the WebM container is mostly just a subset of Matroska (the only incompatibility I’ve noticed is the change in doctype, from matroska to webm). As far as I know, they use the exact same EBML structure for tags, so there’s no reason Exaile or this code shouldn’t be able to read tags from a WebM file.

GIO tutorial: Stream IO

This article is a short tutorial showing GIO’s file reading and writing functionality.

Note: This article is part of my ongoing GIO tutorial series, which currently consists of: (1) File and path operations; (2) File IO (this document); and (3) Launching files.

GIO tutorial: File operations

Recently I ported Exaile’s Files panel to use GIO instead of Python’s os module. The reason is simple: remote browsing.

Yes, you can now browse an SMB share or an SFTP server through the Files panel. Just mount the remote filesystem with gvfs-mount and enter the URL into the location bar. Previously Aren ported Exaile’s core to GIO as well, so remote files can be played just fine.

This is the kind of convenience you get out of using GIO/GVFS. In this article I’ll briefly explain what GIO is and how to do some basic file operations—similar to what the Files panel does—using GIO.

Screenshot of Exaile browsing and playing a file through GIO/GVFS

Note: This article is part of my ongoing GIO tutorial series, which currently consists of: (1) File and path operations (this document); (2) File IO; and (3) Launching files.

Unicode with Python 2 and PyGTK

Playing with Unicode in Python 2 is not fun, and combining this with third-party libraries brings even more headaches. This post explains how Unicode in PyGTK is handled.

Note: This information is only valid for Python 2.x. It will likely change when PyGTK releases support for Python 3.

Calling GTK+ functions: PyGTK accepts str and unicode objects as input. str objects are assumed to be in UTF-8. If you pass a non-UTF-8 str to a GTK+ function, it will work until you try to show it, where you’ll get a “PangoWarning: Invalid UTF-8 string passed to pango_layout_set_text()”.

Handling GTK+ return values: PyGTK functions always return strings as str objects. In most (all?) cases, the strings are encoded in UTF-8. Ideally, Python programs should use unicode strings internally, so it’s wise to convert the output of PyGTK function calls to unicode.

Example:

label1.set_text("Some UTF-8 string")
label1.set_text(u"Some Unicode string")
x = label1.get_text()  # x is an str object containing UTF-8 string.
y = unicode(x, 'utf-8')  # y is the unicode version of x.
y = x.decode('utf-8')  # Same as above.

End of 2008 update

I took a long hiatus from all my involvements in the free/open-source software world, including writing in this blog. Right now I’ve decided to go back, though I may not be as active as before.

Things happening during my break that I want to comment on:

• Development on Exaile 0.3 started.
  • A lot of the database-related codebase was rewritten, but the DB has never been my main interest anyway.
  • I don’t see Unicode support improving. At this stage I’ll just wait for Python 3 support.
  • Portability bugs have crept in; I’ve fixed some of them, but haven’t been able to run Exaile on my only Windows (Vista) box.
  • The main Bazaar branch now contains several symbolic links. The problem is, Bazaar doesn’t support symlinks in Windows. This makes working on Exaile that much of a hassle, since I effectively can’t checkout the main branch from Windows. Yes, there are ways around this, but the truth is, I hate fighting with revision control systems just to be able to do my work. </rant> I worked around this by removing all the symlinks in the tree and re-creating them when necessary.
• Python 3 released.
  • In isolation, this news means little. Python 3 itself is not perfect—I’ve bumped into a couple of problems—so it’ll be a while before it gets mass adoption. However, this release means everyone should start thinking about upgrading their code to the new syntax and API.
  • Rules of three: I don’t realistically expect all of Exaile’s dependencies to be ported to Python 3 anytime soon. There are essentially 3 camps we are waiting on: PyGTK, gst-python, and Mutagen. 3 months is my optimistic estimate of the time we need to wait before we can start porting Exaile. After that it’s probably another 3 months’ work for porting. (This projection is wildly off-mark. It’s been a while now and none of our dependencies support Python 3 yet, as of end of 2009. See also GNOME bug 566641.)
• The Australian government is pushing for Internet content filtering.
  • The founder of iiNet (a big Aussie ISP) made a pretty strong comment against this.

database parameter must be string or APSW Connection object

This is a bug in pysqlite 2.4.0, fixed in 2.4.1.

References:

• [Trac] Re: install headaches of 0.11-dev
• The changeset causing the problem
• Trac’s workaround
• The fix