Making XMMS Effect plugins
Most of us will agree that though there are many audio players for GNU/Linux and the BSDs, the most popular by far is XMMS. Not only is it similar to Winamp, which eases things for people coming from Windows, it is also simple and has all the features one would expect in a complete multimedia player. Best of all, it is extensible with additional plugins for added functionality, file support, visualization and, of course, sound effects.
In this article, I will present an introduction to some functions provided in the XMMS plugin library for developing our own effects, along with some useful techniques. Using an example (a simple echo plugin), I shall walk through the steps, introducing the functions one by one. The only requirement is that you should know C, but you needn't be an audio engineer (I'm not one!) to be able to churn out some simple effects.
Before we begin - an effect plugin, or, for that matter, any other plugin for XMMS, is a shared object (.so) file. Make sure you have gcc, make and other development tools on you machine, along with the XMMS plugin development headers and libraries.
Prerequisites
Check if the XMMS plugin library is installed already on your system. The presence of headers, /usr/include/xmms or /usr/local/include/xmms, is a good indication of this. If it isn't, install the xmms-devel (or equivalent) package from your distribution.
Technique
Though this article should be only about writing XMMS plugins, I also thought that it would be appropriate to mention a buffering technique used all the time in audio processing, called circular buffers. In our case, let us say we want to make the echo sound, say 370 ms (0.37 seconds) later. If we are working with data sampled at 44100 Hz, then the size of the buffer required would be 0.37 * 44100 = 16317 elements each for left and right (assuming you are providing separate echoes for the two channels). Now, the obvious way to solve our problem is, as soon as we get a new sample, we can retrieve the most delayed sample from the buffer for the echo effect, then shift the contents of the buffer so that the second element goes to the first place, third to second and so on and push the new sample to the last position, just like a FIFO queue. However, this would mean about 16317 memory read and write operations have to be executed for every sample processed, which is quite expensive when it comes to CPU use.
A simple solution to this problem is to treat the buffer as circular. Here, we create an index or pointer to the element (or an index) in the buffer which you mark as the last element to be read for the delay, read that element for the echo effect, write the new sample into the same memory location, and increment the index. If the pointer or index reaches the end of the array, it just loops back to the first element. The following picture should give you an idea.
With this, we have reduced the memory operations per sample to just one read and write; many times more efficient than the previous case. For more details, see the Wikipedia article on this topic.
Writing the Plugin
Now, we start writing a minimal plugin that will create a fixed echo effect. The full source is available in exampleecho.c.
A plugin is registered with XMMS using the following function:
EffectPlugin *get_eplugin_info(void);
First, we create the buffers for the echo effect. I have chosen a 16384-sample delay, which is about 0.37 sec. Since we will use these as circular buffers, we shall keep two integers to index the location in the buffers.
#define BUFFER_LENGTH 16384 static gfloat *buffer_left = NULL; static gfloat *buffer_right = NULL; static guint index_left = 0; static guint index_right = 0;
Now, we observe the requirements of the EffectPlugin structure, defined in /usr/include/xmms/plugin.h:
typedef struct
{
void *handle; /* Filled in by xmms */
char *filename; /* Filled in by xmms */
char *description; /* The description that is shown in the preferences box */
void (*init) (void); /* Called when the plugin is loaded */
void (*cleanup) (void); /* Called when the plugin is unloaded */
void (*about) (void); /* Show the about box */
void (*configure) (void); /* Show the configure box */
int (*mod_samples) (gpointer *data, gint length, AFormat fmt,
gint srate, gint nch); /* Modify samples */
void (*query_format) (AFormat *fmt,gint *rate, gint *nch);
}
EffectPlugin;
Though most of the functions specified are optional and you can just make them NULL, we see that we need to write some functions for our plugin to be useful. I will call these query, init, cleanup, configure, about and mod_samples, though you can name them whatever you like. Of these, we will first write about, since it seems to be the easiest!
static char *about_text = "Kumar's test reverberation plugin!";
void
about(void)
{
static GtkWidget *about_dialog = NULL;
if (about_dialog != NULL)
return;
about_dialog = xmms_show_message("About Example Echo Plugin",
about_text, "Ok", FALSE,
NULL, NULL);
gtk_signal_connect(GTK_OBJECT(about_dialog), "destroy",
GTK_SIGNAL_FUNC(gtk_widget_destroyed),
&about_dialog);
}
Next, we write the query function. The query function informs XMMS what type of audio data we wish to process. This is a good idea, because our plugin may not function as expected for data with different attributes, such as a different sampling rate. To make it more clear, let us say we want an echo after 0.1 seconds, that is 0.1 * 44100 samples later when 44100 Hz is the sampling rate. But it should be changed to 0.1 * 22050 for a file with 22050 Hz sampling rate. Yes, it can be done on the fly, but I leave such tricks for you to try out.
We'll make our plugin work with 16 bit little-endian stereo data, sampled at 44100 (stereo) samples per second.
void
query(AFormat *afmt, gint *s_rate, gint *n_channels) {
*afmt = FMT_S16_LE;
*s_rate = 44100;
*n_channels = 2;
}
The init function is used to initialize your plugin's settings, and you may also want to initialize your audio buffers here. This function is called the moment the plugin is enabled, or whenever the plugin loads (like when XMMS is started). In our plugin, we use two audio buffers to hold past left and right samples, which need to be zeroed initially to avoid "noise". Note that g_malloc0 is a GLIB function. In general, it is convenient to use GLIB functions and data types, and there are no extra requirements because XMMS needs them and uses them anyway.
void init(void)
{
if (buffer_left == NULL) {
buffer_left = g_malloc0(sizeof(gfloat) * BUFFER_LENGTH);
}
if (buffer_right == NULL) {
buffer_right = g_malloc0(sizeof(gfloat) *
BUFFER_LENGTH);
}
}
The cleanup, as the name very well suggests, is to free any resources you might have allocated and prepare for releasing the plugin. It is called the moment the plugin is disabled, or when XMMS exits, if the plugin remains enabled at that time. Here, we just free the buffers that we have allocated.
void
cleanup(void)
{
g_free(buffer_left);
buffer_left = NULL;
g_free(buffer_right);
buffer_right = NULL;
}
The configure function is used to allow the user to modify some characteristics of the plugin. It is called when the user clicks "Configure" for your plugin in the Preferences dialog box. In our plugin, I have left it blank, since I wanted to avoid adding a lot of GTK code. However, this is fairly simple, and left as an exercise for the reader. I would suggest that you refer the source of the plugins which come with XMMS for ideas.
Note that the xmms_cfg_* functions make the use of the XMMS configuration extremely easy, with XMMS doing the dirty work of reading and writing the configuration file. They are present in /usr/include/xmms/configfile.h
Finally, we come to the most important function of the effect plugin, mod_samples, in which the samples are provided and are to be appropriately modified. The declaration of this function is as follows:
gint mod_samples(gpointer * d, gint length, AFormat afmt, gint srate, gint nch);
Here, d is the pointer to the data (samples), and length is the number of bytes of data provided to the function in a given instance. This function is called again and again with successive data; that is, in one pass, you are provided the first set of samples, in the next one, the next set of samples and so on, in order. The format of the data is given by the AFormat value. By checking the afmt value, you can dynamically adjust to changes in data type, etc.; however, we are going to just assume it is what we specified in the query function.
gint mod_samples(gpointer * d, gint length, AFormat afmt, gint srate, gint nch)
{
/* Read data as 16 bit values */
gint16 *data = (gint16 *) * d;
gint i;
/* We have "length" bytes, so "length / 2" samples */
for (i = 0; i < length / 2; i += 2) {
gfloat echo_val;
/* For the left sample */
echo_val = buffer_left[left_index]; /* Retrieve delayed value */
buffer_left[left_index++] = data[i]; /* Store latest sample */
data[i] = (1 - ECHO_SCALE) * data[i] + ECHO_SCALE * echo_val; /* Set the echoed value */
left_index = left_index % BUFFER_LENGTH; /* Circular buffer shift */
/* Do the same for the right channel samples */
echo_val = buffer_right[right_index]; /* Retrieve delayed value */
buffer_right[right_index++] = data[i + 1]; /* Store latest sample */
data[i + 1] = (1 - ECHO_SCALE) * data[i + 1] + ECHO_SCALE * echo_val; /* Set the echoed value */
right_index = right_index % BUFFER_LENGTH; /* Circular buffer shift */
}
return length; /* Amount of data we have processed. */
}
We are expected to process the samples and write the modified values back into the same memory locations. There are two things which you must note here. First, since we have to process samples which are 16 bits (or two bytes) each, the number of samples is actually half the number of bytes. So, length / 2 is the actual number of samples we have. That explains the cast to gint16. Also, since we are handling stereo audio, successive samples alternate between left and right. So, we actually have length / 4 number of left and an equal number of right channel samples, such that data[0] is a left sample, data[1] is a right sample, data[2] is a left sample, and so on.
Finally, we register the plugin with XMMS as follows:
EffectPlugin e_plugin = {
NULL,
NULL,
"Example Echo plugin",
init,
cleanup,
about,
configure,
mod_samples,
query};
EffectPlugin *
get_eplugin_info(void)
{
return &e_plugin;
}
Build, install and test
The compilation and linking is just like for any other shared object file; note that we have to take care of the GTK and GLIB dependencies. You can see the Makefile for details. I also add a line to copy the plugin to $HOME/.xmms/Plugins/ after every compile. This ensures that I don't forget to put it there manually after each build. It is very useful for testing!
I strongly recommend that you initially compile your plugin with the -g option, so that you can debug it using gdb. There are many ways to use gdb to debug the plugin. I generally start gdb as gdb /usr/bin/xmms, or find the PID of a running XMMS process and use gdb --pid <PID>.
For best performance, once you are done with debugging the plugin, recompile it with -O2 or a similar optimization to allow the compiler to optimize your code. You may also wish to consider using a profiler to optimize the code further.
Further details
Now that you are ready and raring to write your own XMMS effect plugins, enjoy yourself. I'd also recommend that you go through the source code of various effect plugins available at xmms.org to get an idea of the techniques and possibilites. Also, for serious DSP advice, comp.dsp is the place to go. dspguru also has a good collection of DSP tricks and ideas. Harmony Central is an excellent place to learn more about artificial sound effects.
Comments, corrections and cribs are always welcome. Please e-mail me at the address in my bio.
Talkback: Discuss this article with The Answer Gang
Kumar Appaiah is studying to earn a B.Tech in Electrical Engineering and M.Tech in Communication Engineering (Dual-degree) at the Indian Institute of Technology Madras. He has been using GNU/Linux for the past five years, and believes that Free Software has helped him do work more neatly, quickly and efficiently.
He is a fan of Debian GNU/Linux. He loves using Mutt, GNU Emacs, XCircuit, GNU Octave, SciPy, Matplotlib and ConTeXt.
