I am headed down to Phoenix for the Air Show. See you all on 03/28/09.

I posted this message to the x264 list this evening (see below).

I just had the thought that there were a couple of folks in
ua-developers interested in cluster programming techniques. For all it's
faults, MPI (with C, C++ or Fortran) is still the 'gold standard' in
academia.

While my code is pretty bad, it just may provide a simple introduction
to message passing with MPI for ridiculously parallel jobs (which I made
video encoding into by arbitrarily chunking the input).

It's not much, but maybe someone wants to pick this project up on their
shoulders? Internet fame and glory await (distributed encoding is always
a popular topic on the internet, piracy being what it is these days).

The only thing this really needs is a frame server (define MPI message
that properly represents a frame, then some simple logic to send them)
and then working to make the frame server intelligent (i.e. work closely
with the deeper logic bits of x264) [Well, maybe some optimization to
make sure that the frame server overhead doesn't clobber the
parallelization gains].

I am too busy at work to make this a priority, but I'd be willing to
help out where I can. Hopefully, soon enough you guys will have a proper
cluster at HACKS on which to try this stuff out.

Peace,
Shawn

Subject: Re: [x264-devel] implementing Cluster farming
From: Shawn Nock <nock@nocko.net>Date: Tue, 17 Mar 2009 23:06:42 -0700
To: Mailing list for x264 developers <x264-devel@videolan.org>

I actually worked on this for a bit as a weekend hack / POC. I
implemented a *very basic* [read: not looking for criticism, I know it
is terrible and borderline worthless] x264-mpi workflow last year.

If anyone cares I put it on my gitweb: http://git.nocko.net/?p=x264-mpi
The primary commitdiff of interest is here:
http://git.nocko.net/?p=x264-mpi;a=commitdiff;h=2200ac1a260a20085b2df588936911338f486f95
although there are useful patches (multipass support) later on (look for
commits by 'Shawn Nock').

There is no frame server (so it relies on shared storage). I am sure
that it breaks a lot of optimizations (specifically scene detection,
which is right out).

In a nutshell, It counts the frames and splits the frames into groups
equal to the number of requested MPI processes. Then each process
encodes a frame group separately, outputting to a file with a sequential
extension. Concatenating these files (I don't think I implemented
concatenation in the primary process... memory fades) produces coherent
output, but nothing resembling the baseline output of a normal x264 run.

Caveats aside, It compiles on x86_64 (mpich2) and ia64 (sgi propriatary
mpi for it's numalink technology). If you don't care that it butchers
encoding efficiency (and ultimately the output file), the raw fps
numbers are encouraging.

I'd love to see a proper MPI support and I could provide several testing
platforms if someone was seriously interested in doing this.

Peace,
Shawn

This is a basic summary, a more thorough post is available here.

The head node now has a POC nfsroot that boots into init (which it pulls via NFS). We are having some reliability problems after that (NFS time-out errors at different places in the init script). However, we learned a lot about how disk-less Linux works and we have a plan to fix it in the near future! The boot server can now boot the nodes, and give us BIOS, kernel, and (if we can sort out the NFS time-out) login on the serial server.

We also did quite a bit of work on the EVA5000 and come up empty handed. I tried to access the serial interface, only to learn that it was diagnostics only (no documented management interface) and Jason stalled on the Windows2000 management utilities. We’ll try again soon, I suppose that in the worst case we could use the one giant LUN as-is and not break it up… but let’s hope we can use the space a little more intelligently.

I also fixed the boot-time networking config on the server we made for the ua-developers club. I fat-fingered a conf file entry and the network didn’t come back after Jason power cycled the rack (QA testing I am sure ).

Stay tuned, the fun stuff is on the horizon.

ECE 232a @ 12:30p (until we get tired).

Head-node is up and serving dhcp, tftp and nfs. It took a while to work around a very broken old PXE stack on the cluster nodes (Tyan MPX). The nodes can boot, but not into anything really useful… memtest. I need to set up some NFS root fs areas for the nodes so that we can boot CentOS on them.

Meeting in ECE 232a to start preliminary ground work for reviving the cluster computer formerly known as Neutron. I’ll be there around 12:30a until we get tired.

A lot of work has been going into the room lately… Two new four post racks hold 0.75T of Fast FC storage, 11 new servers (4-way and 8-way), fibre channel switches.

Hi all,

So the nodes of the cluster are all running, thats the good news, they all have the serv_p4 mpi servers running, that two would be good news, one problem…

Happy Thanksgiving all!