UNIX pipes and exit status

Suppose you have a shell pipeline:

a | b

But sometimes a fails in a way that makes b still succeed (maybe b is grep -v something?).

Instead, do:

(F=$(mktemp);rm $F;mkfifo $F;a>$F&A=$!;b<$F&rm $F;wait $! $A)

This would be a billion times easier if the shell would just let me execute pipe(). That is all.

Touhou -- a biassed copyright case study

Some of you will know I'm in Japan right now. Here's a story about Japan.

So today I attended the second day of the massive Comic Market exhibition in Tokyo. Here's a picture of the queue at 7:23 am:

Everyone in this photo (except maybe the people walking in the background) has been waiting for about an hour and a half. It'll be another 2.5 hours before they're let in. That's not the entire queue by any means -- most of them arrived after me, and I got there at 6am. There are likely tens of thousands in the queue at this point.

They got there early just to have first dibs on a variety of 'doujin' merchandise -- items created typically by groups of friends (or just individuals) called 'circles', and self-published via various outlets which specialise in doujin media. Comic Market is one such outlet, and runs twice a year for a 3 day weekend each time, with free entry. The attendance is typically over half a million; you can only see the 4000 or so (my rough count) super-early arrivals above. For the West Hall. There were probably 200,000+ attendees in total today.

The exhibition is made up of 3 large halls of space totalling enough to sit (my rough count) 5000 circles per day (with a completely different set each day), plus a commercial goods room with several hundred stands belonging to various commercial entities. (Roughly half the size of one of the other 3 halls, but about twice as full.)

In case you think that's not huge, 'Comiket' is just the largest of many similar exhibitions. The doujin industry is, according to Wikipedia, worth about 800 million dollars per year.

Why am I posting this and mentioning copyright? Because, basically, most of the industry is based around copyright infringement. Very few of the circles have any kind of authorisation for their works, which are frequently based on commercial properties (in the west, this is somewhat analogous to fan-fiction). Those works are normally released in a run of a few hundred, partly to keep a low profile, so popular works are often coveted and fetch a high price after the event -- hence, of course, the early birds pictured.

My particular interest is in the Touhou series of doujin games, mostly by the circle 'Team Shanghai Alice' (there's a few official fighting games by another group, and some official comic books collaborated with others). The primary (perhaps only?) author, 'ZUN,' provides some guidance on what he considers acceptable conduct with his IP (basically, don't create anything that's otherwise illegal and try to keep the games as a primary focus, ie. only allow sale where the games are sold).

Since the rules are so open, the fanworks surrounding the series have exploded. The popular remixes of the music from the games now cover well over 1TiB of TTA-encoded audio. Around 2/3 of one hall at Comiket is self-published fanworks from the series -- that's about 1000 individual circles.

Does that hurt the sales of the games? I'm going to go with 'no', having walked around a one hour queue to the Team Shanghai Alice stall to spend about 8 seconds there despite being foreign and thus slower than the average. I reckon that with the three-wide queue the circle sold about two to three thousand games in that time. At 1000 yen per game, that's about £20K takings. In an hour. The exhibition runs for 6 hours, and the games also sell in many shops outside Comiket (at a higher price, so I guess the circle gets the same profit per sale): I think ZUN's doing OK. Just for reference, here's the queue (behind, then in front -- there's a massive loop that I'll go around after the queue rounds the corner in front of me; the stall is just in front of the last grey bit in the distance behind me).

The game was available illegally on Bittorrent before I got back from the exhibition. Amusingly, as I don't have an optical drive in my laptop, I'm going to take that download if I want to play it. Again, if that hurts sales, I don't think it matters. Hell, the availability online is what ultimately brought out here.

Here's my stash of CDs so far; most of the top and bottom rows I got from shops before Comiket. Everything from the 4th row down is doujin Touhou stuff, including the latest game at the bottom middle. Most of the top half is original doujin music (except for the two EXIT TRANCE albums, 5th in from the left on the 2nd and 3rd rows, which are released by Quake Holdings, a commercial entity).

Geeky Things to Do in Beijing: Anime Shopping: Gulou & Chongwenmen (Soshow)

I went to Beijing last summer on the Experience Go Trip. I wanted to do geeky things, but it seems other people really didn't. I took some photos with the aim of writing a blog post, but never actually wrote it -- and I just found my notes, so let's see what damage I can do.

Disclaimer: These details are almost a year out of date. Beijing is a changing city. If they're wrong, I won't know. They're also written only from vague notes I took at the time.


Gulou is the drum tower in the middle of Beijing. Gulou East Street is the home of a large number of small anime merchanise stores -- cosplay, figures and all sorts. Shops are often very tight -- enough room to walk down a winding selection of goods. It's a reasonable walk from any subway stations, though. Sorry, I didn't take any photos. :(

Suggested route: take Beixinqiao, join Jiaodaokou East St and continue until it becomes Gulou East St (it's pretty obvious when it does as the road narrows considerably). Continue until you see a shop you like (there's quite a few, but the road is quite long).


For more anime merchandise shops (probably the same amount in total, but packed much closer), you can try Soshow at Chongwenmen. Directions: Take Exit C at Chongwenmen Station, go left onto the underpass, and take the right exit. You should see Soshow some distance in front of you:

The first few floors are all fashion. Continue up the escalators until the floor before the food area. You'll know when you've arrived. Here's a few photos:

(It's a lot bigger than this -- there's cosplay, figures, posters, books, games and even dolls on sale at the various shops.)

Geeky Things to Do in Beijing: Gaming Arcades: Shuangjing

I went to Beijing last summer on the Experience Go Trip. I wanted to do geeky things, but it seems other people really didn't. I took some photos with the aim of writing a blog post, but never actually wrote it -- and I just found my notes, so let's see what damage I can do.

Disclaimer: These details are almost a year out of date. Beijing is a changing city. If they're wrong, I won't know. They're also written only from vague notes I took at the time.


Shuangjing is home to the huge 'VIVA' shopping mall. As well as huge numbers of high prices fashion shops, it also houses an alright gaming arcade.

Directions: Take Exit A from Shuangjing underground station (Guangqumen Outer St). Turn right before the final escalator. Enter VIVA. Keep going forwards and taking every upward escalator until you see some neon lights:

That's the arcade. Some photos (sorry, it was closed when I did this photo run):

They have a pretty good and fairly busy fighting game section, plus some rhythm games and the usual fare. Price is 2 RMB per token, 1 token per game, but I think there's some steep discounts for regulars. I didn't hang out here much because the previous two arcades I mentioned were far better.

Geeky Things to Do in Beijing: Gaming/Anime: Zhongguancun

I went to Beijing last summer on the Experience Go Trip. I wanted to do geeky things, but it seems other people really didn't. I took some photos with the aim of writing a blog post, but never actually wrote it -- and I just found my notes, so let's see what damage I can do.

Disclaimer: These details are almost a year out of date. Beijing is a changing city. If they're wrong, I won't know. They're also written only from vague notes I took at the time.


Home of Beijing's technology district and markets, this place is fantastic. You don't need too many directions to find cool stuff here, just go to Zhongguancun station and explore. There's a number of huge electronics markets -- most big buildings around the station house them. You can buy anything from mobile phones to RAM sticks. Examples:

There's also the best gaming arcade I found in Beijing. Directions: Take Exit D (Hilon Electric City), go straight up the stairs and at the first major junction, turn to see e-plaza, pictured below with a huge Alienware logo above it:

Take the escalator in front of the door down to B1. Turn right, and carry on until you exit the building. Turn left into the arcade:

They aren't too great on arcade shooters, but they have tonnes of Rhythm games -- 3 DJ Max Technika machines, a jubeat and a custom DDR machine when I visited -- and that's just the stuff I used. Prices are (I think) 1 RMB per token; 1-2 tokens per game. Some photos:

Go up the indoor stairs and turn right to find a small anime/cosplay market area:

There were often a bunch of quite friendly teenagers playing around the anime area. One of them spoke some English and helped me out with one of the market shop keepers.

There was allegedly a maid cafe around the subway station, but the building it was reputed to be in seemed to have been shut down when I visited. Full of locked doors and stored boxes instead of the anime market I was promised.

Geeky Things to Do in Beijing: Gaming Arcades: Xidan

I went to Beijing last summer on the Experience Go Trip. I wanted to do geeky things, but it seems other people really didn't so I did most of them on my own (cue 'forever alone'). I took some photos with the aim of writing a blog post, but never actually wrote it -- and I just found my notes, so let's see what damage I can do.

Disclaimer: These details are almost a year out of date. Beijing is a changing city. If they're wrong, I won't know. They're also written only from vague notes I took at the time.

Anyway -- I did some hunting for gaming arcades and found 2 good ones, a few OK ones and a small-sized one at a shopping mall in Wudaokou (where we stayed).


One of the two best arcades I found was the 'Coolhoo' arcade at Xidan. This one is on the top floor of the underground '77th Street' shopping mall under Xidan Cultural Square. They had a DDR 8th Mix machine, numerous top-down shooters, Taiko no Tatsujin and plenty of fighting games. It was 1 RMB per token; 1-2 tokens per credit. There's discounts if you buy lots of tokens and some sort of membership club with gifts on your birthday.

There was another arcade some distance from the station in one of the malls, but it was seriously inferior: more expensive and not as good.

Anyway, directions: Exit Xidan underground through exits A or E and turn right, or H and turn left. You'll see the 77th street mall entrance:

Go in, down one flight of steps, but not the second one pictured here:

Now go left when you reach the T junction. Hopefully you'll see this:

The path turns right, and looked like:

Go left at the T junction and then up the escalator. You're there (sorry; was closed when I did my photograph trip)!

Token desk is the one glowing red. DDR machine and fighting games:

You can also get in by taking the escalator down from this street (through some large glass doors on the right):

Or by entering through the mysterious "blood donation" building:

More to come!

darcs in gource!

My housemate was playing around with Gource, which is a visualiser of activity of a revision control system. It doesn't work on darcs (which I use at work), but no matter! It accepts a rather easy to generate custom log format.

Converting a darcs repository to a gource input log file.

darcs log -v \
|grep -v '^    [-+]' \
|perl -ne 'if(/^((?:mon|tue|wed|thu|fri|sat|sun).*?\d{4})  (.*)/i){
    $date = `date +%s -d "$1"`;
    chomp $date;
    undef $file;
}elsif(/hunk (.*) \d+/) {
    print "$date|$name|M|$1\n" if $file ne $1;
}elsif(/addfile (.*)/) {
    print "$date|$name|A|$1\n";
} elsif(/rmfile (.*)/) {
    print "$date|$name|D|$1\n";
}'| \
tee /dev/stderr| \
sort > gource.log

Now just run gource gource.log. Done.


I've been hibernating and suspending my machines a lot to save power recently, but of course this involves all SSH sessions getting dropped. Obviously I can use screen to fix this, but then I have to manually allocate a screen name per session -- and I still have to wait for a timeout.

Solution: ass.



exec autossh $tohost -t -- screen -r -d "$name" '||' screen -e "\\0377\\0377" -S "$name"

You'll need autossh locally, and screen on the target machine. You'll get a new session for each local controlling terminal on each machine you SSH from, which is the sensible behaviour. Syntax is just 'ass hostname'. You can manually force reconnect by killing the autossh process it spawns with -USR1.

Be warned that it'll leak screen sessions if you don't log them out manually (ie. turning off your PC will leak sessions).


Edit: Removed requirement for dc to generate characters for screen escape (see comment below for where I'd used it).

Updating vimperator in Debian

I had another dabble in Debian package management this morning; having updated Iceweasel to 3.6, vimperator had broken. I'm dropping the session I used here more as a reference to myself than anything else. (Using the website's xpi may work for most people, but not when you have over 10 Firefox profiles to install it onto.)

apt-get source iceweasel-vimperator
sudo apt-get install mercurial
hg clone https://vimperator-labs.googlecode.com/hg/ vimperator-labs
cd vimperator-labs/
hg update vimperator-2.3
cd vimperator
make xpi
cd ../downloads/
unzip vimperator_2.3.xpi 
cd chrome/
unzip vimperator.jar 
cd ../..
rm downloads/vimperator/vimperator.jar downloads/vimperator_2.3.xpi 
mv downloads ~/vimperator-2.3
cd ~/vimperator-2.3
cp -r ../vimperator-2.1/debian .
vim debian/changelog # Need to bump version
fakeroot debian/rules binary
sudo dpkg -i ../iceweasel-vimperator_2.3_all.deb

Most of my time was spent figuring out the specifics of how it'd been packaged.

Progcomp Chain, Link 7 Results

In order to prevent this thing filling up my blog and driving away people who perhaps don't care, I'm contemplating setting up a new site of some sort to host it.

Anyway. This problem wasn't very imaginative; I hadn't thought of anything seriously fun by the deadline and rather than let it drag on, put something up to give me at least 4 weeks to think of a new one and pass-the buck. Apologies, then, to anyone who thought it was rather similar to Link 6. I'll try to do better next time. A few people claimed to have found bugs, but it seems they were just misinterpreting the message that was returned on death -- it highlighted jumps, but not the points where they drove their buggy straight into holes.

I let people have a minute to solve the problems, so that automation was utterly unnecessary.

Anyway, 10 people solved the problem, which was to provide essentially an AI for a simple version of the Linux moon-buggy game by Jochen Voss (the real version has LASERS and slightly different physics).



Fice and faux were tied for first place starting, both taking just 1 second to get the page. It seems fice had the better luck in actually solving it, though. His entry was a C# application, as usual, with some funky indenting. Winning submission, after 19 minutes 45 seconds.

It's fully automated. It precomputes which spots can be jumped or driven to, passes them as an array and then recursively evaluated all paths, culling as soon as the buggy explodes. I'm not sure why driving forward is "beating", but it certainly beat the problem, so.


Coming in a whole 30 seconds late, Connorhd's submission scored the next place after 37 minutes. His solution was a Javascript bookmarklet, which obviously makes it the most readable thing ever.

It doesn't help that his code is replicated everywhere, and I don't even know how it detects holes. Eww. Anyway, it's a recursive solution similar to fice's, but doesn't do any precomputation.


Taking an hour and 5 minutes, Faux came in next with a verbose Java solution.

Like fice's, it's fully automated. It contains much copy/paste code, in common with Connor's -- but at least I can read this one! Other than the copious amounts of debugging it presumably produces, there's not much more to say about this one.


Monk got the first page within about 30 seconds, and submitted just over 2 hours afterwards. He wrote a Python solution which uses rather depressingly normal code. It does however produce a tonne of debugging output.

One notable feature is that the ASCII used for the car in the output is a customisable parameter to the solver.


Not content with normality, Jessica made a solution in TCL (63 mins solving time; run as ./buggy.tcl '<landscape'), and then followed up with three C solutions, all tweaks on the previous.

The code is neat and is not very copy-paste at all. Also, I can read it, which I don't remember being the case for TCL before.


Michelle submitted next, with a C solution, which was apparently annoying to write in vim. Heresy!

The code is a basic tree search, which is nicely geometrical. It's manually run, and does no caching. It does contain the phrase "HURR DURR DURR HURR HURR FAIL", however.


Uncertain how long it took him to solve it; my logger claims 33 seconds but I find that unlikely, though he did use Haskell, so it's within reason. Having been reading a Haskell tutorial of late, I can read some of his code!

Colin's code doesn't use enough monads, and I think is longer than my Python solver. I feel he has room to grow as a Haskell coder. It's a non-automated recursive solution which pre-caches valid moves.


Fyorl submitted on Codd, so I don't have an accurate time-to-solution, but he solved after Colin, which is what counts. His PHP code is a fully-automated recursive solution with no caching.

Interestingly, he appears to be describing the problem explicitly as a graph exploration, though unlike Faux's problem before it, the exploration is rather more like a tree. Still, it's nice to see the geometry written out explicitly.


Ryan submitted a Haskell solution just after 25 minutes from first fetch, thus being defeated by Colin in the Haskell subgame by a factor of about 50. His solution is similarly lacking in monads, however. I wonder how these people expect to obtain the respect of their peers?

The code implements a non-automated recursive solution without caching.


Coming in last and also least, since his time between first fetch and solution was only 16 mins 15 secs, Moltenfire rounds off my submissions list with a solution in Java.

His solution is recursive with no caching, and no automation. He opts to reformat the problem string into integers, presumably to increase the memory consumption of his app slightly (or because the symbols are ordered by badness afterwards).

Winner, etc.

Fice has won, and should be setting up a new problem soon. Expect more info on his blog. Moltenfire and Ryan should be quicker at starting. Haskell users need less readable code. Jessica needs to specify whether new solutions overwrite old ones.

My code is of course also available!