Keep Your Source Clean: Don’t Worry About Generated Files

Don’t dirty your source and don’t worry about generated files.

Working with OpenSim recently, I didn’t want to have to worry about checking in generated files by accident, or filtering my git status output in order to determine what I changed and was generated. In this post, I’ll describe how I do this and outline some tools I use to make it easy.

The basic premise in my development style is that I build the project in a separate directory than where the checked out code lives:

# checked out code lives in ~/source/opensim
$ cd ~/source
$ svn co http://opensimulator.org/svn/opensim/trunk opensim

# the workspace is where I build and run the project
$ mkdir ~/source/workspace

Then, to build and run the code:

# first I rsync the code to the workspace, skipping .svn folders
# and other unneeded files
$ rsync -avz --exclude=\.svn [...] ~/source/opensim/ ~/source/workspace/

# then I run prebuild and compile
$ cd ~/source/workspace
$ mono bin/Prebuild.exe /target nant
$ nant

This way I never have to filter through generated files when inspecting or committing my changes, and I don’t commit generated files by mistake.

Tools

I have a set of scripts I use when developing OpenSim on Linux (I also use some of the scripts on Windows with cygwin). The scripts are available on github.

All those above commands are encapsulated in os_build.sh, a single script which takes two arguments, the first is the name of the directory in ~/source/ in which the source
lives, and the second is the name of the directory in ~/source/ in which to build:

# this one command performs the rsync, prebuild and nant steps from above
$ os_build.sh opensim workspace

Another key part is the os_clean_workspace.sh script which removes all build artifacts from the workspace directory, while keeping all configuration files in
place. I use this whenever I need to do a clean build.

My typical session looks like this:

$ cd ~/source/opensim
$ git svn rebase                   # pull the latest changes from upstream
$ vim -p file1 file2 ...           # open the files I'll be working on

... edit source ...

$ os_build.sh opensim workspace    # rsync, prebuild, build

... in another terminal, run & test ...
... make more edits ...

# sync only changed files, rebuild only what's needed
$ os_build_sh opensim workspace

... make big changes requiring clean build ...

# removes everything except config files
$ os_clean_workspace.sh workspace
$ os_build.sh opensim workspace

There are also other helper scripts for cleaning the database, running unit tests, and other tasks. Have a look!

git post-receive Hook

Notes on setting up automated tasks when commits are pushed to a central git repository.

At my job we use git for version control, and I wanted to do a few things whenever we pushed changes to the main tree. It took me a little bit of effort to figure out how it works, and below are my notes.

The post-receive hook is called when the repository receives changes. Let’s assume I have a git repository on my development machine that is a clone of a public git repository on a server. Then, when I run git push in my local repository in order to push my commits to the public repository, the post-receive hook is called on the public repository on the server.

Email Notification

On Debian-based systems there is an example email notification post-receive hook in /usr/share/doc/git-core/contrib/hooks/post-receive-email. It’s also available online from the git repo on github in the contrib/hooks directory.

In the repository that will be receiving commits (the public repository on the server, in the example above), first we need to make the post-receive hook executable:

$ cd /path/to/git/repo/.git/hooks
$ chmod +x post-receive

Then we need to call the post-receive-email script from inside this post-receive hook. Open .git/hooks/post-receive with your favorite editor and make sure this line exists:

. /path/to/post-receive-email

Notice we’re not executing the post-receive-email script, we’re sourcing it.

Next, we need to tell git where to send the email to. The hooks.mailinglist property controls this:

$ cd /path/to/git/repo
$ git config hooks.mailinglist "mailinglist@example.com"

At this stage emails will be sent each time changes are pushed to the public git repository containing a summary of the new changes.

By default, emails refer to the project as “UNNAMED PROJECT”. To change this, edit .git/description.

For further customization options, look inside post-receive-email. For example:

$ cd /path/to/git/repo
# envelopesender? see `man sendmail` for the -f switch
$ git config hooks.envelopesender "git@example.com"
$ git config hooks.emailprefix "[MY PREFIX] "  # note the trailing space

Kicking Off a Buildbot Build

You may also want to have Buildbot start a build from the post-receive hook. Like the email script, the Buildbot git script expects revision hashes on standard input. The post-receive script then needs to be modified a little to pass the hashes it receives via standard input to all the scripts it invokes. Here’s a new copy of post-receive:

while read oldrev newrev refname
do
   echo $oldrev $newrev $refname | /bin/sh /path/to/post-receive-email
   echo $oldrev $newrev $refname | /usr/bin/python /path/to/git_buildbot.py
done

This script first reads oldrev, newrev, and refname from standard input, then pipes those values into post-receive-email and git_buildbot.py. We loop until there is no more input, presumably because another push may happen while we’re executing.

Japan: The Home Stretch

Things often don’t turn out as planned, and this includes our stay in Japan. At the end of April we’ll be moving back to Singapore.

The month of April will be a busy one filled with traveling and being a tourist. This weekend my parents and sister are coming to Tokyo for a visit. We’ll be exploring all of Tokyo and also heading to Kyoto for a couple nights. It looks like they’ll arrive just as the cherries are blossoming in Tokyo. Here’s a couple pictures taken yesterday:

Sakura in Ichigaya

Sakura in Ichigaya

The last weekend in March was also quite busy. On Saturday we checked out the 36th annual Tokyo Motorcycle Show, and happened to visit Roppongi at night where Roppongi Art Night was happening. It was a busy day. On Sunday we joined some friends from my office and their friends for hanami at a park in Asagaya. It was our first hanami, a very pleasant afternoon. Pictures are here, here, and here.

Spring Snowboarding at Nakazato

A weekend well spent in the sun, working on our mountain carving skills.

This past weekend some people from Shux’s work went on their annual ski/snowboard trip, and we tagged along. This year the destination was mount Nakazato, near the skiing mecca of Echigo-Yuzawa in Niigata prefecture.

Unlike last year, we had extremely warm weather. Too warm, perhaps; the snow was wet and sticky and heavy. Not the greatest.

It was surprisingly not busy (particularly for Japan), I think I waited in a short queue for the lift about twice the whole weekend. Maybe the weather put most people off? The ski resort also celebrated their 50th anniversary that Saturday night with a pretty fireworks display.

Both of us worked on our snowboarding over the weekend, and we’ve made some progress. I can turn most of the time, and Shux is excellent on her heel edge.

Check out some pictures in the usual place. Was a lot of fun! If all goes well, I’ll head out another time or two this spring.

Early Spring this Year?

Just popping up to say “Hi, spring is coming?”

We’ve been having some warm weather recently, and it looks like the cherry blossoms might be early this year. We’ve seen a few trees with pink flowers blooming already around our neighborhood:

Sakura in February?

With spring seemingly just around the corner, I’ve been doing some spring cleaning. Today I went through the pictures I took with my mobile and uploaded some to Zooomr.

I haven’t been blogging much lately, but I do post updates to Tiwtter (so I have been microblogging). Check it out!

Sumiko

Who is Sumiko? Mike finally gets to the bottom of it.

One Saturday in October Shux and I headed out of town a little ways to an American-style shopping mall. At one of the clothing stores two songs were being played on repeat over the loudspeakers. The songs were in Japanese, so I asked what it was, thinking I’ll go look for the CD later on. They said it was Sumiko, an anime (cartoon) character.

There’s tons of anime characters in Japan, and I assumed this was yet another one. But I couldn’t find anything on Sumiko on the net (as it turns out, I was looking for ใ‚นใƒŸใ‚ณ while her name is ใ‚นใƒŸๅญ). Even worse, I couldn’t find anything at the record stores either. I thought it’d be easy, given the amount of publicity the songs were getting at that clothing store.

This week I was at the Tower Records in Shinjuku and decided to get to the bottom of this matter. I asked at the information counter about this Japanese anime character named Sumiko… and after a few minutes of the lady Googling, we realized Sumiko was owned by that clothing store, and the merchandise was only available there.

Now that I know how to spell her name, though, I was able to find the music videos online. Enjoy!




Summer 2008 Roundup

Yep, it’s been a while. In this post, we round up events from the summer and fall of 2008. Lots of pictures!

So the summer has come and gone, and boy was it a hot one in Tokyo. Now winter’s upon us, and it’s still pretty warm. Quite unseasonably warm, the locals say.

Let’s have a look at what’s happened since June, shall we?

In July we checked out Kawagoe, a little town north of Tokyo. We began with a visit to an American-style diner that served great burgers. Unfortunately, these burgers made me miss two days of work. But they were delicious.

In the evening we went to the BayStars/Giants game at the Tokyo dome. This was actually our first baseball game in Japan, and our first exposure to the baseball “spirit” the fans show game in, game out.

Pictures from Kawagoe and the Giants game are online.

A couple weekends later we headed to Yokohama for one of the first fireworks displays of the season. In Japan, fireworks displays are held regularly throughout the summer, and lots of people come out to watch. Everyone brings a blanket or tarp and picnic food and beer. Many are dressed in the traditional yukata. The fireworks are great too, lasting about an hour. It was probably an ad for the summertime fireworks that Shux saw in late 2007 that led us to Yokohama for New Year’s Eve fireworks.

Pictures from the fireworks display are also online.

August came and went, and in September we took some time off work and headed west to check out western Honshu. We stopped by Osaka, Kyoto, Kobe, Himeji, Hiroshima and Miyajima island. Along the way we ate excellent food, saw tons of cool temples, old Japan in Kyoto, and more Japanese baseball. Great time.

Pictures from the trip are — you guessed it — online.

After that we’ve just been working and otherwise spending time exploring Tokyo and area. One Sunday we headed to Yoyogi Park to hang with the rockabillies. We met up with Yuki, Shux’s friend from long ago. We also caught the last regular season game the Yakult Swallows played, after which three guys retired in a teary ceremony.

We had plans to do paragliding close to Mount Fuji. The rain looked threatening, but we still started our trip early on to make it to the paragliding place on time for our flights. On the way, we received a call that, due to the heavy rain at the site, all flights were off. Our thirst for adventure was not to be thwarted, though, as we changed our plans to drive down the Izu Peninsula to Shimoda instead. It was a long drive, and after eating some of the freshest sushi ever we didn’t get much time to visit the town, but it turned out to be a fun trip nonetheless.

We also visited Mount Takao, not too far from Tokyo. The summit became very popular in the last year or so after it was mentioned in the Michelin guide for some reason. From the summit, which is easy to get to, we were treated with a nice vew of Mount Fuji:

Mt. Fuji from Mt. Takao

Mt. Fuji from Mt. Takao

When Shux’s coworker was in Tokyo for a business trip, we went to Hakone for a day. It was a long weekend, and a hugely popular destination for many Tokyoites. We spent lots of time waiting in line for the bus, for the train, for the cablecar, and just barely caught our train back into the city. It was all worth it, though, for some nice sunset views:

Sunset in Hakone

Sunset in Hakone with Mt. Fuji

Sunset in Hakone with Mt. Fuji

Since then we’ve been doing shopping, getting ready for Christmas in Canada. Can’t wait!