Keep Your Source Clean: Don’t Worry About Generated Files

Don’t dirty your source and don’t worry about generated files.

Working with OpenSim recently, I didn’t want to have to worry about checking in generated files by accident, or filtering my git status output in order to determine what I changed and was generated. In this post, I’ll describe how I do this and outline some tools I use to make it easy.

The basic premise in my development style is that I build the project in a separate directory than where the checked out code lives:

# checked out code lives in ~/source/opensim
$ cd ~/source
$ svn co http://opensimulator.org/svn/opensim/trunk opensim

# the workspace is where I build and run the project
$ mkdir ~/source/workspace

Then, to build and run the code:

# first I rsync the code to the workspace, skipping .svn folders
# and other unneeded files
$ rsync -avz --exclude=\.svn [...] ~/source/opensim/ ~/source/workspace/

# then I run prebuild and compile
$ cd ~/source/workspace
$ mono bin/Prebuild.exe /target nant
$ nant

This way I never have to filter through generated files when inspecting or committing my changes, and I don’t commit generated files by mistake.

Tools

I have a set of scripts I use when developing OpenSim on Linux (I also use some of the scripts on Windows with cygwin). The scripts are available on github.

All those above commands are encapsulated in os_build.sh, a single script which takes two arguments, the first is the name of the directory in ~/source/ in which the source
lives, and the second is the name of the directory in ~/source/ in which to build:

# this one command performs the rsync, prebuild and nant steps from above
$ os_build.sh opensim workspace

Another key part is the os_clean_workspace.sh script which removes all build artifacts from the workspace directory, while keeping all configuration files in
place. I use this whenever I need to do a clean build.

My typical session looks like this:

$ cd ~/source/opensim
$ git svn rebase                   # pull the latest changes from upstream
$ vim -p file1 file2 ...           # open the files I'll be working on

... edit source ...

$ os_build.sh opensim workspace    # rsync, prebuild, build

... in another terminal, run & test ...
... make more edits ...

# sync only changed files, rebuild only what's needed
$ os_build_sh opensim workspace

... make big changes requiring clean build ...

# removes everything except config files
$ os_clean_workspace.sh workspace
$ os_build.sh opensim workspace

There are also other helper scripts for cleaning the database, running unit tests, and other tasks. Have a look!

git post-receive Hook

Notes on setting up automated tasks when commits are pushed to a central git repository.

At my job we use git for version control, and I wanted to do a few things whenever we pushed changes to the main tree. It took me a little bit of effort to figure out how it works, and below are my notes.

The post-receive hook is called when the repository receives changes. Let’s assume I have a git repository on my development machine that is a clone of a public git repository on a server. Then, when I run git push in my local repository in order to push my commits to the public repository, the post-receive hook is called on the public repository on the server.

Email Notification

On Debian-based systems there is an example email notification post-receive hook in /usr/share/doc/git-core/contrib/hooks/post-receive-email. It’s also available online from the git repo on github in the contrib/hooks directory.

In the repository that will be receiving commits (the public repository on the server, in the example above), first we need to make the post-receive hook executable:

$ cd /path/to/git/repo/.git/hooks
$ chmod +x post-receive

Then we need to call the post-receive-email script from inside this post-receive hook. Open .git/hooks/post-receive with your favorite editor and make sure this line exists:

. /path/to/post-receive-email

Notice we’re not executing the post-receive-email script, we’re sourcing it.

Next, we need to tell git where to send the email to. The hooks.mailinglist property controls this:

$ cd /path/to/git/repo
$ git config hooks.mailinglist "mailinglist@example.com"

At this stage emails will be sent each time changes are pushed to the public git repository containing a summary of the new changes.

By default, emails refer to the project as “UNNAMED PROJECT”. To change this, edit .git/description.

For further customization options, look inside post-receive-email. For example:

$ cd /path/to/git/repo
# envelopesender? see `man sendmail` for the -f switch
$ git config hooks.envelopesender "git@example.com"
$ git config hooks.emailprefix "[MY PREFIX] "  # note the trailing space

Kicking Off a Buildbot Build

You may also want to have Buildbot start a build from the post-receive hook. Like the email script, the Buildbot git script expects revision hashes on standard input. The post-receive script then needs to be modified a little to pass the hashes it receives via standard input to all the scripts it invokes. Here’s a new copy of post-receive:

while read oldrev newrev refname
do
   echo $oldrev $newrev $refname | /bin/sh /path/to/post-receive-email
   echo $oldrev $newrev $refname | /usr/bin/python /path/to/git_buildbot.py
done

This script first reads oldrev, newrev, and refname from standard input, then pipes those values into post-receive-email and git_buildbot.py. We loop until there is no more input, presumably because another push may happen while we’re executing.

Getting With the Times

You can find Mike in various places across the world wide web. Read the post for more details 🙂

Web 2.0Recently I’ve buckled to the immense peer pressure and joined Facebook. Once that happened, I went on a Web 2.0 spree, joining a bunch of other services. Here they are, with links to my profiles:

If you’re using any of those services, add me or link to me or whatever is appropriate in the given context if you like. See you in cyberspace 🙂

Nerding Out for Serious!

Mike’s getting his nerd on!

I’m going to have a super geeky two days this weekend. I’m participating in code::XtremeApps::, a 24 hour programming competition.

In a nutshell, teams of up to 3 people sign up to take part. At 12:00 noon on Saturday we are given a general theme and have 24 hours to complete a web application that is somehow tied to this theme. The weapons of choice are either Ruby on Rails or Java compiled with the Google Web Toolkit. We’re not touching Java.

This is my first programming competition and should prove to be pretty fun. I’ve teamed up with two co-workers to go head-to-head with the other 72 teams. Neither of us has any experience with Ruby on Rails (or Ajax) beyond poking at it out of curiosity months ago, though we’ve played with it in recent weeks to get up to speed.

Keeping with the nerdy theme, I will (try to) stop sending email notifications to everyone when I write about computers and other geeky stuff like that. If you’re still interested, subscribe to the RSS feed 🙂

Big Boss Man’s Watching You!

Friendly public service announcement: don’t waste time at work!

Big brother is watchingWell, maybe your boss isn’t, but they easily could. This article describes software which managers can use to

track workers’ activity not only on the network or in the browser, but also in email, chatrooms, applications, and shared files. And at any unannounced moment, a manager can capture an employee’s screen, read it, and even record it for posterity.

In the article they mention that this was recently installed at a corporation. Once the employees were notified of this, internet usage dropped 90%. Astounding!

Now your company may not have this particular software installed but don’t be fooled into thinking they don’t already read your email.

So think twice before writing that gossipy letter or checking how your e*trade portfolio is doing. You never know what power hungry manager or ethically-challenged sysadmin might be “lurking” over your shoulder.

Back to the Dark Ages

Mike’s company has decided to use torches to light interior spaces because today’s modern lights might electrocute people!

Earlier this year an Adobe Reader cross-site scripting vulnerability was discovered. All pre-SP2 Internet Explorer versions are affected as is Firefox. Adobe has already fixed the issue with Adobe Reader versions 6.0.6, 7.0.9 and 8.0.

Today I received an email from our internal computer security department that Firefox has been banished from the organization. Previously it was on a list of allowable freeware with other programs like iTunes (iTunes? officially sanctioned at work?) and such. Not anymore.

If you’ve ever used Firefox you’re probably aware of the immense improvement it is over IE (IE7 not yet approved). The tabbed browsing, /search and quick search bookmarks are essential for my workflow way of life! Especially at work.

I always have many pages open, all kinds of information, the countless forms I always have to fill out to document every little thing I do, instructions… forcing me to use IE for this kind of work is like they forced left-handed kids to write with their right hands at school in the 50s! Why not cut one of my arms off altogether, I’ll probably be just as efficient.

I checked the version of Adobe Reader installed on my machine and it’s already version 7.0.9, meaning I’m technically not vulnerable. But why is the official decision to remove Firefox across the organization instead of update the vulnerable program, here Adobe Reader?

I emailed the computer security team with this question. I wonder what their rationale is. I’m sure they’re experts and they know what they’re doing and they have their (very good) reasons but they sure didn’t explain these in their bulletin.

And I thought work would get better…

WWM Now Supports OpenID

Find out what OpenID is and try it out today 🙂

I installed the OpenID plugin on my blog. What is this? How does it affect you? Read on…

OpenID provides you with one identifier and password that you can use anywhere OpenID is accepted. How does it work? You sign up for a URI (basically a URL or web address) with an OpenID server (for free) such as MyOpenID. Whenever you need to identify yourself to an OpenID enabled site or service (such as when posting comments here), you put in your URI (for example mike.myopenid.com). This site will contact your OpenID provider (myopenid.com) which will ask you to put in your password. If your info checks out, the OpenID provider will tell the site asking about you that you’re kosher and grants you access.

What’s the big deal? Today, you probably identify yourself to many websites and services with a different user name and password for each. After a while you may forget your passwords, especially for those sites you only visit once in a while. Some companies have tried to provide a solution to this; some time ago Microsoft tried to get Passport (now Windows Live ID) to be the de-facto authentication system for the internet. Yahoo has Browser Based Authentication and Google has their Google Accounts Authentication. I don’t know about you but I wouldn’t trust my entire online ID to one single entity.

So how does OpenID solve this Big Brother problem? The beauty is that there can be many autonomous OpenID providers. If you own a domain, you can even roll your own (mike.beachtemple.com could be me)!

As a user, you have one URI to remember, one password, and one identity Internet wide. If you come across a cool new website you want to sign up for, you won’t find your user name taken.

Go ahead, try it out. Go register for an OpenID identity at MyOpenID or any of the other providers, and try posting a comment 😀 Then explore some of the OpenID enabled sites.