A complete idiot’s guide to git-svn-migrate: 3 steps to batch convert Subversion to Git

By John, 1 September, 2010

If you read my previous post about converting Subversion repositories to git, you’ll know that to do a proper Subversion-to-Git transformation on a batch of repositories is going to take some time (what with all that command line typing). I had 142 legacy project Subversion repositories lying around I wanted converted to Git and, since I’m lazy, I pulled on my bash boots and wrote me a script to do the work!

With the git-svn-migrate scripts I wrote, you can batch convert all of your Subversion repositories in just 3 steps. And I’ve GPLed them and put them on GitHub if you’d like to collaborate and improve them; see the git-svn-migrate project page.

Image
svn boxes go into the factory; git ponies come out.
git-svn-migrate: a reverse glue factory

0. Download the git-svn-migrate scripts

This isn’t really one of the 3 steps, but obviously you need the scripts. You can either download the latest official release from GitHub or you can get the most recent development release by cloning the repository:

git clone git://github.com/JohnAlbin/git-svn-migrate.git

1. Create a list of Subversion repositories to convert

Create a file called “repository-list.txt” with one Subversion URL per line:

svn+ssh://example.org/svn/awesomeProject
file:///svn/secretProject
https://example.com/svn/evilProject

With this format the name of the project is assumed to be the last part of the URL. So these repostitories would be converted into awesomeProject.git, secretProject.git and evilProject.git, respectively.

If the project name of your repository is not the last part of the URL, or you wish to have more control over the final name of the Git repository, you can specify the repository list in tab-delimited format with the first field being the name to give the Git repository and the second field being the URL of the Subversion repository:

awesomeProject    svn+ssh://example.org/svn/awesomeProject/repo
evilproject     file:///svn/evilProject
notthedroidsyourlookingfor  https://example.com/svn/secretProject

With this format you can use any name for the final Git repo. In the first example above, we’re using the second-to-last part of the URL instead of the last part of the URL. In the second example, we’re just changing the name to all lowercase (recommended). And in the final example, move along. Move along.

2. Create a list of transformations for Subversion usernames to Git committers

Using the repository list created in step 1, run the fetch-svn-authors.sh script to create a list of unique usernames for all the commits in your repositories. The output of the script should be redirected to a file.

./fetch-svn-authors.sh --url-file=repository-list.txt > author-transform.txt

Edit the raw list of Subverions usernames to provide full names and emails suitable for Git committers. The output of the fetch-svn-authors.sh script will be of the form:
username = username
You should edit each line to be:
username = Full name

For example, change:
jwilkins = jwilkins
into:
jwilkins = John Albin Wilkins @example.org>

3. Convert the Subverion repositories into bare Git repositories

This is the easiest step. To place all of your new bare Git repositories in /var/git, simply run:

./git-svn-migrate.sh --url-file=repository-list.txt --authors-file=author-transform.txt /var/git

This may take a while. (My 142 repos took about 6 hours to convert.) But you’ll see the progress as the underlying git-svn pulls commits out of all of your Subversion repositories.

Enjoy!

Topics: Drupal, Git, subversion

Comments 15

The content of this field is kept private and will not be shown publicly.

Mark (not verified)

10 years 9 months ago

I just started working with git. Though, it is pretty safe against corruption and all, it is somewhat hard to follow. Git svn migration is painful when you are not thorough with git in the first place. But your three step conversion is totally idiot proof. A life saver it is. Even though it took me quiet a few days to figure it out completely, in the end it was worth it. And hey where can I find an idiot proof tutorial to git?! Mark

I had problems using this with an HTTPS-based SVN repo, as git-svn doesn't support a non-interactive password prompt. Problem was solved by making a small script which simply does: echo 'password', and running as follows:

GIT_ASKPASS=$PWD/askpass ./git-svn-migrate.sh --url-file=repolist.txt --authors-file=userlist.txt /var/git

Downside is it means (at least temporarily) storing your password on disk, but you can always shred the file later.

m-ric (not verified)

9 years ago

I run into a problem where I got following output:

fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions

It happened when git-svn-migrate.sh issued 'git svn show-ignore --id trunk >> .gitignore;' command.
I don't know svn at all, and it took me some time to figure out that my repo url was wrong... I got that url from pjproject (http://www.pjsip.org/download.htm). Removing the /trunk prefix made my day. If anyone got into similar issue, check your svn repo url.

Thanks for those scripts, very useful :-).

Small hint: in order to use "git svn" migration your svn code should at least contain 'trunk'. I had a repo with random files in it's root, w/o any branches or trunk, so before migration had to move it all under 'trunk' subfolder.

steve (not verified)

8 years 8 months ago

Awesome...worked beautifully. Thanks soooo much for doing the work to put these together, and of course for making it available to everyone. Note that in your instructions on this page, in step 2 there is a small typo: the command should be "./fetch-svn-authors.sh", and not "./fetch-svn-author.sh" - author needs to be plural. Easy enough to fix, but I thought you might want to know.

Worked great! Can I make two suggestions? 1) add signal catching for ctrl-c stop execution and 2) allow for username and password to be set at script execution. Thanks for your work here! Saved me a ton of time!

Camden Narzt (not verified)

7 years 10 months ago

When the script calls 'git svn show-ignore --id trunk >> .gitignore;' I get the following output:

- Converting svn:ignore properties into a .gitignore file...
fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git [...] -- [...]'
rev-list --first-parent --pretty=medium HEAD: command returned error: 128

and when the script calls 'git push bare' I get the following:

- Pushing to new bare repository...
No refs in common and none specified; doing nothing.
Perhaps you should specify a branch such as 'master'.
fatal: The remote end hung up unexpectedly
error: failed to push some refs to '~/DataAnalytics.git'

This is on OSX 10.8.5 with git version 1.8.3.4 (Apple Git-47).
Does anyone have any ideas what is going wrong? I tried messing around with the url as was suggested above but it didn't help.

Chris Maes (not verified)

7 years 6 months ago

a small adaptation for the fetch-svn-authors script: when you add these lines at the end of the script and you specify a list of authors that is already populated; only the new users will be added at the end of your list.

# Output to the specified destination file.
cat $tmp_file | sort -u >> $destination;
# remove duplicates from the destination file:
awk '!x[$1]++' $destination > $tmp_file;
cat $tmp_file > $destination;

Alanna (not verified)

6 years 5 months ago

Hey! I'm work for a very small company, am new to Git and have been asked to migrate all our SVN repositories into Git. I came across this post last week and was given the go-ahead to try it out on one of our smaller repositories. I've been attempting to use your scripts today and have encountered an error that I don't know how to fix when attempting to do step 2: ./fetch-svn-authors.sh: line 150: unlink: command not found

My tmp-authors-transform.txt file ends up being empty.

I'm not sure what I'm doing wrong and would love some assistance.

I'm using Git 1.9.5 for Windows.

Thanks!

This rules! I did a Subversion -> Git conversion several years ago at a previous job and remember it being kind of a pain. Now I'm looking at doing another one at my current gig and this makes it SO MUCH EASIER. Thanks for sharing!