If you read my previous post about converting Subversion repositories to git, you’ll know that to do a proper Subversion-to-Git transformation on a batch of repositories is going to take some time (what with all that command line typing). I had 142 legacy project Subversion repositories lying around I wanted converted to Git and, since I’m lazy, I pulled on my bash boots and wrote me a script to do the work!
With the git-svn-migrate scripts I wrote, you can batch convert all of your Subversion repositories in just 3 steps. And I’ve GPLed them and put them on GitHub if you’d like to collaborate and improve them; see the git-svn-migrate project page.
0. Download the git-svn-migrate scripts
This isn’t really one of the 3 steps, but obviously you need the scripts. You can either download the latest official release from GitHub or you can get the most recent development release by cloning the repository:
git clone git://github.com/JohnAlbin/git-svn-migrate.git
1. Create a list of Subversion repositories to convert
Create a file called “repository-list.txt” with one Subversion URL per line:
svn+ssh://example.org/svn/awesomeProject
file:///svn/secretProject
https://example.com/svn/evilProject
With this format the name of the project is assumed to be the last part of the URL. So these repostitories would be converted into awesomeProject.git, secretProject.git and evilProject.git, respectively.
If the project name of your repository is not the last part of the URL, or you wish to have more control over the final name of the Git repository, you can specify the repository list in tab-delimited format with the first field being the name to give the Git repository and the second field being the URL of the Subversion repository:
awesomeProject svn+ssh://example.org/svn/awesomeProject/repo
evilproject file:///svn/evilProject
notthedroidsyourlookingfor https://example.com/svn/secretProject
With this format you can use any name for the final Git repo. In the first example above, we’re using the second-to-last part of the URL instead of the last part of the URL. In the second example, we’re just changing the name to all lowercase (recommended). And in the final example, move along. Move along.
2. Create a list of transformations for Subversion usernames to Git committers
Using the repository list created in step 1, run the fetch-svn-authors.sh script to create a list of unique usernames for all the commits in your repositories. The output of the script should be redirected to a file.
./fetch-svn-authors.sh --url-file=repository-list.txt > author-transform.txt
Edit the raw list of Subverions usernames to provide full names and emails suitable for Git committers. The output of the fetch-svn-authors.sh script will be of the form:
username = username
You should edit each line to be:
username = Full name
For example, change:
jwilkins = jwilkins
into:
jwilkins = John Albin Wilkins
3. Convert the Subverion repositories into bare Git repositories
This is the easiest step. To place all of your new bare Git repositories in /var/git, simply run:
./git-svn-migrate.sh --url-file=repository-list.txt --authors-file=author-transform.txt /var/git
This may take a while. (My 142 repos took about 6 hours to convert.) But you’ll see the progress as the underlying git-svn pulls commits out of all of your Subversion repositories.
Enjoy!
Comments15
I just started working with
I just started working with git. Though, it is pretty safe against corruption and all, it is somewhat hard to follow. Git svn migration is painful when you are not thorough with git in the first place. But your three step conversion is totally idiot proof. A life saver it is. Even though it took me quiet a few days to figure it out completely, in the end it was worth it. And hey where can I find an idiot proof tutorial to git?! Mark
This graphic is fantastic!
This graphic is fantastic!
I can't thank you enough for
I can't thank you enough for these scripts. After several unsuccessful tries w/ svn2git (which lost commits!), git-svn-migrate did the job of converting the Minify SVN repo nicely.
Using this with HTTPS SVN repos.
I had problems using this with an HTTPS-based SVN repo, as git-svn doesn't support a non-interactive password prompt. Problem was solved by making a small script which simply does: echo 'password', and running as follows:
GIT_ASKPASS=$PWD/askpass ./git-svn-migrate.sh --url-file=repolist.txt --authors-file=userlist.txt /var/git
Downside is it means (at least temporarily) storing your password on disk, but you can always shred the file later.
I run into a problem where I
I run into a problem where I got following output:
fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions
It happened when git-svn-migrate.sh issued 'git svn show-ignore --id trunk >> .gitignore;' command.
I don't know svn at all, and it took me some time to figure out that my repo url was wrong... I got that url from pjproject (http://www.pjsip.org/download.htm). Removing the /trunk prefix made my day. If anyone got into similar issue, check your svn repo url.
Thanks for those scripts, very useful :-).
Does this method convert the
Does this method convert the entire history or just the latest revision?
Of course! The entire history
Of course! The entire history and any branches and tags you had in Subversion. :-)
John, your scripts literally
John, your scripts literally saved me whole week of svn to git migration for all my svn repositories.
Thank you a lot! No issues encountered during migration!
Small hint: in order to use
Small hint: in order to use "git svn" migration your svn code should at least contain 'trunk'. I had a repo with random files in it's root, w/o any branches or trunk, so before migration had to move it all under 'trunk' subfolder.
Awesome...worked beautifully.
Awesome...worked beautifully. Thanks soooo much for doing the work to put these together, and of course for making it available to everyone. Note that in your instructions on this page, in step 2 there is a small typo: the command should be "./fetch-svn-authors.sh", and not "./fetch-svn-author.sh" - author needs to be plural. Easy enough to fix, but I thought you might want to know.
Worked great! Can I make two
Worked great! Can I make two suggestions? 1) add signal catching for ctrl-c stop execution and 2) allow for username and password to be set at script execution. Thanks for your work here! Saved me a ton of time!
When the script calls 'git
When the script calls 'git svn show-ignore --id trunk >> .gitignore;' I get the following output:
- Converting svn:ignore properties into a .gitignore file...
fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git [...] -- [...]'
rev-list --first-parent --pretty=medium HEAD: command returned error: 128
and when the script calls 'git push bare' I get the following:
- Pushing to new bare repository...
No refs in common and none specified; doing nothing.
Perhaps you should specify a branch such as 'master'.
fatal: The remote end hung up unexpectedly
error: failed to push some refs to '~/DataAnalytics.git'
This is on OSX 10.8.5 with git version 1.8.3.4 (Apple Git-47).
Does anyone have any ideas what is going wrong? I tried messing around with the url as was suggested above but it didn't help.
a small adaptation for the
a small adaptation for the fetch-svn-authors script: when you add these lines at the end of the script and you specify a list of authors that is already populated; only the new users will be added at the end of your list.
# Output to the specified destination file.
cat $tmp_file | sort -u >> $destination;
# remove duplicates from the destination file:
awk '!x[$1]++' $destination > $tmp_file;
cat $tmp_file > $destination;
Hey! I'm work for a very
Hey! I'm work for a very small company, am new to Git and have been asked to migrate all our SVN repositories into Git. I came across this post last week and was given the go-ahead to try it out on one of our smaller repositories. I've been attempting to use your scripts today and have encountered an error that I don't know how to fix when attempting to do step 2: ./fetch-svn-authors.sh: line 150: unlink: command not found
My tmp-authors-transform.txt file ends up being empty.
I'm not sure what I'm doing wrong and would love some assistance.
I'm using Git 1.9.5 for Windows.
Thanks!
This rules! I did a
This rules! I did a Subversion -> Git conversion several years ago at a previous job and remember it being kind of a pain. Now I'm looking at doing another one at my current gig and this makes it SO MUCH EASIER. Thanks for sharing!