How to migrate svn to gitlab with full history logs

meimei10251314

　　When we are trying to migrate the SCM from svn to gitlab, we also would like to consider the svn logs migration. There are three typical scenarios with tools to complete.

Prerequisite
　　This article only applies to the migrations that, each project in svn will be migrated to gitlab with specified project space, such as:
　　The projectA in svn will be migrated to http://your/gitlab/url/yourgroup/projectA.git
　　Otherwise, the operations described here may not be served for your purpose.

Typical scenarios

Standard svn layout
　　If you are not quite familiar with the svn standard layout, come to this page for reference
　　http://svnbook.red-bean.com/en/1.5/svn.branchmerge.maint.html
　　In summary, it should have similar form as:
　　$repo/paint/trunk
　　$repo/paint/branches
　　$repo/paint/tags
　　$repo/calc/trunk
　　$repo/calc/branches
　　$repo/calc/tags

No standard layout but the path never changed
　　For example, the svn repo has consistent hierarchy
　　$repo/pkg/paint
　　$repo/pkg/calc
　　$repo/branches/paint-skeleton
　　$repo/branches/paint-hotfix
　　$repo/tags/paint/v0.1.0
　　$repo/tags/paint/v0.1.2
　　$repo/tags/calc/v0.1.1

No standard layout and the path was changed before
　　For example, before the svn repo has hierarchy as:
　　$repo/pkg/paint
　　$repo/pkg/calc suite/calc
　　$repo/branches/paint-skeleton
　　$repo/branches/paint-hotfix
　　Later on, the svn repo changed the organization:
　　$repo/trunk/paint
　　$repo/trunk/calc suite/calc
　　$repo/tags/paint-skeleton
　　$repo/tags/paint-hotfix
　　In summary, the original ‘pkg’ was renamed to ‘trunk’; the original ‘branches’ was renamed to ‘tags’; and no ‘branches’ exists any more.

Standard layout Migration

　　Standard layout migration is>　　The subgit is a tool, which is portable and no installation steps.
　　Its official download site is https://subgit.com/download.html
　　1, choose any clean folder as your operation root directory; we call this dir as $root
　　2, in your local svn project directory which you want to migrated, execute command , to get the authors
　　

svn log | awk '($0 ~ /^r/) {print $3}' | sort –u 　　

　　3, switch back to $root, add a new file authormap.txt; this file should contain all mappings between svn(before the equal sign) and gitlab(after the equal sign) authors you listed in step2, example:
　　Nicolas wang = xiaoqiang.wang Xiaoqiang.wang@example.com
　　4, execute command in $root
　　

path/to/subgit-3.2.2/bin/subgit import --non-interactive --default-domain YOUR_DOMAIN --authors-file authormap.txt --trunk trunk --tags tags --branches branches --username SVN_USERNAME --password SVN_PASSWORD --svn-url http://svn.example.com/path/to/repo repo.git　　

　　If any error occurred during the migration and the process was interrupted, execute ‘subgit import repo.git’ to recover
　　5, clone a bare repo and remove the useless svn metadata
　　 git clone --bare repo.git repo-clone.git
　　6, change to directory repo-clone.git
　　 cd repo-clone.git
　　7, push local git repo to remote gitlab repository
　　

git remote add gitlab  http://gitlab.example.com/path/to/repo.git　　

　　
git push gitlab --all
　　

　　
git push gitlab --tags
　　


No standard layout migration and the path was changed before
　　This is the key part of this article. Here we could not use subgit anymore, because this tool will only migrate the logs after renaming pkg->trunk and branches->tags. The logs generated in pkg or branches before, will not be migrated.
　　What we used here is git svn commands.
　　Take the calc mentioned above as an example.
　　1, Choose any clean folder as your operation root directory; we call this dir as $root
　　2, In your local svn project directory which you want to migrated, execute command , to get the authors
　　 svn log | awk '($0 ~ /^r/) {print $3}' | sort –u                  
　　3, Switch back to $root, add a new file authormap.txt; this file should contain all mappings between svn(before the equal sign) and gitlab(after the equal sign) authors you listed in step2, example:
　　

Nicolas wang = xiaoqiang.wang <Xiaoqiang.wang@example.com>　　

　　4, also in $root, execute the command
　　

git svn clone --no-minimize-url --trunk "trunk/calc suite/calc" --authors-file authormap.txt "http://svn/repo/path/" new.git　　

　　5, then execute the command
　　

git svn clone --no-minimize-url --trunk "pkg/calc suite/calc" --authors-file authormap.txt "http://svn/repo/path/" old.git　　

　　The only difference compared to step4, is step5 points to the clone of objects before renaming.
　　6, change to directory old.git, execute command 'git log' and get the last commit’s SHA in log history, such as 56a0f2578fce65a85436453f37535f416a3c6c07
　　7, change to directory new.git,execute command 'git log' and get the last commit’s SHA in log history, such as 63c91bddc647588a3eccf6102df2a4146d447629
　　8, also in new.git, execute commands
　　

git remote add oldstuff ../old.git　　

　　
git fetch oldstuff
　　

　　9, then execute commands
　　

git replace [first-commit-sha-from-new.git] [last-commit-sha-from-old.git]　　

　　
　　10, rewrite the SHA hashes so everything is kosher and you no longer need the replace ref
　　

git filter-branch　　

　　11, clean up the no-longer-needed references
　　

git remote rm oldstuff　　

　　

rm -rf .git/refs/replace　　

　　12, removing git-svn-id messages
　　

git filter-branch -f --msg-filter 'sed -e "/git-svn-id:/d"'     　　

　　Explanation: When migrating from Subversion, every commit messages has a git-svn-id line appended to it like this one:
　　git-svn-id: http://svn/repo/url/trunk@9837 1eab27b1-3bc6-4acd-4026-59d9a2a3569e
　　If you are planning on migrating away from your old Subversion repository entirely, there’s no need to keep these.
　　13, remove empty commit messages
　　

git filter-branch -f --msg-filter '　　

　　
read msg
　　

　　

if [ -n "$msg" ] ; then　　

　　echo "$msg"
　　

　　
else
　　

　　echo "<empty commit message>"
　　

　　
fi'
　　

　　Explanation: Subversion allows empty commit messages, but Git does not. Any empty commit messages in your newly migrated git repository should be replaced so commands like git rebase will work on these commits.
　　14, push local git repo to remote gitlab repository
　　

git remote add gitlab http://gitlab.example.com/path/to/repo.git　　

　　
git push gitlab --all
　　

　　
git push gitlab --tags
　　

　　The only drawback of this solution, is that it will overwrite the first commit of new.git using the last commit of old.git, so the first commit of new.git will lose

No standard layout migration and the path was never changed
　　Either of the above two scenarios could cover this topic, it depends on the actual situation which tool (subgit or git svn) you should use, and it will not be extended here

References
　　http://treyhunner.com/2011/11/migrating-from-subversion-to-git/
　　http://blog.woobling.org/2009/06/git-svn-abandon.html
　　https://stackoverflow.com/questions/12938189/how-do-i-keep-svn-history-in-git-when-trunk-has-moved
　　http://www.jianshu.com/p/29f7f98f0396
　　https://subgit.com/import-book.html
　　https://groups.google.com/forum/#!topic/subgit-users/-2Xon4RIWd8