18.12.09

Convert etckeeper repository from Bazaar to Git

Quite a while ago I've installed etckeeper and changed the configuration to use Bazaar as its backend for myself (and sponsored/helped with a patch to change the default in Ubuntu).

However, already the first comment asked me why I would be using you Bazaar, if Git was that much faster (and required less space).

At that time I've thought, that Bazaar would catch up, and they (luckily) have done so in some areas, but Git is still a lot faster.

Therefore I've decided to change the repository from bzr to git. I've done so on my home machine and will do so on my dedicated server boxes in the next days, so it's a not only a recommendation but also documentation.

Howto convert a bzr repository to git (etckeeper)

Open a root shell, then you should export $GIT_DIR first:

export GIT_DIR=/etc/.git

The following will then convert /etc from a bzr to a git repository ("fast-export" is included in bzr-fastimport - you may have to install this first):

bzr fast-export --export-marks=$GIT_DIR/bzr.mark /etc | git fast-import --export-marks=$GIT_DIR/git.mark

After this (which will take a while depending on your history) you want to adjust the VCS setting in etckeeper.conf  (uncomment VCS=git and comment VCS=bzr):

sed -i -r -e s/'#\s*(VCS="git)"'/'\1'/ -e s/'VCS="bzr"'/'# \0'/ /etc/etckeeper/etckeeper.conf

I don't remember correctly, but the Git repository was not really setup correctly in the end - but the following fixed it:

etckeeper init

Benchmarks

I've done some benchmarking, comparing "$DVCS status" against each other. This will look if there are any files modified in the current tree, and it's what etckeeper does before and after upgrading (IIRC).
The following shows the commands with cold and warm caches (I've cleared the disk caches when changing to a new set).

What you can see is not only that "git status" is twice as fast with a cold cache (and even more with a warm one), but also that bzr takes even more time to print a single line of "It sure does!".

$ sudo time git st
0.01user 0.04system 0:05.61elapsed 1%CPU (0avgtext+0avgdata 0maxresident)k
29232inputs+688outputs (113major+1605minor)pagefaults 0swaps
$ sudo time git st
0.02user 0.02system 0:00.06elapsed 59%CPU (0avgtext+0avgdata 0maxresident)k
16inputs+688outputs (0major+1716minor)pagefaults 0swaps

$ sudo time bzr st
0.14user 0.06system 0:11.74elapsed 1%CPU (0avgtext+0avgdata 0maxresident)k
17840inputs+8outputs (36major+3750minor)pagefaults 0swaps
$ sudo time bzr st
0.16user 0.03system 0:00.32elapsed 62%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+3787minor)pagefaults 0swaps

$ sudo time bzr rocks
It sure does!
0.20user 0.05system 0:08.19elapsed 3%CPU (0avgtext+0avgdata 0maxresident)k
17392inputs+8outputs (34major+3422minor)pagefaults 0swaps
$ sudo time bzr rocks
It sure does!
0.16user 0.01system 0:00.20elapsed 92%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+3458minor)pagefaults 0swaps

I'll update this post when scripting for the update of my various OpenVZ containers, but the basic information should stand.

From now on, I will save 5+ seconds on any "aptitude safe-upgrade". Hopefully this multiplies somehow, so the time investment into this post pays back.. ;)

2 comments

Comment from: Nate [Visitor]
NateIf the package for etckeeper in Ubuntu Lucid does not list bzr as a dependency, it should not default to bzr. You can argue about which cm backend you wish to use, but it should work out of the box when you do an 'apt-get install etckeeper'

I think the correct behavior is to see whats installed and use one of them.



2010-09-07 @ 22:46
Comment from: Andre [Visitor]
AndreI was having severe bzr performance issues on one of my servers with etckeeper.

Found that what was problematic was some dynamic files (one unmanaged logfile in particular) that was the real issue. Updating the .bzrignore and removed those files from the repository helped muchly.

I took the plunge and moved to git anyways, since after suffering much slowness, I'll take all the performance I can get.

Thanks for the export/import tip. My bzr2git.sh now looks like this:


#!/bin/sh
export GIT_DIR=/etc/.git

git init
bzr fast-export --export-marks=$GIT_DIR/bzr.mark /etc | git fast-import --export-marks=$GIT_DIR/git.mark

cp /etc/.bzrignore /etc/.gitignore
echo '.bzr/*' >> /etc/.gitignore

I follow up by doing an 'etckeeper commit' after updating the etckeeper.conf file.
2012-03-04 @ 23:59

Leave a comment


Your email address will not be revealed on this site.
(Line breaks become <br />)
(Name, email & website)
(Allow users to contact you through a message form (your email will not be revealed.)
This is a captcha-picture. It is used to prevent mass-access by robots.
Please enter the characters from the image above. (case insensitive)
You can just use your OpenID to provide your name, e-mail and url.
Seitenleiste