Sunday, June 23, 2013

Master of android repo mirrors

Sometimes you want to mirror an entire android platform repo, locally, so that you can init & sync to a working directory without being connected to the internet. Handy if your internet connection sucks, or you travel a lot. I'm sure you can use your imagination. Updating means you just have to keep track of a list of mirrors and make sure they are synced in order (for reference chaining).

For AOSP, this task is pretty simple, and the first repository you'll probably mirror:

    popd
    mkdir aosp-mirror
    pushd $_
    repo init -u https://android.googlesource.com/mirror/manifest --mirror
    repo sync -j[dl-jobs]

NOTES: $_ in bash is the first argument to the previous command, also dl-jobs is the number of download jobs. popd is the directory we started in. If say, I were on my Mac10.8 and made a case-sensitive sparse disk, and: mkdir /Volumes/Android/mirrors && cd /Volumes/Android/mirrors/ then popd will just return the current directory, as current working directory is the only item on the stack. We pushd to aosp-mirror, init and sync.

Go outside and do something. This will probably take a while.

For CyanogenMod, it's a little bit more difficult. There is no mirror/manifest for CyanogenMod, so we have to do that part ourselves:

    popd
    mkdir cyanogenmod-mirror
    pushd $_
    git ls-remote -h git://github.com/CyanogenMod/android.git > CM-$(date +%Y%m%d%H%M%S).xml
    for branch in $(awk 'BEGIN { FS = "/" } ; { print $3 }' $(ls -t1 CM-*.xml | head -n1)); do \
        repo init -u git://github.com/CyanogenMod/android.git -b ${branch} --mirror --reference /Volumes/Android/aosp-mirror; \
        mv .repo .repo-${branch}; \
        ln -sf .repo-${branch} .repo; \
        repo sync -j4; \
        rm .repo; \
    done

Notice a pattern? {pre,post}-{init,sync}? Yes! pre-init we get a list of branches, post-init we move .repo to be specific for the current branch, and post-sync we remove the symbolic link to the current branch's .repo. The rest is the same, except this repo mirror references the aosp-mirror.

That's pretty much it for now. You can now go up a directory from your aosp-mirror and cyanogenmod-mirror directories (probably in /Volumes/Android/ if you followed my directions) and sync a local working directory:

    cd /Volumes/Android/mirrors/cyanogenmod-mirror/
    ln -sf .repo-cm10.1 .repo
    mkdir -p /Volumes/Android/work/cm-10.1
    cd $_
    repo init -u file:///Volumes/Android/mirrors/cyanogenmod-mirror/ -b cm-10.1
    repo sync

This could be done for a number of other aosp-based android platform repository mirrors.

I'm starting with this post on a series of disconnected android platform development on Mac OS X. It should work on Linux, and Windows/Cygwin/mingw(not tested!)... whatever you use on windows.

I will be working on tutorials for setting up mirror management (soon a tool for doing this, repo-mm on my github account), setting up eclipse for development, and eventually some tips and tricks.

Stay tuned!

2 comments:

  1. This method works fine if all you want is the git repositories in the manifests. Chirayu Desai helped me figure out how to mirror all repositories reported by gerrit with the following gist: https://gist.github.com/bhundven/5848646

    ReplyDelete