Skip to main content

Using repo-mm's gerrit-mirror.sh

Overview

From my previous post on the topic of using a local mirror to do android platform development while not connected to the internet, I've started working on a small shell project called repo-mm.

On the road to completing repo-mm's goal, i've created a small component called gerrit-mirror.sh.
Eventually, this will be a part of repo-mm's script, but this is a WIP...

In this post, we'll sync AOSP and CyanogenMod, and setup a cm-10.1 working directory. I'll also show some manual steps that will be later managed by repo-mm.

The scope does not include building CyanogenMod. Their wiki does a great job with that.

Install and Setup

First step is to get the repo-mm repository and fetch the submodules. I use Mac OS X 10.8.4 and homebrew. But this should also work on Linux and Windows[Cygwin/MinGW].

On Mac OS X, don't forget to make a case-sensitive disk image! I called mine Android. Go figure. I recommend making a large sparse image. This way you only use up the space you use. You may also want to consider keeping your mirror on a separate image from your working directory.
I use the default bash from Mac:

GNU bash, version 3.2.48(1)-release (x86_64-apple-darwin12)
Copyright (C) 2007 Free Software Foundation, Inc.

The image is the top level of my android development work.
cd /Volumes/Android
mkdir mirrors work

Get and setup repo-mm in your path:
git clone https://github.com/bhundven/repo-mm
cd repo-mm
git submodule init
git submodule update
export PATH="$(pwd):$PATH"
cd ..

Sync mirrors, then go outside or do something for a long time.

The next step is the most time consuming. Syncing the mirrors. On the plus side, once you have a full set of mirrors downloaded, updating takes little time/bandwidth (if you update regularly... maybe with cron! That exercise is left up to the reader).
cd mirrors
gerrit-mirror.sh https://android-review.googlesource.com https://android.googlesource.com
gerrit-mirror.sh http://review.cyanogenmod.org https://github.com

We still have a few repositories we need to manually clone, because a few of them (for some reason) are not on github.com/CyanogenMod, but are are available on the gerrit server:
git clone --mirror http://review.cyanogenmod.org/CyanogenMod/CMStatsServer.git CyanogenMod/CMStatsServer.git
git clone --mirror http://review.cyanogenmod.org/CyanogenMod/android_device_htc_m7.git CyanogenMod/android_device_htc_m7.git
git clone --mirror http://review.cyanogenmod.org/CyanogenMod/android_device_htc_m7wls.git CyanogenMod/android_device_htc_m7wls.git

The following repository seems to be corrupt (at the time of writing this post) on github:
git clone --mirror http://review.cyanogenmod.org/CyanogenMod/android_kernel_samsung_aries.git CyanogenMod/android_kernel_samsung_aries.git

This last one isn't in cm gerrit, but is needed by cm-10.1:
git clone --mirror http://github.com/CyanogenMod/android_external_google.git CyanogenMod/android_external_google.git

So now you have a few tools at hand to sync personal repositories that you might add to your local_manifest.xml, just clone with --mirror to the right directory/repository.git you will use, for instance:
git clone --mirror https://github.com/TheMuppets/proprietary_vendor_samsung.git TheMuppets/proprietary_vendor_samsung.git

Setup your working directory

Now that your mirror is synced, you can create a new working directory, completely offline:
cd ../work
mkdir cm-10.1
cd cm-10.1
repo init -u file:///Volumes/Android/mirrors/CyanogenMod/android.git --reference /Volumes/Android/mirrors -b cm-10.1 --repo-url=file:///Volumes/Android/mirrors/tools/repo.git
repo sync

You didn't even need to pass download jobs to repo sync. All referenced locally, so very fast.

In Hindsight (it is 20/20)

The mirrors directory ended up being 67G. My cm-10.1 working directory is 6.4G. My sparse image is only 100G, so I think I'm going to move the mirrors directory to the root of the volume and rename the image to AndroidMirrors. Then create a new image for each project.
Updating repositories that were not tracked by gerrit means just doing a fetch on all '*\.git' directories in the mirrors directory instead of only fetching the ones in gerrit. I smell some commits brewing up. :D

I'll write a new post on this topic when I get some headway on the repo-mm script.

Comments

  1. UnknownJune 30, 2013 at 1:54:00 AM PDT

    1. it's android-review, not review-android (.googlesource.com)
    2. CyanogenMod/android_kernel_samsung_aries does exist on GitHub, and it's used by many devices (Galaxy S and Galaxy Tab family, first generation).

    ReplyDelete
  2. bhundvenJune 30, 2013 at 2:00:00 AM PDT

    (future blog post note: don't post when tired)

    1) typo fixed

    2) I forgot to put a separate reason for aries.

    Thanks for the corrections!

    ReplyDelete
  3. bhundvenJune 30, 2013 at 2:08:00 AM PDT

    I think the bug for aries issue might be filed under the wrong category. Should I invalidate it and open a new one issue in the right place? Or is it possible to move/rename it?

    ReplyDelete

Post a Comment

Popular posts from this blog

Master of android repo mirrors

Sometimes you want to mirror an entire android platform repo, locally, so that you can init & sync to a working directory without being connected to the internet. Handy if your internet connection sucks, or you travel a lot. I'm sure you can use your imagination. Updating means you just have to keep track of a list of mirrors and make sure they are synced in order (for reference chaining). For AOSP, this task is pretty simple, and the first repository you'll probably mirror: popd mkdir aosp-mirror pushd $_ repo init -u https://android.googlesource.com/mirror/manifest --mirror repo sync -j[dl-jobs] NOTES: $_ in bash is the first argument to the previous command, also dl-jobs is the number of download jobs. popd is the directory we started in. If say, I were on my Mac10.8 and made a case-sensitive sparse disk, and: mkdir /Volumes/Android/mirrors && cd /Volumes/Android/mirrors/ then popd will just return the current directory, as current w...
1 comment
Read more

What the SHELL!?!?

Have you ever had that lingering question of: What else can I do with the SHELL environment variable in GNU/Make? Well, I did and tried a few different little tests and figured I'd share them with you. If you do much work with GNU/Make, you've probably seen some projects specifically use /bin/bash instead of /bin/sh. But outside of that, maybe you've never seen SHELL set to anything else. I have two new favorite features in GNU/Make 3.82: .SHELLFLAGS .ONESHELL:  .SHELLFLAGS The argument(s) passed to the shell are taken from the variable .SHELLFLAGS . The default value of .SHELLFLAGS is -c normally, or -ec in POSIX-conforming mode. .ONESHELL: Sometimes you would prefer that all the lines in the recipe be passed to a single invocation of the shell. Lets have some fun! Lets say instead of using a posix shell for our Makefile recipe, lets use Python instead! SHELL = /usr/bin/python .PHONY: all clea...
Post a Comment
Read more