Categories
git git-submodules

How do I “git clone” a repo, including its submodules?

2452

How do I clone a git repository so that it also clones its submodules?

Running git clone $REPO_URL merely creates empty submodule directories.

4

  • 8

    That command would be git clone --recurse-submodules --remote-submodules (Q3 2019 Git 2.23): it will clone and update the submodules in one command. See my edited answer below.

    – VonC

    Jul 22, 2019 at 15:46


  • 1

    There’s a chance you want to leave off --remote-submodules and just do --recurse-submodules. Beware.

    Dec 23, 2020 at 21:50

  • 4

    @BenjaminBerman Could you elaborate as to why you wouldn’t want to use git clone --recurse-submodules --remote-submodules?

    Mar 5, 2021 at 14:55

  • 3

    @LiamCrowley , the parent (hosting, containing) repo might depend on a particular version of the submodule for a variety of reasons. For example, the maintainers of the host repo might not be ready to deal with updates just yet.

    May 21, 2021 at 12:13

3477

With version 2.13 of Git and later, --recurse-submodules can be used instead of --recursive:

git clone --recurse-submodules -j8 git://github.com/foo/bar.git
cd bar

Editor’s note: -j8 is an optional performance optimization that became available in version 2.8, and fetches up to 8 submodules at a time in parallel — see man git-clone.

With version 1.9 of Git up until version 2.12 (-j flag only available in version 2.8+):

git clone --recursive -j8 git://github.com/foo/bar.git
cd bar

With version 1.6.5 of Git and later, you can use:

git clone --recursive git://github.com/foo/bar.git
cd bar

For already cloned repos, or older Git versions, use:

git clone git://github.com/foo/bar.git
cd bar
git submodule update --init --recursive

26

  • 182

    Is there any way to specify this behavior as default in your git repository, so that less-informed cloners will automatically get an initialized submodule?

    – NHDaly

    Feb 20, 2013 at 6:37

  • 19

    @NHDaly Sadly, no. (Not that I know of, at least.)

    Feb 20, 2013 at 7:25

  • 9

    And logically thinking git clone –recursive will also populate any submodules of a submodule, right?

    – jayarjo

    Apr 10, 2013 at 17:31

  • 13

    I think I come back to this answer once a month… Why don’t git just ask upon cloning if it should also download the submodules?

    Oct 12, 2017 at 21:44

  • 20

    Also to make clear (since I wanted to know and couldn’t find an answer except by looking at the source), the git clone --recursive and --recurse-submodules options behave identically. They result in a call to the same function.

    Nov 28, 2018 at 0:52

602

You have to do two things before a submodule will be filled:

git submodule init 
git submodule update

7

  • 11

    I was afraid of that… it doesn’t make any sense since you’re checking out a partial project in that case. I understand that the submodule updates aren’t automatic, but why isn’t the bound version automatically checked out?? Is there any way to force it? I have a project with 3-levels of submodules and it seems absurd to have to traverse that far just to do a checkout.

    – None

    Sep 26, 2010 at 7:26

  • 13

    Please read the git-submodule(1) man page (kernel.org/pub/software/scm/git/docs/git-submodule.html). You’ll find out that git submodule update supports a nice parameter called --recursive.

    – joschi

    Sep 26, 2010 at 7:30

  • 123

    Why not just do both of them in one command? git submodule update --init (Also see my answer).

    Dec 15, 2010 at 13:04


  • 11

    I think its better to answer the question with these two commands. Its explains better how to accomplish the task.

    – schmijos

    Apr 17, 2013 at 18:20


  • 6

    @MathiasBynens A machine that I just logged into only has git 1.5.5.6, which apparently does not support the shortened instruction, but does support it as two commands.

    Jul 24, 2013 at 18:40

321

Git 2.23 (Q3 2019): if you want to clone and update the submodules to their latest revision:

git clone --recurse-submodules --remote-submodules

If you just want to clone them at their recorded SHA1:

git clone --recurse-submodules

See below.

Note that Git 2.29 (Q4 2020) brings a significant optimization around submodule handling.

See commit a462bee (06 Sep 2020) by Orgad Shaneh (orgads).
(Merged by Junio C Hamano — gitster in commit 2ce9d4e, 18 Sep 2020)

submodule: suppress checking for file name and ref ambiguity for object ids

Signed-off-by: Orgad Shaneh

The argv argument of collect_changed_submodules() contains only object ids (the objects references of all the refs).

Notify setup_revisions() that the input is not filenames by passing assume_dashdash, so it can avoid redundant stat for each ref.

Also suppress refname_ambiguity flag to avoid filesystem lookups for each object. Similar logic can be found in cat-file, pack-objects and more.

This change reduces the time for git fetch(man) in my repo from 25s to 6s.


Original answer 2010

As joschi mentions in the comments, git submodule now supports the --recursive option (Git1.6.5 and more).

If --recursive is specified, this command will recurse into the registered submodules, and update any nested submodules within.

See Working with git submodules recursively for the init part.
See git submodule explained for more.

With version 1.6.5 of git and later, you can do this automatically by cloning the super-project with the –-recursive option:

git clone --recursive git://github.com/mysociety/whatdotheyknow.git

Update 2016, with git 2.8: see “How to speed up / parallelize downloads of git submodules using git clone --recursive?

You can initiate fetching the submodule using multiple threads, in parallel.
For instances:

git fetch --recurse-submodules -j2

Even better, with Git 2.23 (Q3 2019), you can clone and checkout the submodule to their tracking branch in one command!

See commit 4c69101 (19 May 2019) by Ben Avison (bavison).
(Merged by Junio C Hamano — gitster in commit 9476094, 17 Jun 2019)

clone: add --remote-submodules flag

When using git clone --recurse-submodules there was previously no way to
pass a --remote switch to the implicit git submodule update command for
any use case where you want the submodules to be checked out on their
remote-tracking branch rather than with the SHA-1 recorded in the superproject.

This patch rectifies this situation.
It actually passes --no-fetch to git submodule update as well on the grounds they the submodule has only just been cloned, so fetching from the remote again only serves to slow things down.

That means:

--[no-]remote-submodules:

All submodules which are cloned will use the status of the submodule’s remote-tracking branch to update the submodule, rather than the superproject’s recorded SHA-1. Equivalent to passing --remote to git submodule update.

10

  • 10

    So it took Git 14 years to start adding proper support for submodules, huh. Thanks for the update! What if I already have a clone of the main repo without submodules and without a recorded SHA1, and I want to pull in the latest version of each submodule. Is it doable?

    Sep 9, 2019 at 9:48


  • 1

    @VioletGiraffe If that cloned repository has submodules, it has “recorded SHA1”. And a git submodule update --init --recursive --remote should update them to the latest commit of their respective branch. (ex: stackoverflow.com/a/56981834/6309)

    – VonC

    Sep 9, 2019 at 10:14


  • 1

    Let me clarify with an example: I have a template project on Github that uses submodules, and I even commited specific revisions of the submodules into this template repo. But when I create a new project out of this repo, none of the commands you listed (neither clone --recurse-submodules --remote-submodules nor submodule update --init --recursive --remote) let me actually fetch the subrepos. All I get is a .gitmodules file, and I couldn’t find any way to init the subrepos other than manually cloning them one by one. I’d like to at least have a script to do it with submodule foreach

    Sep 10, 2019 at 5:12


  • If you know a solution, I’d ask a separate question that you could answer. Here’s the test repo that I can’t find any way to init other than by hand: github.com/VioletGiraffe/TEST

    Sep 10, 2019 at 5:13


  • 1

    @VioletGiraffe That is because you have added and committed the .gitmodules but not the gitlink (stackoverflow.com/a/16581096/6309, special entries in the index: stackoverflow.com/a/19354410/6309) Here is a repository which does have the proper gitlink registered: github.com/tiagomazzutti/antlr4dart

    – VonC

    Sep 10, 2019 at 6:20