git-needed Code
git extension to clone multiple dependent repositories
Status: Beta
Brought to you by:
meinls
Menu
▾
▴
Tree [94ef88] master / History
Read Me
git-needed
https://sourceforge.net/projects/git-needed/
© 2019–2022 Stefan Meinlschmidt
git-needed is an extension command for the git version control system. It is used for cloning several repositories under the same directory that are dependent on each other.
git-needed requires python 2.7 or later or some python 3 and of course git.
git-needed is free software. It is available under the terms of the so-called “MIT license” (see LICENSE.txt).
Use
You have some big project that resides in multiple git repositories. Some of these repositories make sense on their own (e.g. a base library), some cannot be built or used without other repositories (e.g. some application using the base library).
git-needed helps you cloning all repositories you need, one beside the other. All you need to do is declare the dependencies. For this you put a file in the toplevel directory of the dependent repository (i.e. the application) called needed-repos. This file contains the git URLs of all prerequisite repos (i.e. the base library), one per line.
Now you create some toplevel directory where you want to clone the repositories, and in that dir you do
git needed clone url://to/the/dependent/repo.git
This will do the same as git clone url://to/the/dependent/repo.git, then reads the repository’s needed-repos and clones the listed repos, too, recursively.
When later the needed-repos have changed and you need to clone the newly added repos, first make sure that everything already cloned is current, then again do
git needed clone url://to/the/dependent/repo.git
This will not clone what is already there but still read its needed-repos and clone any missing repositories.
For everything else regarding the multiple repos, e.g. pull, stage, commit, push, … use git-needed all which runs the same command on all [1] repositories. For example you can make all repositories current by issuing
git needed all pull
If you want to fix versions, i.e. create a baseline accross repositories so you can checkout matching versions of each repo that belong to each other, use consistent tags and checkout those.
I used to rely on 3rd party functionality for this in the past, suggesting git-multi. Unfortunately the magic it applyies to git’s output breaks interactive commands like mergetool, severely limiting its usefullness. Therefore I have incorporated the functinoality into git-needed, completely refraining from any magic.
Subcommands
git-needed uses subcommands for its different functionalitites. You can always get help on them with
git needed -h
or
git needed <SUBCOMMAND> -h
- clone
- Clone a repository and all its (recursive) dependencies. This starts out as git clone, but then it reads its dependencies from the cloned needed-repos file, clones those repositories, read their dependencies etc.
- all
- Call any git command on all [1] repositories
- list
- List what git-needed considers all. Hint: non-repository directories and repositories neither containing a needed-repos file nor being mentioned in one are ignored.
- whats
- Query about what kind of thing git-needed considers some filesystem paths to be. Use this to find out why something is not included in all.
Options
git-needed supports some command line options, both global and per-subcommand. You can always get help on the global ones with
git needed -h
and on the additional ones of a specific subcommand with
git needed <SUBCOMMAND> -h
Global Options
Aside -h for help the options are:
| -v | Print program version and exit |
| -f <DEPFILE>, --depfile <DEPFILE> | |
| Use the filename <DEPFILE> to read dependencies from instead of needed-repos. | |
These need to be put before the subcommand.
clone Options
First of all any option you pass after the root repo url will get passed to git-clone verbatim. This enables partial clones, picking branches or tags and similar. Note that options that stop git-clone from checking anything out will also stop git-needed from finding any dependencies.
| -n, --dry-run | This disables actual cloning. Instead git-needed will output the commands not executed at the end so that you can issue them manually if needed. Of course if a repository is not getting cloned, its dependencies cannot be determined. Therefore you might want to run this again after manual cloning. |
| -i, --interactive | |
| Ask before cloning. You can use this to only clone certain repositories and leave out others. | |
all Options
| -r, --repos | Also process repositories that are not part of any dependency relation. |
| -d, --subdirs | Process subdirectories regardless whether they are git repositories. |
| -i, --interactive | |
| Ask before calling git. You can use this to only process certain repositories and leave out others. | |
| -k, --keep-going | |
| If git returns an error on one repository, do not stop but keep processing all remaining repositories as if nothing had happened. | |
| -c, --command | Do not prefix the command with git, i.e. run any command. |
Install
When you just become root and unpack the .tbz file into /usr or better /usr/local everything falls into place nicely. Instead you can simply extract git-needed into some directory on your path, and if you also extract git-needed.1 to where section-1-manpages are found even git help will work.
If you just clone the git repository instead you can symlink the files into the paths. Just remember that what’s called git-needed.py in the repo must be git-needed without .py when installed.
| [1] | (1, 2) “All” repositories in this context refers to all subdirectories of the current directory that are a git repository and either declare dependencies or are a declared dependency of another repo from “all”. Repositories independent of any dependencies are ignored as well as non-repository directories, except this can be altered with -r or -d. |
