Sparse Directories
Prev | Chapter 3. Advanced Topics | Next

## Sparse Directories

By default, most Subversion operations on directories act in a recursive manner. For example, svn checkout creates a working copy with every file and directory in the specified area of the repository, descending recursively through the repository tree until the entire structure is copied to your local disk. Subversion 1.5 introduces a feature called sparse directories (or shallow checkouts) that allows you to easily check out a working copy—or a portion of a working copy—more shallowly than full recursion, with the freedom to bring in previously ignored files and subdirectories at a later time.

For example, say we have a repository with a tree of files and directories with names of the members of a human family with pets. (It's an odd example, to be sure, but bear with us.) A regular svn checkout operation will give us a working copy of the whole tree:

    $ svn checkout file:///var/svn/repos mom
    A    mom/son
    A    mom/son/grandson
    A    mom/daughter
    A    mom/daughter/granddaughter1
    A    mom/daughter/granddaughter1/bunny1.txt
    A    mom/daughter/granddaughter1/bunny2.txt
    A    mom/daughter/granddaughter2
    A    mom/daughter/fishie.txt
    A    mom/kitty1.txt
    A    mom/doggie1.txt
    Checked out revision 1.
    $

Now, let's check out the same tree again, but this time we'll ask Subversion to give us only the topmost directory with none of its children at all:

    $ svn checkout file:///var/svn/repos mom-empty --depth empty
    Checked out revision 1
    $

Notice that we added to our original svn checkout command line a new `--depth` option. This option is present on many of Subversion's subcommands and is similar to the `--non-recursive` (`-N`) and `--recursive` (`-R`) options. In fact, it combines, improves upon, supercedes, and ultimately obsoletes these two older options. For starters, it expands the supported degrees of depth specification available to users, adding some previously unsupported (or inconsistently supported) depths. Here are the depth values that you can request for a given Subversion operation:

`--depth empty`

  Include only the immediate target of the operation, not any of its file or directory children.

`--depth files`

  Include the immediate target of the operation and any of its immediate file children.

`--depth immediates`

  Include the immediate target of the operation and any of its immediate file or directory children. The directory children will themselves be empty.

`--depth infinity`

  Include the immediate target, its file and directory children, its children's children, and so on to full recursion.

Of course, merely combining two existing options into one hardly constitutes a new feature worthy of a whole section in our book. Fortunately, there is more to this story. This idea of depth extends not just to the operations you perform with your Subversion client, but also as a description of a working copy citizen's ambient depth, which is the depth persistently recorded by the working copy for that item. Its key strength is this very persistence—the fact that it is sticky. The working copy remembers the depth you've selected for each item in it until you later change that depth selection; by default, Subversion commands operate on the working copy citizens present, regardless of their selected depth settings.

| Tip
You can check the recorded ambient depth of a working copy using the svn info command. If the ambient depth is anything other than infinite recursion, svn info will display a line describing that depth value:
    $ svn info mom-immediates | grep "^Depth:"
    Depth: immediates
    $

Our previous examples demonstrated checkouts of infinite depth (the default for svn checkout) and empty depth. Let's look now at examples of the other depth values:

    $ svn checkout file:///var/svn/repos mom-files --depth files
    A    mom-files/kitty1.txt
    A    mom-files/doggie1.txt
    Checked out revision 1.
    $ svn checkout file:///var/svn/repos mom-immediates --depth immediates
    A    mom-immediates/son
    A    mom-immediates/daughter
    A    mom-immediates/kitty1.txt
    A    mom-immediates/doggie1.txt
    Checked out revision 1.
    $

As described, each of these depths is something more than only the target, but something less than full recursion.

We've used svn checkout as an example here, but you'll find the `--depth` option present on many other Subversion commands, too. In those other commands, depth specification is a way to limit the scope of an operation to some depth, much like the way the older `--non-recursive` (`-N`) and `--recursive` (`-R`) options behave. This means that when operating on a working copy of some depth, while requesting an operation of a shallower depth, the operation is limited to that shallower depth. In fact, we can make an even more general statement: given a working copy of any arbitrary—even mixed—ambient depth, and a Subversion command with some requested operational depth, the command will maintain the ambient depth of the working copy members while still limiting the scope of the operation to the requested (or default) operational depth.

In addition to the `--depth` option, the svn update and svn switch subcommands also accept a second depth-related option: `--set-depth`. It is with this option that you can change the sticky depth of a working copy item. Watch what happens as we take our empty-depth checkout and gradually telescope it deeper using `svn update --set-depth `NEW-DEPTH` `TARGET``:

    $ svn update --set-depth files mom-empty
    Updating 'mom-empty':
    A    mom-empty/kittie1.txt
    A    mom-empty/doggie1.txt
    Updated to revision 1.
    $ svn update --set-depth immediates mom-empty
    Updating 'mom-empty':
    A    mom-empty/son
    A    mom-empty/daughter
    Updated to revision 1.
    $ svn update --set-depth infinity mom-empty
    Updating 'mom-empty':
    A    mom-empty/son/grandson
    A    mom-empty/daughter/granddaughter1
    A    mom-empty/daughter/granddaughter1/bunny1.txt
    A    mom-empty/daughter/granddaughter1/bunny2.txt
    A    mom-empty/daughter/granddaughter2
    A    mom-empty/daughter/fishie1.txt
    Updated to revision 1.
    $

As we gradually increased our depth selection, the repository gave us more pieces of our tree.

In our example, we operated only on the root of our working copy, changing its ambient depth value. But we can independently change the ambient depth value of any subdirectory inside the working copy, too. Careful use of this ability allows us to flesh out only certain portions of the working copy tree, leaving other portions absent altogether (hence the “sparse” bit of the feature's name). Here's an example of how we might build out a portion of one branch of our family's tree, enable full recursion on another branch, and keep still other pieces pruned (absent from disk).

    $ rm -rf mom-empty
    $ svn checkout file:///var/svn/repos mom-empty --depth empty
    Checked out revision 1.
    $ svn update --set-depth empty mom-empty/son
    Updating 'mom-empty/son':
    A    mom-empty/son
    Updated to revision 1.
    $ svn update --set-depth empty mom-empty/daughter
    Updating 'mom-empty/daughter':
    A    mom-empty/daughter
    Updated to revision 1.
    $ svn update --set-depth infinity mom-empty/daughter/granddaughter1
    Updating 'mom-empty/daughter/granddaughter1':
    A    mom-empty/daughter/granddaughter1
    A    mom-empty/daughter/granddaughter1/bunny1.txt
    A    mom-empty/daughter/granddaughter1/bunny2.txt
    Updated to revision 1.
    $

Fortunately, having a complex collection of ambient depths in a single working copy doesn't complicate the way you interact with that working copy. You can still make, revert, display, and commit local modifications in your working copy without providing any new options (including `--depth` and `--set-depth`) to the relevant subcommands. Even svn update works as it does elsewhere when no specific depth is provided—it updates the working copy targets that are present while honoring their sticky depths.

You might at this point be wondering, “So what? When would I use this?” One scenario where this feature finds utility is tied to a particular repository layout, specifically where you have many related or codependent projects or software modules living as siblings in a single repository location (`trunk/project1`, `trunk/project2`, `trunk/project3`, etc.). In such scenarios, it might be the case that you personally care about only a handful of those projects—maybe some primary project and a few other modules on which it depends. You can check out individual working copies of all of these things, but those working copies are disjoint and, as a result, it can be cumbersome to perform operations across several or all of them at the same time. The alternative is to use the sparse directories feature, building out a single working copy that contains only the modules you care about. You'd start with an empty-depth checkout of the common parent directory of the projects, and then update with infinite depth only the items you wish to have, like we demonstrated in the previous example. Think of it like an opt-in system for working copy citizens.

The original (Subversion 1.5) implementation of shallow checkouts was good, but didn't support de-telescoping of working copy items. Subversion 1.6 remedied this problem. For example, running `svn update --set-depth empty` in an infinite-depth working copy will discard everything but the topmost directory.[20] Subversion 1.6 also introduced another supported value for the `--set-depth` option: `exclude`. Using `--set-depth exclude` with svn update will cause the update target to be removed from the working copy entirely—a directory target won't even be left present-but-empty. This is especially handy when there are more things that you'd like to keep in a working copy than things you'd like to not keep.

Consider a directory with hundreds of subdirectories, one of which you would like to omit from your working copy. Using an “additive” approach to sparse directories, you might check out the directory with an empty depth, then explicitly telescope (using `svn update --set-depth infinity`) each and every subdirectory of the directory except the one you don't care about.

    $ svn checkout http://svn.example.com/repos/many-dirs --depth empty
    …
    $ svn update --set-depth infinity many-dirs/wanted-dir-1
    …
    $ svn update --set-depth infinity many-dirs/wanted-dir-2
    …
    $ svn update --set-depth infinity many-dirs/wanted-dir-3
    …
    ### and so on, and so on, ...

This could be quite tedious, especially since you don't even have stubs of these directories in your working copy to deal with. Such a working copy would also have another characteristic that you might not expect or desire: if someone else creates any new subdirectories in this top-level directory, you won't receive those when you update your working copy.

Beginning with Subversion 1.6, you can take a different approach. First, check out the directory in full. Then run `svn update --set-depth exclude` on the one subdirectory you don't care about.

    $ svn checkout http://svn.example.com/repos/many-dirs
    …
    $ svn update --set-depth exclude many-dirs/unwanted-dir
    D         many-dirs/unwanted-dir
    $

This approach leaves your working copy with the same stuff as in the first approach, but any new subdirectories which appear in the top-level directory would also show up when you update your working copy. The downside of this approach is that you have to actually check out that whole subdirectory that you don't even want just so you can tell Subversion that you don't want it. This might not even be possible if that subdirectory is too large to fit on your disk (which might, after all, be the very reason you don't want it in your working copy).

| Note
While the functionality for excluding an existing item from a working copy was hung off of the svn update command, you might have noticed that the output from `svn update --set-depth exclude` differs from that of a normal update operation. This output betrays the fact that, under the hood, exclusion is a completely client-side operation, very much unlike a typical update.

In such a situation, you might consider a compromise approach. First, check out the top-level directory with `--depth immediates`. Then, exclude the directory you don't want using `svn update --set-depth exclude`. Finally, telescope all the items that remain to infinite depth, which should be fairly easy to do because they are all addressable in your shell.

    $ svn checkout http://svn.example.com/repos/many-dirs --depth immediates
    …
    $ svn update --set-depth exclude many-dirs/unwanted-dir
    D         many-dirs/unwanted-dir
    $ svn update --set-depth infinity many-dirs/*
    …
    $

Once again, your working copy will have the same stuff as in the previous two scenarios. But now, any time a new file or subdirectory is committed to the top-level directory, you'll receive it—at an empty depth—when you update your working copy. You can now decide what to do with such newly appearing working copy items: expand them into infinite depth, or exclude them altogether.

[20] Safely, of course. As in other situations, Subversion will leave on disk any files you've modified or which aren't versioned.

Prev | Up | Next
Keyword Substitution | Home | Locking