Xapian: Code structure

# Index

- ROOT
- api
- backends
  - backends/brass
  - backends/chert
  - backends/flint
  - backends/inmemory
  - backends/multi
  - backends/remote
- bin
- common
- docs
- examples
- expand
- include
- languages
- matcher
- net
- queryparser
- tests
  - tests/harness
  - tests/perftest
  - tests/soaktest
- unicode
- weight

# Directory structure

## ROOT

Top level directory.

## api

API classes and their PIMPL internals.

## backends

This directory contains a subdirectory for each of the available database backends. Each backend corresponds to a different underlying file structure. For example, the inmemory backend holds its databases entirely in RAM, and the flint backend is a fully featured disk based backend.

The directory also contains the implementation of Xapian::Database::Internal (the base class for each backend database class) and the factory functions which are the public interface for instantiating the database backend classes.

### backends/brass

Brass is the development backend for the Xapian 1.2.x series. It's highly efficient, and also (aims to) use significantly less disk space than previous Xapian backends. All Xapian features are supported.

### backends/chert

Chert is the default backend for the Xapian 1.2.x release series. It uses a custom written Btree management system to store posting lists and termlists. This is a highly efficient backend, using compression to store the postlists, and supporting the full range of indexing functionality (positional information, transactions, etc).

### backends/flint

Flint is the default Xapian backend as of Xapian 1.0. It uses a custom written Btree management system to store posting lists and termlists. This is a highly efficient backend, using compression to store the postlists, and supporting the full range of indexing functionality (positional information, transactions, etc).

### backends/inmemory

This backend stores a database entirely in memory. When the database is closed these indexed contents are lost.

This is useful for searching through relatively small amounts of data (such as a single large file) which hasn't previously been indexed.

### backends/multi

The MultiDatabase backend, which enables searches to be performed across several databases. Opening this database involves opening each of the sub-databases and merging them together.

Searches are performed across the sub-databases via MultiPostList and MultiTermList objects, which represent merged sets of postlist and termlist objects.

### backends/remote

The remote backend, which enables searches to be performed across databases on remote machines. Opening this database involves opening a communications channel with a remote database.

RemoteDatabase objects are used with RemoteSubMatch objects.

## bin

Programs relating to the Xapian library.

## common

Header files which are used in various places within the Xapian library code. It does not contain header files which are externally visible: these are kept in the "include" directory.

## docs

Documentation, and scripts to automatically generate further documentation from the source code. If you have the appropriate packages installed (currently, this means Perl), these scripts will be run by make.

## examples

This directory contains example programs which use the Xapian library. These programs are intended to be a good starting point for those trying to get acquainted with the Xapian API. Some of them are really just toy programs, but others are actually useful utilities in their own right (for example: delve, quest, and copydatabase).

## expand

This directory houses the query expansion code.

## include

This directory contains the externally visible header files. Internal header files are kept in the "common" directory.

## languages

Utilities for performing processing of text in various different languages. Current these comprise stemming algorithms. In future language detection, character set normalisation, and other language related utilities will be added.

## matcher

The code for performing the best match algorithm lives here. This is the heart of the Xapian system, and is the code which calculates relevance rankings for the documents in the collection for a given query.

## net

The code implementing the network protocols lives here.

## queryparser

Implementations of the Xapian::QueryParser and Xapian::TermGenerator classes.

## tests

This directory contains various test programs which exercise most parts of the Xapian library.

### tests/harness

This contains the test suite harness, which is linked with by most of the C++ test programs to perform sets of tests.

### tests/perftest

This directory contains various the performance test suite.

### tests/soaktest

This directory contains a testsuite for "soak" testing Xapian - it generates and runs random sequences of operations, checking the validity of the output whenever possible.

## unicode

Unicode and UTF-8 handling classes and functions.

## weight

Implementations of weighting schemes for Xapian.