# About AnnDB

AnnDB is a distributed approximate nearest neighbours database that allows you to provide a next-generation search experience in just a few lines of code. It's also fully hosted so there is no hassle with deploying and managing your infrastructure.

## **Key Features of AnnDB**

* **Fast:** Map-Reduce like search allows us to achieve low latency and high throughput.
* **Highly available:** AnnDB shards and replicates the data across multiple servers to ensure high availability and data resiliency.
* **CRUD:** Unlike most solutions, AnnDB does not need to rebuild the index when the data changes and makes new/updated data available almost instantly.
* **Managed:** Our software-as-a-service offering provides a managed solution for visual search, semantic search, question answering, and approximate nearest neighbours search in high-dimensional vector spaces with billions of items.

## Key Concepts

#### Vector Embeddings

Vector embeddings are abstract numeric representations of images, natural language, etc. They are often generated using deep learning models that are trained to capture visual similarity between images, semantic similarity between sentences, or question-answer relevance of facts and natural language questions.

#### Similarity Search

Unlike traditional search methods which use exact matching, inverted indices, etc., AnnDB leverages properties of vector representations to index and search items. This allows it to quickly find relevant items with high accuracy.

## Datasets

You can currently create datasets that support the following use-cases out-of-the-box:

* [Image-Image similarity](/datasets/images)
* [Text-Image similarity](/datasets/images)
* [Text semantic similarity](/datasets/semantic-search)
* [Question answering](/datasets/question-answering)
* [Vector approximate nearest neighbours](/datasets/vector)

AnnDB handles the embedding of images and strings to vectors internally, so that you do not have to deploy any machine learning models yourself. If you want to use your own embeddings, you can choose vector dataset type and use AnnDB as a hosted approximate nearest neighbours index.


# Quickstart

### Create a Dataset

The first step is to [create a dataset](https://app.anndb.com/datasets/new) for your data. AnnDB provides multiple dataset types which provide out-of-the-box solutions for image similarity search, text semantic search, question answering, or raw approximate nearest neighbours search in case you want to use your own vector embeddings.

![](https://1514770980-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MBjJDID-9E2lsbXx9on%2F-MYkETBBcn9bBpM44tDf%2F-MYkEgiQwI-yWuBWlUbd%2Fimage.png?alt=media\&token=ca88d4a1-3aa7-412f-a8ad-66e978d9a353)

### Create an API Key

Next, you will need to [create an API key](https://app.anndb.com/api_keys/new) to authenticate your client application with the AnnDB API. You can either a universal API key for all datasets, or you can create a dataset-specific API key to limit the access scope.

![](https://1514770980-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MBjJDID-9E2lsbXx9on%2F-MYjwSiRdsyCO6l_j4lY%2F-MYjygWgNhyd6F-beWdV%2Fimage.png?alt=media\&token=a25a6482-6706-4e64-8328-ed4f47dec8ba)

### Install AnnDB

AnnDB provides client implementations in the following languages: [Python](https://github.com/anndb-com/anndb-api-client-python), [Ruby](https://github.com/anndb-com/anndb-api-client-ruby). Clients allow you to modify and search the data stored in your datasets. 

{% tabs %}
{% tab title="Python" %}

```
pip install anndb-api
```

{% endtab %}

{% tab title="Ruby" %}

```
gem install anndb_api
```

{% endtab %}
{% endtabs %}

### Hello, world!

This example application shows, how easy it is to build an image similarity search service with AnnDB in just a few lines of code.

{% tabs %}
{% tab title="Python" %}

```python
import anndb_api

# Create a client instance
client = anndb_api.Client('<YOUR_API_KEY')

# Load the dataset
dataset = client.images('<DATASET_NAME>')

# Insert the data
for url in img_urls:
    id = dataset.insert(url, metadata={'src_url': url})
    
# Delete some of it
dataset.delete(id)

# Query top 5 similar images
items = dataset.search_image(img_urls[-1], 5)

# Query top 5 similar images using textual query
items = dataset.search_text('cute puppy', 5)
```

{% endtab %}

{% tab title="Ruby" %}

```ruby
require 'anndb_api'

# Create a client instance
client = AnndbApi::Client.new("<YOUR_API_KEY")

# Load the dataset
dataset = client.images("<DATASET_NAME>")

img_urls.each do |url|
    id = dataset.insert(url, metadata={ "src_url": url })
end
    
# Delete some of it
dataset.delete(id)

# Query top 5 similar images
items = dataset.search_image(img_urls.last, 5)

# Query top 5 similar images using textual query
items = dataset.search_text("cute puppy", 5)
```

{% endtab %}
{% endtabs %}


# Images

Methods on image datasets accept either image URLs for publically accessible images or image objects for images stored in the filesystem. The maximum image size for locally stored images is 200kB. The transformation from images to vectors is done internally and your application doesn't have to worry about it.

{% hint style="info" %}
Recommended image size is 224x224px. Larger images can cause latency issues while smaller images can hurt the search performance.
{% endhint %}

### Create a Dataset

Create a dataset with `Image` type which tells AnnDB to embed your images and queries to vector representations.

![](https://1514770980-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MBjJDID-9E2lsbXx9on%2F-MYkCJb7wNKAGTLUtvyn%2F-MYkC_PlcCBidy8TmRGq%2Fimage.png?alt=media\&token=41f086c8-5ed5-44f0-9e1e-c74498aa9da0)

In order to manage images in your dataset, create a corresponding dataset instance using the client.

{% tabs %}
{% tab title="Python" %}

```python
dataset = client.images('<DATASET_NAME>')
```

Python client accepts either image URLs or `PIL.Image.Image` objects as images.
{% endtab %}

{% tab title="Ruby" %}

```ruby
dataset = client.images("<DATASET_NAME>")
```

Ruby client accepts either image URLs or image byte arrays as images. To obtain a byte array from `MiniMagick::Image` use `.to_blob`.
{% endtab %}
{% endtabs %}

### Search using images

{% tabs %}
{% tab title="Python" %}

```python
result = dataset.search(img, 10)

for item in result:
    print(item.id, item.metadata)
```

{% endtab %}

{% tab title="Ruby" %}

```ruby
result = dataset.search(img, 10)

result.each do |item|
    puts item.id, item.metadata
end
```

{% endtab %}
{% endtabs %}

### Search using natural language

{% tabs %}
{% tab title="Python" %}

```python
result = dataset.search('cute puppy', 10)

for item in result:
    print(item.id, item.metadata)
```

{% endtab %}

{% tab title="Ruby" %}

```ruby
result = dataset.search("cute puppy", 10)

result.each do |item|
    puts item.id, item.metadata
end
```

{% endtab %}
{% endtabs %}

### Insert

{% tabs %}
{% tab title="Python" %}

```python
# Single image
id = dataset.insert(img, metadata={'key': 'value'})
```

```python
# Batch
result = dataset.insert_batch([
    anndb_api.ImageItem(None, img, {'key': 'value'}),
    ...
])

for r in result:
    print(r.id, r.error)
```

{% endtab %}

{% tab title="Ruby" %}

```ruby
id = dataset.insert(img, metadata={ "key": "value" })
```

```ruby
result = dataset.insert_batch([
    {
        image: img.to_blob,
        metadata: { "key": "value" }
    },
    ...
])

result.each { |r|
    puts r[:id], r[:error]
}
```

{% endtab %}
{% endtabs %}

### Update

{% tabs %}
{% tab title="Python" %}

```python
# Single image
id = dataset.update(id, img, metadata={'key': 'value'})
```

```python
# Batch
result = dataset.update_batch([
    anndb_api.ImageItem(id, img, {'key': 'value'}),
    ...
])

for r in result:
    print(r.id, r.error)
```

{% endtab %}

{% tab title="Ruby" %}

```ruby
id = dataset.update(id, img, metadata={ "key": "value" })
```

```ruby
result = dataset.update_batch([
    {
        id: id,
        image: "https://my.domain/img.jpg",
        metadata: { "key": "value" }
    },
    ...
])

result.each { |r|
    puts r[:id], r[:error]
}
```

{% endtab %}
{% endtabs %}

### Delete

{% tabs %}
{% tab title="Python" %}

```python
# Single image
dataset.delete(id)
```

```python
# Batch
result = dataset.delete_batch([id, ...])

for r in result:
    print(r.id, r.error)
```

{% endtab %}

{% tab title="Ruby" %}

```ruby
dataset.delete(id)
```

```ruby
result = dataset.delete_batch([id, ...])

result.each { |r|
    puts r[:id], r[:error]
}
```

{% endtab %}
{% endtabs %}


# Semantic Search

Semantic textual similarity datasets allow you to go beyond traditional keyword and exact-match search. AnnDB uses state-of-the-art machine learning models to provide accurate and semantically relevant results. Using the API, you can build a dataset of sentences which you can then query using natural language queries to find semantically similar information.

### Create a Dataset

Create a dataset with `Semantic Similarity` type which tells AnnDB to encode your sentences to vectors.

![](https://1514770980-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MBjJDID-9E2lsbXx9on%2F-MYkDSbL82qX_ZankqcO%2F-MYkDYR7niLJMu3Vjchb%2Fimage.png?alt=media\&token=4b85967a-2c14-4703-9370-6d9b5e14d07c)

In order to manage data in your dataset, create a corresponding dataset instance using the client.

{% tabs %}
{% tab title="Python" %}

```python
dataset = client.text('<DATASET_NAME>')
```

{% endtab %}

{% tab title="Ruby" %}

```ruby
dataset = client.text("<DATASET_NAME>")
```

{% endtab %}
{% endtabs %}

### Search

{% tabs %}
{% tab title="Python" %}

```python
result = dataset.search('query', 10)

for item in result:
    print(item.id, item.metadata)
```

{% endtab %}

{% tab title="Ruby" %}

```ruby
result = dataset.search("query", 10)

result.each do |item|
    puts item.id, item.metadata
end
```

{% endtab %}
{% endtabs %}

### Insert

{% tabs %}
{% tab title="Python" %}

```python
# Single item
id = dataset.insert('my sentence', metadata={'key': 'value'})
```

```python
# Batch
result = dataset.insert_batch([
    anndb_api.TextItem(None, 'my sentence', {'key': 'value'}),
    ...
])

for r in result:
    print(r.id, r.error)
```

{% endtab %}

{% tab title="Ruby" %}

```ruby
id = dataset.insert("my sentence", metadata={ "key": "value" })
```

```ruby
result = dataset.insert_batch([
    {
        text: "my sentence",
        metadata: { "key": "value" }
    },
    ...
])

result.each { |r|
    puts r[:id], r[:error]
}
```

{% endtab %}
{% endtabs %}

### Update

{% tabs %}
{% tab title="Python" %}

```python
# Single item
id = dataset.update(id, 'my updated sentence', metadata={'key': 'value'})
```

```python
# Batch
result = dataset.update_batch([
    anndb_api.TextItem(id, 'my updated sentence', {'key': 'value'}),
    ...
])

for r in result:
    print(r.id, r.error)
```

{% endtab %}

{% tab title="Ruby" %}

```ruby
id = dataset.update(id, "my updated sentence", metadata={ "key": "value" })
```

```ruby
result = dataset.update_batch([
    {
        id: id,
        text: "my updated sentence",
        metadata: { "key": "value" }
    },
    ...
])

result.each { |r|
    puts r[:id], r[:error]
}
```

{% endtab %}
{% endtabs %}

### Delete

{% tabs %}
{% tab title="Python" %}

```python
# Single item
dataset.delete(id)
```

```python
# Batch
result = dataset.delete_batch([id, ...])

for r in result:
    print(r.id, r.error)
```

{% endtab %}

{% tab title="Ruby" %}

```ruby
dataset.delete(id)
```

```ruby
result = dataset.delete_batch([id, ...])

result.each { |r|
    puts r[:id], r[:error]
}
```

{% endtab %}
{% endtabs %}


# Question Answering

Question answering datasets provide an out-of-the-box solution for a machine learning powered search engine. AnnDB uses state-of-the-art machine learning models to provide highly relevant search results. Using the API, you can build a dataset of facts that you can later query using natural language queries.

### Create a Dataset

Create a dataset with `Question Answering` type which tells AnnDB to encode your facts and queries to vectors.

![](https://1514770980-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MBjJDID-9E2lsbXx9on%2F-MYkDju8UK5m8Rywh23Q%2F-MYkDtqLyiHJZ4buhnc5%2Fimage.png?alt=media\&token=eef34c28-e0f1-4392-a29f-4dc0520f7e25)

In order to manage data in your dataset, create a corresponding dataset instance using the client.

{% tabs %}
{% tab title="Python" %}

```python
dataset = client.text('<DATASET_NAME>')
```

{% endtab %}

{% tab title="Ruby" %}

```ruby
dataset = client.text("<DATASET_NAME>")
```

{% endtab %}
{% endtabs %}

### Search

{% tabs %}
{% tab title="Python" %}

```python
result = dataset.search('query', 10)

for item in result:
    print(item.id, item.metadata)
```

{% endtab %}

{% tab title="Ruby" %}

```ruby
result = dataset.search("query", 10)

result.each do |item|
    puts item.id, item.metadata
end
```

{% endtab %}
{% endtabs %}

### Insert

{% tabs %}
{% tab title="Python" %}

```python
# Single item
id = dataset.insert(
    'London has 8.9 million inhabitants.',
    metadata={'key': 'value'}
)
```

```python
# Batch
result = dataset.insert_batch([
    anndb_api.TextItem(
        None,
        'London has 8.9 million inhabitants.',
        {'key': 'value'}
    ),
    ...
])

for r in result:
    print(r.id, r.error)
```

{% endtab %}

{% tab title="Ruby" %}

```ruby
id = dataset.insert(
    "London has 8.9 million inhabitants.",
    metadata={ "key": "value" }
)
```

```ruby
result = dataset.insert_batch([
    {
        text: "London has 8.9 million inhabitants.",
        metadata: { "key": "value" }
    },
    ...
])

result.each { |r|
    puts r[:id], r[:error]
}
```

{% endtab %}
{% endtabs %}

### Update

{% tabs %}
{% tab title="Python" %}

```python
# Single item
id = dataset.update(
    id,
    'London has 9 million inhabitants.',
    metadata={'key': 'value'}
)
```

```python
# Batch
result = dataset.update_batch([
    anndb_api.TextItem(
        id,
        'London has 9 million inhabitants.',
        {'key': 'value'}
    ),
    ...
])

for r in result:
    print(r.id, r.error)
```

{% endtab %}

{% tab title="Ruby" %}

```ruby
id = dataset.update(
    id,
    "London has 9 million inhabitants.",
    metadata={ "key": "value" }
)
```

```ruby
result = dataset.update_batch([
    {
        id: id,
        text: "London has 9 million inhabitants.",
        metadata: { "key": "value" }
    },
    ...
])

result.each { |r|
    puts r[:id], r[:error]
}
```

{% endtab %}
{% endtabs %}

### Delete

{% tabs %}
{% tab title="Python" %}

```python
# Single item
dataset.delete(id)
```

```python
# Batch
result = dataset.delete_batch([id, ...])

for r in result:
    print(r.id, r.error)
```

{% endtab %}

{% tab title="Ruby" %}

```ruby
dataset.delete(id)
```

```ruby
result = dataset.delete_batch([id, ...])

result.each { |r|
    puts r[:id], r[:error]
}
```

{% endtab %}
{% endtabs %}


# Vector

Vector datasets allow you to store your own vectors so that they can be efficiently queried later. The dimension of vectors has to match the dimension specified when creating the dataset.

### Create a Dataset

Create a dataset with `Vector` type which tells AnnDB to skip any pre-processing and embedding, and store the vectors as is. The distance metric and dimension **cannot** be changed later.

![](https://1514770980-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MBjJDID-9E2lsbXx9on%2F-MYkE1SHEuj4pLbrODAC%2F-MYkEDcEhKXJmcFLpo7u%2Fimage.png?alt=media\&token=ca4d84f7-ac5e-4b5c-901d-7a1174048cd4)

In order to manage data in your dataset, create a corresponding dataset instance using the client.

{% tabs %}
{% tab title="Python" %}

```python
dataset = client.vector('<DATASET_NAME>')
```

Python client accepts either a list of floats or a NumPy array as vectors.
{% endtab %}

{% tab title="Ruby" %}

```ruby
dataset = client.vector("<DATASET_NAME>")
```

Ruby client accepts a list of floats as vectors.
{% endtab %}
{% endtabs %}

### Search

{% tabs %}
{% tab title="Python" %}

```python
result = dataset.search([0.4506, -0.6739, -0.2360, -1.3630, ...], 10)

for item in result:
    print(item.id, item.metadata)
```

{% endtab %}

{% tab title="Ruby" %}

```ruby
result = dataset.search([0.4506, -0.6739, -0.2360, -1.3630, ...], 10)

result.each do |item|
    puts item.id, item.metadata
end
```

{% endtab %}
{% endtabs %}

### Insert

{% tabs %}
{% tab title="Python" %}

```python
# Single item
id = dataset.insert(
    [0.4506, -0.6739, -0.2360, -1.3630, ...],
    metadata={'key': 'value'}
)
```

```python
# Batch
result = dataset.insert_batch([
    anndb_api.VectorItem(
        None,
        [0.4506, -0.6739, -0.2360, -1.3630, ...],
        {'key': 'value'}
    ),
    ...
])

for r in result:
    print(r.id, r.error)
```

{% endtab %}

{% tab title="Ruby" %}

```ruby
id = dataset.insert(
    [0.4506, -0.6739, -0.2360, -1.3630, ...],
    metadata={ "key": "value" }
)
```

```ruby
result = dataset.insert_batch([
    {
        vector: [0.4506, -0.6739, -0.2360, -1.3630, ...],
        metadata: { "key": "value" }
    },
    ...
])

result.each { |r|
    puts r[:id], r[:error]
}
```

{% endtab %}
{% endtabs %}

### Update

{% tabs %}
{% tab title="Python" %}

```python
# Single item
id = dataset.update(
    id,
    [0.4506, -0.6739, -0.2360, -1.3630, ...],
    metadata={'key': 'value'}
)
```

```python
# Batch
result = dataset.update_batch([
    anndb_api.VectorItem(
        id,
        [0.4506, -0.6739, -0.2360, -1.3630, ...],
        {'key': 'value'}
    ),
    ...
])

for r in result:
    print(r.id, r.error)
```

{% endtab %}

{% tab title="Ruby" %}

```ruby
id = dataset.update(
    id,
    [0.4506, -0.6739, -0.2360, -1.3630, ...],
    metadata={ "key": "value" }
)
```

```ruby
result = dataset.update_batch([
    {
        id: id,
        vector: [0.4506, -0.6739, -0.2360, -1.3630, ...],
        metadata: { "key": "value" }
    },
    ...
])

result.each { |r|
    puts r[:id], r[:error]
}
```

{% endtab %}
{% endtabs %}

### Delete

{% tabs %}
{% tab title="Python" %}

```python
# Single item
dataset.delete(id)
```

```python
# Batch
result = dataset.delete_batch([id, ...])

for r in result:
    print(r.id, r.error)
```

{% endtab %}

{% tab title="Ruby" %}

```ruby
dataset.delete(id)
```

```ruby
result = dataset.delete_batch([id, ...])

result.each { |r|
    puts r[:id], r[:error]
}
```

{% endtab %}
{% endtabs %}