An easier way to build neural search on the cloud
Jina is a deep learning-powered search framework for building cross-/multi-modal search systems (e.g. text, images, video, audio) on the cloud. β±οΈ **Time Saver** - *The* design pattern of neural search systems, from zero to a production-ready system in minutes. π± **Full-Stack Ownership** - Keep an end-to-end stack ownership of your solution, avoid the integration pitfalls with fragmented, multi-vendor, generic legacy tools. π **Universal Search** - Large-scale indexing and querying of unstructured data: video, image, long/short text, music, source code, etc. π§ **First-Class AI Models** - First-class support for [state-of-the-art AI models](https://docs.jina.ai/chapters/all_exec.html), easily usable and extendable with a Pythonic interface. π©οΈ **Fast & Cloud Ready** - Decentralized architecture from day one. Scalable & cloud-native by design: enjoy containerizing, distributing, sharding, async, REST/gRPC/WebSocket. β€οΈ **Made with Love** - Never compromise on quality, actively maintained by a [passionate full-time, venture-backed team](https://jina.ai). ---Docs β’ Hello World β’ Quick Start β’ Learn β’ Examples β’ Contribute β’ Jobs β’ Website β’ Slack
## Installation | π¦
|
**PyCharm** 1. Click menu `Preferences` -> `JSON Schema mappings`; 2. Add a new schema, in the `Schema File or URL` write `https://api.jina.ai/schemas/latest.json`; select `JSON Schema Version 7`; 3. Add a file path pattern and link it to `*.jaml` and `*.jina.yml`. |
|
**VSCode** 1. Install the extension: `YAML Language Support by Red Hat`; 2. In IDE-level `settings.json` add: ```json "yaml.schemas": { "https://api.jina.ai/schemas/latest.json": ["/*.jina.yml", "/*.jaml"], } ``` |
A simple image neural search demo for [Fashion-MNIST](https://hanxiao.io/2018/09/28/Fashion-MNIST-Year-In-Review/). No extra dependencies needed, simply run:
```bash
jina hello fashion # more options in --help
```
...or even easier for Docker users, **no install required**:
```bash
docker run -v "$(pwd)/j:/j" jinaai/jina hello fashion --workdir /j && open j/hello-world.html
# replace "open" with "xdg-open" on Linux
```
For NLP engineers, we provide a simple chatbot demo for answering Covid-19 questions. To run that:
```bash
pip install "jina[chatbot]"
jina hello chatbot
```
This downloads [CovidQA dataset](https://www.kaggle.com/xhlulu/covidqa) and tells Jina to index 418 question-answer pairs with DistilBERT. The index process takes about 1 minute on CPU. Then it opens a web page where you can input questions and ask Jina.
A multimodal-document contains multiple data types, e.g. a PDF document often contains figures and text. Jina lets you build a multimodal search solution in just minutes. To run our minimum multimodal document search demo:
```bash
pip install "jina[multimodal]"
jina hello multimodal
```
This downloads [people image dataset](https://www.kaggle.com/ahmadahmadzada/images2000) and tells Jina to index 2,000 image-caption pairs with MobileNet and DistilBERT. The index process takes about 3 minute on CPU. Then it opens a web page where you can query multimodal documents. We have prepared [a YouTube tutorial](https://youtu.be/B_nH8GCmBfc) to walk you through this demo.
First we look at basic CRUD operations. In Jina, CRUD corresponds to four functions: `index` (create), `search` (read), `update`, and `delete`. With Documents below as an example:
```python
import numpy as np
from jina import Document
docs = [Document(id='π²', embedding=np.array([0, 0]), tags={'guardian': 'Azure Dragon', 'position': 'East'}),
Document(id='π¦', embedding=np.array([1, 0]), tags={'guardian': 'Vermilion Bird', 'position': 'South'}),
Document(id='π’', embedding=np.array([0, 1]), tags={'guardian': 'Black Tortoise', 'position': 'North'}),
Document(id='π―', embedding=np.array([1, 1]), tags={'guardian': 'White Tiger', 'position': 'West'})]
```
Let's build a Flow with a simple indexer:
```python
from jina import Flow
f = Flow().add(uses='_index')
```
`Document` and `Flow` are basic concepts in Jina, which will be explained later. `_index` is a built-in embedding + structured storage that you can use out of the box.
| Index | ```python # save four docs (both embedding and structured info) into storage with f: f.index(docs, on_done=print) ``` |
| Search |
```python
# retrieve top-3 neighbours of π², this print π²π¦π’ with score 0, 1, 1 respectively
with f:
f.search(docs[0], top_k=3, on_done=lambda x: print(x.docs[0].matches))
```
```json
{"id": "π²", "tags": {"guardian": "Azure Dragon", "position": "East"}, "embedding": {"dense": {"buffer": "AAAAAAAAAAAAAAAAAAAAAA==", "shape": [2], "dtype": " |
| Update | ```python # update π² embedding in the storage docs[0].embedding = np.array([1, 1]) with f: f.update(docs[0]) ``` |
| Delete | ```python # remove π¦π² Documents from the storage with f: f.delete(['π¦', 'π²']) ``` |
`Document` is [Jina's primitive data type](https://hanxiao.io/2020/11/22/Primitive-Data-Types-in-Neural-Search-System/#primitive-types). It can contain text, image, array, embedding, URI, and be accompanied by rich meta information. To construct a Document, you can use:
```python
import numpy
from jina import Document
doc1 = Document(content=text_from_file, mime_type='text/x-python') # a text document contains python code
doc2 = Document(content=numpy.random.random([10, 10])) # a ndarray document
```
A Document can be recursed both vertically and horizontally to have nested Documents and matched Documents. To better see the Document's recursive structure, you can use `.plot()` function. If you are using JupyterLab/Notebook, all Document objects will be auto-rendered.
| ```python import numpy from jina import Document d0 = Document(id='π²', embedding=np.array([0, 0])) d1 = Document(id='π¦', embedding=np.array([1, 0])) d2 = Document(id='π’', embedding=np.array([0, 1])) d3 = Document(id='π―', embedding=np.array([1, 1])) d0.chunks.append(d1) d0.chunks[0].chunks.append(d2) d0.matches.append(d3) d0.plot() # simply `d0` on JupyterLab ``` |
|
Jina provides a high-level Flow API to simplify building CRUD workflows. To create a new Flow:
```python
from jina import Flow
f = Flow().add()
```
This creates a simple Flow with one [Pod](https://101.jina.ai/#pod). You can chain multiple `.add()`s in a single Flow.
To visualize the Flow, simply chain it with `.plot('my-flow.svg')`. If you are using a Jupyter notebook, the Flow object will be displayed inline *without* `plot`.
`Gateway` is the entrypoint of the Flow.
Get the vibe? Now we're talking! Let's learn more about the basic concepts and features of Jina:
---
| | |
| --- |---|
| π₯ | [CRUD Functions](#crud-functions) β’ [Document](#document) β’ [Flow](#flow) |
| π£ | [Feed Data](#feed-data) β’ [Fetch Result](#fetch-result) β’ [Add Logic](#add-logic) β’ [Inter & Intra Parallelism](#inter--intra-parallelism) β’ [Decentralize](#decentralized-flow) β’ [Asynchronous](#asynchronous-flow) |
| π₯ | [Customize Encoder](#customize-encoder) β’ [Test Encoder](#test-encoder-in-flow) β’ [Parallelism & Batching](#parallelism--batching) β’ [Add Data Indexer](#add-data-indexer) β’ [Compose Flow from YAML](#compose-flow-from-yaml) β’ [Search](#search) β’ [Evaluation](#evaluation) β’ [REST Interface](#rest-interface) |
### π£ Basic
#### Feed Data
To use a Flow, open it via `with` context manager, like you would open a file in Python. Now let's create some empty Documents and index them:
```python
from jina import Document
with Flow().add() as f:
f.index((Document() for _ in range(10)))
```
Flow supports CRUD operations: `index`, `search`, `update`, `delete`. In addition, it also provides sugary syntax on `ndarray`, `csv`, `ndjson` and arbitrary files.
| Input |
Example of index/search
|
Explain |
numpy.ndarray
|
```python with f: f.index_ndarray(numpy.random.random([4,2])) ``` | Input four `Document`s, each `document.blob` is an `ndarray([2])` |
| CSV | ```python with f, open('index.csv') as fp: f.index_csv(fp, field_resolver={'pic_url': 'uri'}) ``` | Each line in `index.csv` is constructed as a `Document`, CSV field `pic_url` mapped to `document.uri`. |
JSON Lines/ndjson/LDJSON
|
```python with f, open('index.ndjson') as fp: f.index_ndjson(fp, field_resolver={'question_id': 'id'}) ``` | Each line in `index.ndjson` is constructed as a `Document`, JSON field `question_id` mapped to `document.id`. |
| Files with wildcards | ```python with f: f.index_files(['/tmp/*.mp4', '/tmp/*.pdf']) ``` | Each file captured is constructed as a `Document`, and Document content (`text`, `blob`, `buffer`) is auto-guessed & filled. |
Once a request is done, callback functions are fired. Jina Flow implements a Promise-like interface: You can add callback functions `on_done`, `on_error`, `on_always` to hook different events. In the example below, our Flow passes the message then prints the result when successful. If something goes wrong, it beeps. Finally, the result is written to `output.txt`.
```python
def beep(*args):
# make a beep sound
import os
os.system('echo -n "\a";')
with Flow().add() as f, open('output.txt', 'w') as fp:
f.index(numpy.random.random([4, 5, 2]),
on_done=print, on_error=beep, on_always=lambda x: fp.write(x.json()))
```
#### Add Logic
To add logic to the Flow, use the `uses` parameter to attach a Pod with an [Executor](https://101.jina.ai/#executor). `uses` accepts multiple value types including class name, Docker image, (inline) YAML or built-in shortcut.
```python
f = (Flow().add(uses=MyBertEncoder) # the class of a Jina Executor
.add(uses='docker://jinahub/pod.encoder.dummy_mwu_encoder:0.0.6-0.9.3') # the image name
.add(uses='myencoder.yml') # YAML serialization of a Jina Executor
.add(uses='!WaveletTransformer | {freq: 20}') # inline YAML config
.add(uses='_pass') # built-in shortcut executor
.add(uses={'__cls': 'MyBertEncoder', 'with': {'param': 1.23}})) # dict config object with __cls keyword
```
The power of Jina lies in its decentralized architecture: Each `add` creates a new Pod, and these Pods can be run as a local thread/process, a remote process, inside a Docker container, or even inside a remote Docker container.
#### Inter & Intra Parallelism
Chaining `.add()`s creates a sequential Flow. For parallelism, use the `needs` parameter:
```python
f = (Flow().add(name='p1', needs='gateway')
.add(name='p2', needs='gateway')
.add(name='p3', needs='gateway')
.needs(['p1','p2', 'p3'], name='r1').plot())
```
`p1`, `p2`, `p3` now subscribe to `Gateway` and conduct their work in parallel. The last `.needs()` blocks all Pods until they finish their work. Note: parallelism can also be performed inside a Pod using `parallel`:
```python
f = (Flow().add(name='p1', needs='gateway')
.add(name='p2', needs='gateway')
.add(name='p3', parallel=3)
.needs(['p1','p3'], name='r1').plot())
```
#### Decentralized Flow
A Flow does not have to be local-only: You can put any Pod to remote(s). In the example below, with the `host` keyword `gpu-pod`, is put to a remote machine for parallelization, whereas other Pods stay local. Extra file dependencies that need to be uploaded are specified via the `upload_files` keyword.
| 123.456.78.9 | ```bash # have docker installed docker run --name=jinad --network=host -v /var/run/docker.sock:/var/run/docker.sock jinaai/jina:latest-daemon --port-expose 8000 # to stop it docker rm -f jinad ``` |
| Local | ```python import numpy as np from jina import Flow f = (Flow() .add() .add(name='gpu_pod', uses='mwu_encoder.yml', host='123.456.78.9:8000', parallel=2, upload_files=['mwu_encoder.py']) .add()) with f: f.index_ndarray(np.random.random([10, 100]), output=print) ``` |
While synchronous from outside, Jina runs asynchronously under the hood: it manages the eventloop(s) for scheduling the jobs. If the user wants more control over the eventloop, then `AsyncFlow` can be used.
Unlike `Flow`, the CRUD of `AsyncFlow` accepts input and output functions as [async generators](https://www.python.org/dev/peps/pep-0525/). This is useful when your data sources involve other asynchronous libraries (e.g. motor for MongoDB):
```python
from jina import AsyncFlow
async def input_function():
for _ in range(10):
yield Document()
await asyncio.sleep(0.1)
with AsyncFlow().add() as f:
async for resp in f.index(input_function):
print(resp)
```
`AsyncFlow` is particularly useful when Jina is using as part of integration, where another heavy-lifting job is running concurrently:
```python
async def run_async_flow_5s(): # WaitDriver pause 5s makes total roundtrip ~5s
with AsyncFlow().add(uses='- !WaitDriver {}') as f:
async for resp in f.index_ndarray(numpy.random.random([5, 4])):
print(resp)
async def heavylifting(): # total roundtrip takes ~5s
print('heavylifting other io-bound jobs, e.g. download, upload, file io')
await asyncio.sleep(5)
print('heavylifting done after 5s')
async def concurrent_main(): # about 5s; but some dispatch cost, can't be just 5s, usually at <7s
await asyncio.gather(run_async_flow_5s(), heavylifting())
if __name__ == '__main__':
asyncio.run(concurrent_main())
```
`AsyncFlow` is very useful when using Jina inside a Jupyter Notebook. As Jupyter/ipython already manages an eventloop and thanks to [`autoawait`](https://ipython.readthedocs.io/en/stable/interactive/autoawait.html), `AsyncFlow` can run out-of-the-box in Jupyter.
That's all you need to know for understanding the magic behind `hello-world`. Now let's dive deeper into it!
---
| | |
| --- |---|
| π₯ | [CRUD Functions](#crud-functions) β’ [Document](#document) β’ [Flow](#flow) |
| π£ | [Feed Data](#feed-data) β’ [Fetch Result](#fetch-result) β’ [Add Logic](#add-logic) β’ [Inter & Intra Parallelism](#inter--intra-parallelism) β’ [Decentralize](#decentralized-flow) β’ [Asynchronous](#asynchronous-flow) |
| π₯ | [Customize Encoder](#customize-encoder) β’ [Test Encoder](#test-encoder-in-flow) β’ [Parallelism & Batching](#parallelism--batching) β’ [Add Data Indexer](#add-data-indexer) β’ [Compose Flow from YAML](#compose-flow-from-yaml) β’ [Search](#search) β’ [Evaluation](#evaluation) β’ [REST Interface](#rest-interface) |
### π₯ Breakdown of `hello-world`
#### Customize Encoder
Let's first build a naive image encoder that embeds images into vectors using an orthogonal projection. To do this, we simply inherit from `BaseImageEncoder`: a base class from the `jina.executors.encoders` module. We then override its `__init__()` and `encode()` methods.
```python
import numpy as np
from jina.executors.encoders import BaseImageEncoder
class MyEncoder(BaseImageEncoder):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
np.random.seed(1337)
H = np.random.rand(784, 64)
u, s, vh = np.linalg.svd(H, full_matrices=False)
self.oth_mat = u @ vh
def encode(self, data: 'np.ndarray', *args, **kwargs):
return (data.reshape([-1, 784]) / 255) @ self.oth_mat
```
Jina provides [a family of `Executor` classes](https://101.jina.ai/#executor), which summarize frequently-used algorithmic components in neural search. This family consists of encoders, indexers, crafters, evaluators, and classifiers, each with a well-designed interface. You can find the list of [all 107 built-in executors here](https://docs.jina.ai/chapters/all_exec.html). If they don't meet your needs, inheriting from one of them is the easiest way to bootstrap your own Executor. Simply use our Jina Hub CLI:
```bash
pip install jina[hub] && jina hub new
```
#### Test Encoder in Flow
Let's test our encoder in the Flow with some synthetic data:
```python
def validate(req):
assert len(req.docs) == 100
assert NdArray(req.docs[0].embedding).value.shape == (64,)
f = Flow().add(uses='MyEncoder')
with f:
f.index_ndarray(numpy.random.random([100, 28, 28]), on_done=validate)
```
All good! Now our `validate` function confirms that all one hundred 28x28 synthetic images have been embedded into 100x64 vectors.
#### Parallelism & Batching
By setting a larger input, you can play with `batch_size` and `parallel`:
```python
f = Flow().add(uses='MyEncoder', parallel=10)
with f:
f.index_ndarray(numpy.random.random([60000, 28, 28]), batch_size=1024)
```
#### Add Data Indexer
Now we need to add an indexer to store all the embeddings and the image for later retrieval. Jina provides a simple `numpy`-powered vector indexer `NumpyIndexer`, and a key-value indexer `BinaryPbIndexer`. We can combine them in a single YAML file:
```yaml
jtype: CompoundIndexer
components:
- jtype: NumpyIndexer
with:
index_filename: vec.gz
- jtype: BinaryPbIndexer
with:
index_filename: chunk.gz
metas:
workspace: ./
```
- `jtype:` defines the class name of the structure;
- `with:` defines arguments for initializing this class object.
[π‘ Config your IDE to enable autocompletion on YAML](#yaml-completion-in-ide)
Essentially, the above YAML config is equivalent to the following Python code:
```python
from jina.executors.indexers.vector import NumpyIndexer
from jina.executors.indexers.keyvalue import BinaryPbIndexer
from jina.executors.indexers import CompoundIndexer
a = NumpyIndexer(index_filename='vec.gz')
b = BinaryPbIndexer(index_filename='vec.gz')
c = CompoundIndexer()
c.components = lambda: [a, b]
```
#### Compose Flow from YAML
Now let's add our indexer YAML file to the Flow with `.add(uses=)`. Let's also add two shards to the indexer to improve its scalability:
```python
f = Flow().add(uses='MyEncoder', parallel=2).add(uses='myindexer.yml', shards=2).plot()
```
When you have many arguments, constructing a Flow in Python can get cumbersome. In that case, you can simply move all arguments into one `flow.yml`:
```yaml
jtype: Flow
version: '1.0'
pods:
- name: encode
uses: MyEncoder
parallel: 2
- name:index
uses: myindexer.yml
shards: 2
```
And then load it in Python:
```python
f = Flow.load_config('flow.yml')
```
#### Search
Querying a Flow is similar to what we did with indexing. Simply load the query Flow and switch from `f.index` to `f.search`. Say you want to retrieve the top 50 documents that are similar to your query and then plot them in HTML:
```python
f = Flow.load_config('flows/query.yml')
with f:
f.search_ndarray(numpy.random.random([10, 28, 28]), shuffle=True, on_done=plot_in_html, top_k=50)
```
#### Evaluation
To compute precision recall on the retrieved result, you can add `_eval_pr`, a built-in evaluator for computing precision & recall.
```python
f = (Flow().add(...)
.add(uses='_eval_pr'))
```
You can construct an iterator of query and groundtruth pairs and feed to the flow `f`, via:
```python
from jina import Document
def query_generator():
for _ in range(10):
q = Document()
# now construct expect matches as groundtruth
gt = Document(q, copy=True) # make sure 'gt' is identical to 'q'
gt.matches.append(...)
yield q, gt
f.search(query_iterator, ...)
```
#### REST Interface
In practice, the query Flow and the client (i.e. data sender) are often physically separated. Moreover, the client may prefer to use a REST API rather than gRPC when querying. You can set `port_expose` to a public port and turn on [REST support](https://api.jina.ai/rest/) with `restful=True`:
```python
f = Flow(port_expose=45678, restful=True)
with f:
f.block()
```
That is the essence behind `jina hello fashion`. It is merely a taste of what Jina can do. Weβre really excited to see what you do with Jina! You can easily create a Jina project from templates with one terminal command:
```bash
pip install jina[hub] && jina hub new --type app
```
This creates a Python entrypoint, YAML configs and a Dockerfile. You can start from there.
## Learn
|
|
|
π |
NLP Semantic Wikipedia Search with Transformers and DistilBERTBrand new to neural search? See a simple text-search example to understand how Jina works |
π |
Add Incremental Indexing to Wikipedia SearchIndex more effectively by adding incremental indexing to your Wikipedia search |
π |
Search Lyrics with Transformers and PyTorchGet a better understanding of chunks by searching a lyrics database. Now with shiny front-end! |
πΌοΈ |
Google's Big Transfer Model in (PokΓ©-)ProductionUse SOTA visual representation for searching PokΓ©mon! |
π§ |
Search YouTube audio data with VggishA demo of neural search for audio data based Vggish model. |
ποΈ |
Search Tumblr GIFs with KerasEncoderUse prefetching and sharding to improve the performance of your index and query Flow when searching animated GIFs. |
Apart from the learning resources above, We highly recommended you go through our [**documentation**](https://docs.jina.ai) to master Jina.
Our docs are built on every push, merge, and release of Jina's master branch. Documentation for older versions is archived [here](https://github.com/jina-ai/docs/releases).
Are you a "Doc"-star? Join us! We welcome all kinds of improvements on the documentation.
## Contributing
We welcome all kinds of contributions from the open-source community, individuals and partners. We owe our success to your active involvement.
- [Contributing guidelines](CONTRIBUTING.md)
- [Release cycles and development stages](RELEASE.md)
### Contributors β¨
[](#contributors-)
## Community
- [Code of conduct](https://github.com/jina-ai/jina/blob/master/.github/CODE_OF_CONDUCT.md) - play nicely with the Jina community
- [Slack workspace](https://slack.jina.ai) - join #general on our Slack to meet the team and ask questions
- [YouTube channel](https://youtube.com/c/jina-ai) - subscribe to the latest video tutorials, release demos, webinars and presentations.
- [LinkedIn](https://www.linkedin.com/company/jinaai/) - get to know Jina AI as a company and find job opportunities
- [](https://twitter.com/JinaAI_) - follow and interact with us using hashtag `#JinaSearch`
- [Company](https://jina.ai) - know more about our company and how we are fully committed to open-source.
## Open Governance
As part of our open governance model, we host Jina's [Engineering All Hands]((https://hanxiao.io/2020/08/06/Engineering-All-Hands-in-Public/)) in public. This Zoom meeting recurs monthly on the second Tuesday of each month, at 14:00-15:30 (CET). Everyone can join in via the following calendar invite.
- [Add to Google Calendar](https://calendar.google.com/event?action=TEMPLATE&tmeid=MHIybG03cjAwaXE3ZzRrYmVpaDJyZ2FpZjlfMjAyMDEwMTNUMTIwMDAwWiBjXzF0NW9nZnAyZDQ1djhmaXQ5ODFqMDhtY200QGc&tmsrc=c_1t5ogfp2d45v8fit981j08mcm4%40group.calendar.google.com&scp=ALL)
- [Download .ics](https://hanxiao.io/2020/08/06/Engineering-All-Hands-in-Public/jina-ai-public.ics)
The meeting will also be live-streamed and later published to our [YouTube channel](https://youtube.com/c/jina-ai).
## Join Us
Jina is an open-source project. [We are hiring](https://jobs.jina.ai) full-stack developers, evangelists, and PMs to build the next neural search ecosystem in open source.
## License
Copyright (c) 2020-2021 Jina AI Limited. All rights reserved.
Jina is licensed under the Apache License, Version 2.0. [See LICENSE for the full license text.](LICENSE)
