# DeepPavlov
**Repository Path**: deeplearningrepos/DeepPavlov
## Basic Information
- **Project Name**: DeepPavlov
- **Description**: An open source library for deep learning end-to-end dialog systems and chatbots.
- **Primary Language**: Unknown
- **License**: Apache-2.0
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No
## Statistics
- **Stars**: 0
- **Forks**: 0
- **Created**: 2021-03-30
- **Last Updated**: 2021-08-31
## Categories & Tags
**Categories**: Uncategorized
**Tags**: None
## README
[](https://github.com/deepmipt/DeepPavlov/blob/master/LICENSE)

[](https://pepy.tech/project/deeppavlov)
DeepPavlov is an open-source conversational AI library built on [TensorFlow](https://www.tensorflow.org/), [Keras](https://keras.io/)
and [PyTorch](https://pytorch.org/).
DeepPavlov is designed for
* development of production ready chat-bots and complex conversational systems,
* research in the area of NLP and, particularly, of dialog systems.
## Quick Links
* Demo [*demo.deeppavlov.ai*](https://demo.deeppavlov.ai/)
* Documentation [*docs.deeppavlov.ai*](http://docs.deeppavlov.ai/)
* Model List [*docs:features/*](http://docs.deeppavlov.ai/en/master/features/overview.html)
* Contribution Guide [*docs:contribution_guide/*](http://docs.deeppavlov.ai/en/master/devguides/contribution_guide.html)
* Issues [*github/issues/*](https://github.com/deepmipt/DeepPavlov/issues)
* Forum [*forum.deeppavlov.ai*](https://forum.deeppavlov.ai/)
* Blogs [*medium.com/deeppavlov*](https://medium.com/deeppavlov)
* Tutorials [*examples/*](https://github.com/deepmipt/DeepPavlov/tree/master/examples) and [extended colab tutorials](https://github.com/deepmipt/dp_tutorials)
* Docker Hub [*hub.docker.com/u/deeppavlov/*](https://hub.docker.com/u/deeppavlov/)
* Docker Images Documentation [*docs:docker-images/*](http://docs.deeppavlov.ai/en/master/intro/installation.html#docker-images)
Please leave us [your feedback](https://forms.gle/i64fowQmiVhMMC7f9) on how we can improve the DeepPavlov framework.
**Models**
[Named Entity Recognition](http://docs.deeppavlov.ai/en/master/features/models/ner.html) | [Slot filling](http://docs.deeppavlov.ai/en/master/features/models/slot_filling.html)
[Intent/Sentence Classification](http://docs.deeppavlov.ai/en/master/features/models/classifiers.html) | [Question Answering over Text (SQuAD)](http://docs.deeppavlov.ai/en/master/features/models/squad.html)
[Knowledge Base Question Answering](http://docs.deeppavlov.ai/en/master/features/models/kbqa.html)
[Sentence Similarity/Ranking](http://docs.deeppavlov.ai/en/master/features/models/neural_ranking.html) | [TF-IDF Ranking](http://docs.deeppavlov.ai/en/master/features/models/tfidf_ranking.html)
[Morphological tagging](http://docs.deeppavlov.ai/en/master/features/models/morphotagger.html) | [Syntactic parsing](http://docs.deeppavlov.ai/en/master/features/models/syntaxparser.html)
[Automatic Spelling Correction](http://docs.deeppavlov.ai/en/master/features/models/spelling_correction.html) | [ELMo training and fine-tuning](http://docs.deeppavlov.ai/en/master/apiref/models/elmo.html)
[Speech recognition and synthesis (ASR and TTS)](http://docs.deeppavlov.ai/en/master/features/models/nemo.html) based on [NVIDIA NeMo](https://nvidia.github.io/NeMo/index.html)
[Entity Linking](http://docs.deeppavlov.ai/en/master/features/models/entity_linking.html) | [Multitask BERT](http://docs.deeppavlov.ai/en/master/features/models/multitask_bert.html)
**Skills**
[Goal(Task)-oriented Bot](http://docs.deeppavlov.ai/en/master/features/skills/go_bot.html) | [Seq2seq Goal-Oriented bot](http://docs.deeppavlov.ai/en/master/features/skills/seq2seq_go_bot.html)
[Open Domain Questions Answering](http://docs.deeppavlov.ai/en/master/features/skills/odqa.html) | [eCommerce Bot](http://docs.deeppavlov.ai/en/master/features/skills/ecommerce.html)
[Frequently Asked Questions Answering](http://docs.deeppavlov.ai/en/master/features/skills/faq.html) | [Pattern Matching](http://docs.deeppavlov.ai/en/master/features/skills/pattern_matching.html)
**Embeddings**
[BERT embeddings for the Russian, Polish, Bulgarian, Czech, and informal English](http://docs.deeppavlov.ai/en/master/features/pretrained_vectors.html#bert)
[ELMo embeddings for the Russian language](http://docs.deeppavlov.ai/en/master/features/pretrained_vectors.html#elmo)
[FastText embeddings for the Russian language](http://docs.deeppavlov.ai/en/master/features/pretrained_vectors.html#fasttext)
**Auto ML**
[Tuning Models with Evolutionary Algorithm](http://docs.deeppavlov.ai/en/master/features/hypersearch.html)
**Integrations**
[REST API](http://docs.deeppavlov.ai/en/master/integrations/rest_api.html) | [Socket API](http://docs.deeppavlov.ai/en/master/integrations/socket_api.html) | [Yandex Alice](http://docs.deeppavlov.ai/en/master/integrations/yandex_alice.html)
[Telegram](http://docs.deeppavlov.ai/en/master/integrations/telegram.html) | [Microsoft Bot Framework](http://docs.deeppavlov.ai/en/master/integrations/ms_bot.html)
[Amazon Alexa](http://docs.deeppavlov.ai/en/master/integrations/amazon_alexa.html) | [Amazon AWS](http://docs.deeppavlov.ai/en/master/integrations/aws_ec2.html)
## Installation
0. We support `Linux` and `Windows` platforms, `Python 3.6` and `Python 3.7`
* **`Python 3.5` is not supported!**
* **installation for `Windows` requires `Git`(for example, [git](https://git-scm.com/download/win)) and `Visual Studio 2015/2017` with `C++` build tools installed!**
1. Create and activate a virtual environment:
* `Linux`
```
python -m venv env
source ./env/bin/activate
```
* `Windows`
```
python -m venv env
.\env\Scripts\activate.bat
```
2. Install the package inside the environment:
```
pip install deeppavlov
```
## QuickStart
There is a bunch of great pre-trained NLP models in DeepPavlov. Each model is
determined by its config file.
List of models is available on
[the doc page](http://docs.deeppavlov.ai/en/master/features/overview.html) in
the `deeppavlov.configs` (Python):
```python
from deeppavlov import configs
```
When you're decided on the model (+ config file), there are two ways to train,
evaluate and infer it:
* via [Command line interface (CLI)](#command-line-interface-cli) and
* via [Python](#python).
#### GPU requirements
To run supported DeepPavlov models on GPU you should have [CUDA](https://developer.nvidia.com/cuda-toolkit) 10.0
installed on your host machine and TensorFlow with GPU support (`tensorflow-gpu`)
installed in your python environment. Current supported TensorFlow version is 1.15.2.
Run
```
pip install tensorflow-gpu==1.15.2
```
before installing model's package requirements to install supported `tensorflow-gpu` version.
Before making choice of an interface, install model's package requirements
(CLI):
```bash
python -m deeppavlov install
```
* where `` is path to the chosen model's config file (e.g.
`deeppavlov/configs/ner/slotfill_dstc2.json`) or just name without
*.json* extension (e.g. `slotfill_dstc2`)
### Command line interface (CLI)
To get predictions from a model interactively through CLI, run
```bash
python -m deeppavlov interact [-d]
```
* `-d` downloads required data -- pretrained model files and embeddings
(optional).
You can train it in the same simple way:
```bash
python -m deeppavlov train [-d]
```
Dataset will be downloaded regardless of whether there was `-d` flag or not.
To train on your own data you need to modify dataset reader path in the
[train config doc](http://docs.deeppavlov.ai/en/master/intro/config_description.html#train-config).
The data format is specified in the corresponding model doc page.
There are even more actions you can perform with configs:
```bash
python -m deeppavlov [-d]
```
* `` can be
* `download` to download model's data (same as `-d`),
* `train` to train the model on the data specified in the config file,
* `evaluate` to calculate metrics on the same dataset,
* `interact` to interact via CLI,
* `riseapi` to run a REST API server (see
[doc](http://docs.deeppavlov.ai/en/master/integrations/rest_api.html)),
* `telegram` to run as a Telegram bot (see
[doc](http://docs.deeppavlov.ai/en/master/integrations/telegram.html)),
* `msbot` to run a Miscrosoft Bot Framework server (see
[doc](http://docs.deeppavlov.ai/en/master/integrations/ms_bot.html)),
* `predict` to get prediction for samples from *stdin* or from
** if `-f ` is specified.
* `` specifies path (or name) of model's config file
* `-d` downloads required data
### Python
To get predictions from a model interactively through Python, run
```python
from deeppavlov import build_model
model = build_model(, download=True)
# get predictions for 'input_text1', 'input_text2'
model(['input_text1', 'input_text2'])
```
* where `download=True` downloads required data from web -- pretrained model
files and embeddings (optional),
* `` is path to the chosen model's config file (e.g.
`"deeppavlov/configs/ner/ner_ontonotes_bert_mult.json"`) or
`deeppavlov.configs` attribute (e.g.
`deeppavlov.configs.ner.ner_ontonotes_bert_mult` without quotation marks).
You can train it in the same simple way:
```python
from deeppavlov import train_model
model = train_model(, download=True)
```
* `download=True` downloads pretrained model, therefore the pretrained
model will be, first, loaded and then train (optional).
Dataset will be downloaded regardless of whether there was ``-d`` flag or
not.
To train on your own data you need to modify dataset reader path in the
[train config doc](http://docs.deeppavlov.ai/en/master/intro/config_description.html#train-config).
The data format is specified in the corresponding model doc page.
You can also calculate metrics on the dataset specified in your config file:
```python
from deeppavlov import evaluate_model
model = evaluate_model(, download=True)
```
There are also available integrations with various messengers, see
[Telegram Bot doc page](http://docs.deeppavlov.ai/en/master/integrations/telegram.html)
and others in the Integrations section for more info.
## Breaking Changes
**Breaking changes in version 0.7.0**
- in dialog logger config file [dialog_logger_config.json](deeppavlov/utils/settings/dialog_logger_config.json) `agent_name` parameter was renamed to `logger_name`,
the default value was changed
- Agent, Skill, eCommerce Bot and Pattern Matching classes were moved to [deeppavlov.deprecated](deeppavlov/deprecated)
- [AIML Skill](http://docs.deeppavlov.ai/en/0.7.0/features/skills/aiml_skill.html),
[RASA Skill](http://docs.deeppavlov.ai/en/0.7.0/features/skills/rasa_skill.html),
[Yandex Alice](http://docs.deeppavlov.ai/en/0.7.0/integrations/yandex_alice.html),
[Amazon Alexa](http://docs.deeppavlov.ai/en/0.7.0/integrations/amazon_alexa.html),
[Microsoft Bot Framework](http://docs.deeppavlov.ai/en/0.7.0/integrations/ms_bot.html) and
[Telegram integration](http://docs.deeppavlov.ai/en/0.7.0/integrations/telegram.html) interfaces were changed
- `/start` and `/help` Telegram messages were moved from `models_info.json` to [server_config.json](deeppavlov/utils/settings/server_config.json)
- [risesocket](http://docs.deeppavlov.ai/en/0.7.0/integrations/socket_api.html) request and response format was changed
- [riseapi](http://docs.deeppavlov.ai/en/0.7.0/integrations/rest_api.html#advanced-configuration) and
[risesocket](http://docs.deeppavlov.ai/en/0.7.0/integrations/socket_api.html#advanced-configuration) model-specific
properties parametrization was changed
**Breaking changes in version 0.6.0**
- [REST API](http://docs.deeppavlov.ai/en/0.6.0/integrations/rest_api.html):
- all models default endpoints were renamed to `/model`
- by default model arguments names are taken from `chainer.in`
[configuration parameter](http://docs.deeppavlov.ai/en/0.6.0/intro/configuration.html) instead of pre-set names
from a [settings file](http://docs.deeppavlov.ai/en/0.6.0/integrations/settings.html)
- swagger api endpoint moved from `/apidocs` to `/docs`
- when using `"max_proba": true` in
a [`proba2labels` component](http://docs.deeppavlov.ai/en/0.6.0/apiref/models/classifiers.html) for classification,
it will return single label for every batch element instead of a list. One can set `"top_n": 1`
to get batches of single item lists as before
**Breaking changes in version 0.5.0**
- dependencies have to be reinstalled for most pipeline configurations
- models depending on `tensorflow` require `CUDA 10.0` to run on GPU instead of `CUDA 9.0`
- scikit-learn models have to be redownloaded or retrained
**Breaking changes in version 0.4.0!**
- default target variable name for [neural evolution](https://docs.deeppavlov.ai/en/0.4.0/intro/hypersearch.html#parameters-evolution-for-deeppavlov-models)
was changed from `MODELS_PATH` to `MODEL_PATH`.
**Breaking changes in version 0.3.0!**
- component option `fit_on_batch` in configuration files was removed and replaced with adaptive usage of the `fit_on` parameter.
**Breaking changes in version 0.2.0!**
- `utils` module was moved from repository root in to `deeppavlov` module
- `ms_bot_framework_utils`,`server_utils`, `telegram utils` modules was renamed to `ms_bot_framework`, `server` and `telegram` correspondingly
- rename metric functions `exact_match` to `squad_v2_em` and `squad_f1` to `squad_v2_f1`
- replace dashes in configs name with underscores
**Breaking changes in version 0.1.0!**
- As of `version 0.1.0` all models, embeddings and other downloaded data for provided configurations are
by default downloaded to the `.deeppavlov` directory in current user's home directory.
This can be changed on per-model basis by modifying
a `ROOT_PATH` [variable](http://docs.deeppavlov.ai/en/master/intro/configuration.html#variables)
or related fields one by one in model's configuration file.
- In configuration files, for all features/models, dataset readers and iterators `"name"` and `"class"` fields are combined
into the `"class_name"` field.
- `deeppavlov.core.commands.infer.build_model_from_config()` was renamed to `build_model` and can be imported from the
`deeppavlov` module directly.
- The way arguments are passed to metrics functions during training and evaluation was changed and
[documented](http://docs.deeppavlov.ai/en/0.4.0/intro/config_description.html#metrics).
## License
DeepPavlov is Apache 2.0 - licensed.
## The Team
DeepPavlov is built and maintained by [Neural Networks and Deep Learning Lab](https://www.facebook.com/deepmipt/)
at [MIPT](https://mipt.ru/english/).