A list of technologies similar to inner Yandex technologies.
This repository inspired by xg2xg.
This file may be good for experienced developers who:
- Starts to working at Yandex. In that case this document helps you translate architecture based for example on Kafka and Airflow to Logbroker and Nirvana.
- Leave Yandex. In that case this document helps you translate architecture based for example on Logbroker and Nirvana to Kafka and Airflow.
- Just want to see a list of "big guns".
| Technology | Yandex internal | Analogs |
|---|---|---|
| Version control system | arc |
git (Open source) svn (Open source) perforce (Commercial) |
| Build tool | ya make |
CMake (Open source) Bazel (Open source) |
| Technology | Yandex internal | Analogs |
|---|---|---|
| Content-Based Image Retrieval | Siberia |
| Technology | Yandex internal | Analogs |
|---|---|---|
| Morphological analyzer | MyStem |
MyStem (Free by Yandex) Natasha (Open source) |
| Structured data extractor | Tomita parser | Tomita parser (Open source by Yandex) |
| Technology | Yandex internal | Analogs |
|---|---|---|
| Map-reduce |
Real Time MapReduce YT |
YTsaurus (Open source by Yandex) Hadoop (Open source) Spark (Open source) |
| Map-reduce cascading | Nile | Airflow (Open source) |
| Querry language over map-reduce | YQL | Hive (Open source) |
| Column oriented database | ClickHouse |
ClickHouse (Open source by Yandex) Vertica (Commercial by Vertica) |
| Workflow manager | Nirvana |
Airflow (Open source by Apache) NiFi (Open source by Apache) PLynx (Open source) n8n (Open source) for some cases |
| Data deploy system | Market Access | |
| Message broker | Logbroker | Kafka (Open source) |
| Distributed key-value storage | Elliptics |
Elliptics (Open source) Redis (Open source) Consul (KV part, open source by HashiCorp) |
| Multithreaded PostgreSQL connection pool | Odyssey |
Odyssey (Open source by Yandex) |
| Distributed DataBase | Yandex Database (YDB) | Yandex Database (YDB) (Open source by Yandex) |
| Technology | Yandex internal | Analogs |
|---|---|---|
| BEM | BEM | BEM (Open source) |
| Frontend testing frameworks |
Gemini Hermione HTML Elements |
Gemini (Open source by Yandex) Hermione (Open source by Yandex) Html Elements (Open source by Yandex) |
| Technology | Yandex internal | Analogs |
|---|---|---|
| C++ backend framework | userver | userver (Open source by Yandex) |
| Technology | Yandex internal | Analogs |
|---|---|---|
| Stress testing tool | Yandex Tank |
Yandex Tank (Open source by Yandex) JMeter (Open source by Apache) Gatling (Open source) |
| Stress testing manager | Lunapark | |
| Testing report framework | Allure | Allure (Open source) |
| Java Matchers library | Matchers Java |
Matchers Java (Open source by Yandex) Hamcrest matchers (Open source by Hamcrest) |
| Test data generator | ObjectBuilders | ObjectBuilders (Open source by Yandex) |
| Microservice-oriented test framework | TestSuite | TestSuite (Open source by Yandex) |
| Technology | Yandex internal | Analogs |
|---|---|---|
| Service mesh | AppHost | Consul (Open source by HashiCorp) |
| Microservices firewall | TVM | Consul (Open source by HashiCorp) |
| Continuous Integration | CI |
Jenkins (Open source) TeamCity (Commercial by JetBrains) |
| Container virtualization | Porto |
Porto (Open source by Yandex) Docker (Open source) |
| Iaas/PaaS |
Cocaine Yandex.Cloud |
Cocaine (Open source by Yandex) OpenShift (Open source by RedHat) Yandex.Cloud (Commercial by Yandex) AWS (Commercial by Amazon) Azure (Commercial by Microsoft) |
| Nginx configuration tool | Gixy | Gixy (Open source by Yandex) |
| L7 load balancer | Balancer (Archived) | Envoy |
| Cluster resource planning | Yandex Planner | Kubernetes |
| Distributed rate limiter | YARL |
| Technology | Yandex internal | Analogs |
|---|---|---|
| Documentation framework | Diplodoc | Diplodoc (Open source by Yandex) |
| Technology | Yandex internal | Analogs |
|---|---|---|
| Machine learning framework |
FML MatrixNet | Caliban (Open source by Google) |
| Simple tasks outsoursing | Toloka |
Toloka (Commercial by Yandex) Amazon Mechanical turk (Commercial by Amazon) |
| Gradient boosting library | CatBoost |
CatBoost (Open source by Yandex) XGBoost (Open source) LightGBM (Open source by Microsoft) |
| Sharded Data Parallelism framework | YaFSDP | YaFSDP (Open source by Yandex) |
| Technology | Yandex internal | Analogs |
|---|---|---|
| Issue tracker | Tracker |
Tracker (Commercial by Yandex) Jira (Commercial by Atlassian) |
| Wiki engine | Wiki |
Wiki (Commercial by Yandex) Mediawiki (Open source) Confluence (Commercial by Atlassian) |
Feel free to add any technologies. If you add internal Yandex technology, you must provide link that prooves this technology is not under NDA. Usually this is link to official Yandex blog on habr.com, link to official Yandex technologies catalog or link to official Yandex GitHub repository.
Big markdown tables are messy. Big html tables are just a little less messy, so I choosed html.