In the Nebula Graph Overview section, we take you through the Nebula Graph architecture, repository distribution, code structure, and module planning.

1. The architecture

Nebula Graph is an open source distributed graphics database. Nebula uses a storage computing separation design that decouples storage from computing. In addition to the database kernel, we also provide many peripheral tools, such as data import, monitoring, deployment, visualization, graph calculation and so on. For Nebula Design, see Overview of Graph Databases and Nebula’s Practices in Graph Database Design.

The overall architecture design is shown in the figure below:

The query engine adopts stateless design, which can be easily extended horizontally. It is divided into several main parts, such as syntax analysis, semantic analysis, optimizer and execution engine.

For detailed design, see Nebula Graph 2.0 Query Engine Design.

The query engine architecture design is shown in the figure below:

A Storage consists of two parts: meta Storage, called meta Service, and data Storage, called Storage Service.

A Storage Service has three layers: Store Engine. On top of that is our Consensus layer, which implements Multi Group Raft; The top layer, our Storage interfaces, defines a set of graph-related apis.

For detailed design, see Storage Design of graph database.

The architecture design of the storage engine is shown as follows:

2. Code repository overview

Welcome to the Vesoft Repository, the developer of The Nebula Graph database.

The Nebula product architecture currently contains the Graphical database kernel, client, tools, test framework, compilation, visualization, monitoring, and more.

The main purpose of this article is to briefly introduce the Nebula Graph main Repo code structure and explain the basic functionality of the individual modules. More detailed design instructions will follow. Hopefully, this will help Nebula readers understand the Nebula Graph and contribute to the Nebula community by submitting features, fixing bugs, and submitting documentation.

Most of the code repositories in the Vesoft – Inc repository are listed below:

  • Nebula: The nebula 1.0 kernel code
  • Nebula Graph: Nebula 2.0 Query computing engine
  • Nebula Storage: Nebula 2.0 Storage engine
  • Nebula Common: The Nebula 2.0 kernel toolkit
  • Nebula Clients
    • Nebula – Java: Java client
    • Nebula – CPP: CPP client
    • Nebula – Go: Go client
    • Nebula – Python: Python client
  • Nebula Tools
    • Nebula – Importer: A high-performance data importer based on the Go client implementation
    • Nebula – Spark-utils: Contains spark Connector, Exchange, and Algorithm
    • Nebula -br: A backup and restoration tool
    • Nebula – Ansible, nebula- Operator: deployment tools
  • Nebula Test
    • Nebula -bench: Pressure and performance test engineering
    • Nebula – Chaos: A test project for chaos
  • Compiling
    • Nebula -third-party: a third-party package that the Nebula Graph database kernel relies on
    • Nebula – Gears: The Nebula Graph database kernel tool chain
  • Nebula graph-Studio: A Nebula Graph visualization tool

3. Code structure and module description

3.1 Nebula Graph

├ ─ ─ cmake ├ ─ ─ the conf ├ ─ ─ LICENSES ├ ─ ─ package ├ ─ ─ resources ├ ─ ─ scripts ├ ─ ─ the SRC │ ├ ─ ─ the context │ ├ ─ ─ daemons │ ├ ─ ─ executor │ ├ ─ ─ optimizer │ ├ ─ ─ parser │ ├ ─ ─ planner │ ├ ─ ─ the scheduler │ ├ ─ ─ service │ ├ ─ ─ the session │ ├ ─ ─ stats │ ├ ─ ─ util │ ├ ─ ─ The validator │ └ ─ ─ the visitor └ ─ ─ tests ├ ─ ─ the admin ├ ─ ─ bench ├ ─ ─ common ├ ─ ─ data ├ ─ ─ job ├ ─ ─ maintain ├ ─ ─ mutate ├ ─ ─ query └ ─ ─ tckCopy the code
  • Conf / : Queries the engine configuration file directory
  • Package / : graph packaging script
  • Resources / : Resource files
  • Scripts / : Startup script
  • SRC / : query engine source directory
    • SRC /context/ : Information about the context of the query, including the AST, Execution Plan, Execution results, and other computation-related resources.
    • SRC /daemons/ : Queries the main engine process
    • SRC /executor/ : executor, implementation of each operator
    • SRC/Optimizer / : RBO (rule-based optimization) implementation, and optimization rules
    • SRC /parser/ : lexical parsing, syntax parsing, : AST structure definition
    • SRC/Planner / : Operator, and execution plan generation
    • SRC /scheduler/ : The scheduler that executes the schedule
    • SRC /service/ : the Query engine service layer, providing authentication and executing Query interfaces
    • SRC /session/ : session management
    • SRC /stats/ : Perform statistics, such as P99, slow query statistics, etc
    • SRC /util/ : utility functions
    • SRC/Validator / : Semantic analysis implementation, used to check for semantic errors and perform some simple rewrite optimizations
    • SRC /visitor/ : Expression visitor, used to extract expression information, or optimize
  • Tests / : A BDD-based integration testing framework that tests all of the features offered by Nebula Graph

3.2 Nebula Storage

├ ─ ─ cmake ├ ─ ─ the conf ├ ─ ─ docker ├ ─ ─ docs ├ ─ ─ LICENSES ├ ─ ─ package ├ ─ ─ scripts └ ─ ─ the SRC ├ ─ ─ the codec ├ ─ ─ daemons ├ ─ ─ kvstore ├─ Meta └── Heavy ├─ Storage ├─ Tools ├── bass ├─ versionCopy the code
  • Conf / : storage engine configuration file directory
  • Package / : storage Packaging script
  • Scripts / : Startup script
  • SRC / : storage engine source directory
    • SRC /codec/ : serialization deserialization tool
    • SRC /daemons/ : main process of the storage engine and metadata engine
    • SRC/Kvstore / : Distributed KV storage implementation based on RAFT
    • SRC /meta/ : Enables metadata management services based on KVStore, which can be used to manage metadata, clusters, and long-duration tasks
    • SRC /storage/ : Implements the graph data storage engine based on KVStore
    • SRC /tools/ : Some gadget implementations
    • SRC /utils/ : code utility functions

3.3 Nebula Common

├ ─ ─ cmake │ └ ─ ─ nebula ├ ─ ─ LICENSES ├ ─ ─ the SRC │ └ ─ ─ common │ ├ ─ ─ algorithm │ ├ ─ ─ base │ ├ ─ ─ charset │ ├ ─ ─ clients │ ├ ─ ─ Concurrent │ ├─ conf │ ├── Context │ ├─ CPP │ ├─ datatypes │ Encryption │ ├─ expression │ ├─ fs │ ├─ function │ ├ ─ ─ graph │ ├ ─ ─ HDFS │ ├ ─ ─ HTTP │ ├ ─ ─ interface │ ├ ─ ─ meta │ ├ ─ ─ network │ ├ ─ ─ the plugin │ ├ ─ ─ process │ ├ ─ ─ the session │ ├ ─ ─ stats │ ├ ─ ─ the test │ ├ ─ ─ thread │ ├ ─ ─ thrift │ ├ ─ ─ time │ ├ ─ ─ version │ └ ─ ─ webservice └ ─ ─ third - partyCopy the code

The Nebula Common repository code is a toolkit for the Nebula kernel code that provides efficient implementations of Common tools. Some of the common toolkits are probably familiar to engineers as well. Only those directories that are closely related to the graph database are described here.

  • SRC /common/clients/ : meta, CPP implementation for storage clients
  • SRC /common/datatypes/ : Nebula Graph definitions of datatypes and computations such as string, int, bool, float, Vertex, Edge, etc.
  • Rc /common/expression/ : Defines expressions in nGQL
  • SRC /common/function/ : defines functions in nGQL
  • SRC /common/interface/ : Graph, meta, storage service interface definition

The above is the introduction of this article.

Ac graph database technology? To join the Nebula Exchange group, please fill out your Nebula card and Nebula Assistant will bring you into the group