Planning Editor | Natalie
Nuclear Coke
AI Front Line introduction:In 2015, Apple acquired data provider FoundationDB to improve its App Store, iTunes Connect, and iTunes services in the cloud. FoundationDB then went from open source to closed source, and the open source community complained. Now, three years later, Apple has announced that FoundationDB is open source again. Now that FoundationDB has been in development and production at Apple for years, how does it bring new changes to the competitive distributed database landscape?






Please pay attention to the wechat public account “AI Front”, (ID: AI-front)

FoundationDB is “a distributed database that can host large-scale structured data on multiple cluster servers.” The database system focuses on high performance, high scalability and good fault tolerance. This means that projects using the database run more efficiently and are less expensive to maintain.

FoundationDB was developed in 2009 by Dave Rosenthal, Dave Scherer, and Nick Lavezzo Aim to build a NoSQL database that complies with ACID constraints (ACID is a database mechanism that guarantees data integrity even when errors occur).

Apple acquired FoundationDB in 2015 and may already be using it for iCloud. In an introduction to CloudKit, Apple engineers mention the use of a NoSQL database that allows application developers to synchronize user data across devices in a common and easy-to-use way. CloudKit provides data storage services for many iCloud services, including iOS backup, photo syncing, iWork office, and iCloud Web disk.

Apple, known for its closed systems, was interested in a small team product like FoundationDB because it continues to fill in its DNA in the era of big data, and FoundationDB is an enhanced NoSQL (similar to Hbase, And run ACID transactions), which is a good way for Apple to boost its cloud capabilities.

However, at that time, the open source community was dissatisfied. Developers and participants of the open source ecosystem expressed concern and anxiety about the commercial acquisition of open source projects, believing that it would increase the risk for the contributors and participants of open source projects, and even hurt or even kill the open source projects themselves.

Just in the wee hours of this morning, Apple announced that the FoundationDB distributed database is open source again!

The news caused a stir on HackerNews, where it received more than 300 likes and 78 comments in the first hour. As of press time, the story has received 1,381 likes and 278 comments. Wwilson said:

This news is really exciting! FoundationDB is the best software I’ve ever used! This is an excellent set of prototypes for anyone who wants to build distributed systems.

In short, FoundationDB is a massively scalable, fast, distributed database with the best testing and fault tolerance. FoundationDB is widely used by Apple and many other large vendors.

But the really interesting part is that it provides a very efficient underlying interface for other systems that need scalability and storage consistency. At FoundationDB, our primary goal is to use FoundationDB to create a variety of database frontends that use different data models and query languages (SQL, document, etc.) and ensure that these languages store data in the same underlying system. On this basis, customers can choose the database front-end they need at will, or even select multiple at the same time, but still only need to face the same distributed database solution.

Such thinking is undoubtedly far-sighted. The ease of implementing the Zookeeper API on FoundationDB further reduces the burden. Additionally, FoundationDB perfectly solves the need for metadata storage on distributed file systems, as well as distributed task queues. You can even replace the Lucene/ElasticSearch index with a more scalable solution!

That’s why Apple wisely chose FoundationDB. Apple is a focused company and there is no reason for them to write different storage layers for different types of data. By choosing FoundationDB and investing in development resources, every product that comes out of the community will pay off for Apple. It really is an ideal virtuous circle.

FoundationDB website statement
A new chapter

Starting today, FoundationDB will begin its next chapter as an open source project.

FoundationDB is a distributed data storage solution designed with the deployment requirements of commercial hardware clusters in mind. These clusters allow users to add machines at will and scale well, automate hardware failure recovery, and provide simple apis. Its key-value store supports comprehensive global, cross-row ACID transactions, thus providing the highest level of data consistency. More specifically, such strong consistency will make everyone’s application code simpler, data models more efficient, and failure patterns easier to deal with.

Most importantly, FoundationDB is very mature, having been in development and production at Apple for years. We plan to transform FoundationDB into a community project, and we warmly welcome your participation.

Strong abstraction ability

We believe FoundationDB will be the foundation for the next generation of distributed databases. Since its birth in 2010, the entire database world has been gradually aligning with FoundationDB to support internal data consistency.

FoundationDB’s vision is to start with a simple, powerful kernel and then extend it by adding “layers.” The core of this open source key-value store database focuses on integrating functionality that cannot be written into the layers. Each layer adds functionality to simulate specific types of data while handling their access patterns for core extension.

FoundationDB’s infrastructure, including its use of tier concepts, significantly complements best practices for system scalability and manageability. By running multiple layers (such as the document storage layer and the graphics layer) on a single cluster, users can match specific applications to the best data models, reducing the cost of running the infrastructure and ultimately helping businesses save on operational and technical overhead.

By open-source the FoundationDB core, we expect the number and variety of layers to grow rapidly. Looking at the FoundationDB community, we drive growth around the core itself and the layer ecosystem it supports.

Building open Communities

By making the FoundationDB project open source, our goal is to build an open technology community. All major development outcomes will be achieved in the context of openness. We have issued an overview document aimed at ensuring transparency and taking into account the views of community members. In addition, we have taken a series of steps to develop a project governance approach that will provide an infrastructure to help community members actively participate and have a greater say in decision making.

We want to build a healthy and vibrant community for FoundationDB. To this end, we have also adopted a code of conduct based on the contributor engagement, which sets out the types of behaviour we encourage and prohibit.

We look forward to your participation. You can join our community in the following ways:

  • Ask questions in the FoundationDB community forum: forums.foundationdb.org. Here we discuss various user-specific issues (such as how to use a feature) and development issues (how to dig deep into the core of FoundationDB and tweak a feature). Come and say hello to everyone!

  • Improve the software by reporting bugs via GitHub.

  • Contribute to the core software and documentation (see the project’s contribution guide).

Start now!

The latest release of FoundationDB is 5.1.5. The FoundationDB source code is available at

github.com/apple/foundationdb

Please refer to the guidelines: https://apple.github.io/foundationdb/getting-started-mac.html, in order to understand and how to install, use and development FoundationDB related basic knowledge. Can also visit www.foundationdb.org/download/ to obtain suitable for macOS, Windows and Linux binary installer.

If you’re interested in FoundationDB, check out the project’s GitHub address. Please share your experience with AI Front!

Original link:

https://www.foundationdb.org/blog/foundationdb-is-open-source/


Please pay attention to the wechat public account “AI Front”, (ID: AI-front)