What is a server architecture? You can download a free book to learn about it: Links

directory

  • None Server architecture description
  • An example of asynchronous processing
  • Understand the serverless components of the AWS ecosystem
  • Hybrid non-serverless architecture services
  • Use Java for data processing
  • conclusion

None Server architecture description

There is an alternative to building systems on the cloud: a serverless architecture. It is the basis of Backends-as-A-Service (BAAS). It is easy to expand and handle databases, messages, and user management. They include the common Functions-as-a-Service (FAAS) — small, event-triggered, functional Functions that can be customized and deployed on a fully managed platform that hosts all deployment, resource allocation, extension, maintenance, and monitoring.

Designing a serverless system means infrastructure savings because it shields you from low-level infrastructure construction, error handling, low-level operations, and significantly shortens the time-to-market of a product or feature. There are three types of applications that can take advantage of serverless:

  1. An API interface for synchronous response behind the scenes for mobile, single-page Web, or B2B products. A no-service architecture can speed up its time-to-market cycle;
  2. A combination of logical “glue” for multiple operations or API integration. The no-service architecture works super well because it’s best for writing small pieces of logic.
  3. Asynchronous message and file processing. A no-service architecture can automatically scale the load. This article will cover step 3.

An example of asynchronous processing

“Asynchronous messaging and file processing” is not a rigorous description, but the concept is very general, so the application or component described here should have the following features:

  1. Triggered by a message, files or other types of objects are passed in their context;
  2. Perform some input event processing;
  3. Output to data stores, message components, API endpoints, rather than synchronously returning to the event source;
  4. You can use local state and pull data from external components for processing.

One of the applications of AWS Lambda on the AWS cloud is image file processing. This is great if you have a mobile App or a single page WebApp and need to handle uploaded photos! Just like the following process.



Compare the above proposed features, picture file processing applications:

  1. Triggered by the creation of AWS S3 raw files
  2. Encapsulate the new version of the library with images
  3. Output different versions of images to S3
  4. Stateless, no external components are required.

    A more complex example is a data pipeline. A data pipeline is linked to an ecosystem of asynchronous services that perform data processing, ETL processing, import to an analysis system, or others. These systems are particularly popular for continuously processing big data and providing mobile apps with the ability to view it.

    Serverless data pipeline is one of the applications of the FAAS platform, such as AWS Lambda, which always handles data storage and transfer with a batch of services hosted to serverless BAAS for computing purposes, as shown in the figure below.

Note that the figure above simplifies actual product scenarios, such as loading data into RedShift using Kinesis Firehouse. Against the above definition:

  • Trigger events on the message bus, such as creation of AWS Kinesis Streams or SNS or AWS S3 files or objects, or update of AWS DynamoDB table data;
  • Classic query or transfer events, or disassemble events, or combine events
  • You can create new messages on different buses, write new S3 files, update statistical databases, and communicate with other HTTP services
  • Request remote services to obtain event data to be processed. These services can be non-services, such as DynamoDB or Lambda Http services, or traditional services, such as Redis, Sql databases, etc. Components can cache local state across multiple messages, but cannot use their own local memory or hard disk data because of the short life cycle of FAAS.

Understand the serverless components of the AWS ecosystem

In the example above, a serverless component on the cloud is used by Amazon, although other cloud providers have similar components as well. AWS component functions are described as follows:

  • Lambda: AWS FAAS platform, which provides a large extensible, lightweight programming model that runs in a choice of languages: Javascript, Python, Java/JVM, and c#
  • S3: One of the oldest services in AWS. It is an object storage component that offers high scale, high availability, and accurate and reasonable cost. S3 events can easily be combined with Lambda services.
  • Kinesis Stream: a high-throughput, persistent, and durable sort logging component for AWS, as opposed to Kafaka for Apache. Lambda integrates it and performs message processing.
  • SNS: SMS component;
  • DynamoDB: AWS’s high-performance NoSql data service. It can be triggered as an event source for unserved data processing.

Hybrid non-serverless architecture services

Non-serverless components or services can also be part of a serverless architecture, which would be a perfect world. For example, an Sql database needs to be used within a Lambda service, but it is not a service-free architecture; it may still be part of AWS RDS. The thing to keep in mind about mixing is that the scalability is different, and you need to scale in different ways.

Use Java for data processing

We already mentioned that Lambda can write FAAS code in the AWS ecosystem, and Java is one of the supported languages. Java has a bad reputation in the Lambda world because it has a higher cold start cost compared to Javascript and Python. Cold start occurs when AWS creates a new container for your functions. It usually occurs when you deploy a new version of the service and run it for the first time. In some scenarios, it takes a few seconds to start Java cold. However, In data processing scenarios, Java is often chosen:

  • With some fairly simple adjustments, the cold start effect can usually be reduced to a few hundred milliseconds. This is perfectly reasonable for many asynchronous event processing systems. In addition, for systems that process events continuously, only a small percentage are likely to get a cold start. We mean tiny, maybe 0.001%.
  • Java can typically execute with lower latency than Javascript and Python when dealing with a fairly continuous stream of events. Not only is this good for application performance, but for moderately complex lambda functions that take more than 100 milliseconds to run, you can see the cost savings of using Java as your environment, since lambda’s billing is very precise.
  • Because a number of existing non-serverless data processing applications have been implemented in Java, you can see opportunities for code reuse by porting Java domain logic from older applications to new Lambda components.

conclusion

Serverless architectures offer significant advantages in terms of infrastructure cost, labor, and time to market. In this article, we show how the serverless architecture can be used to build serverless data processing applications of various types and complexities.

Serverless is still the new way to build systems – when you use these ideas, you won’t be able to use conventions to guide your every step. Over the next few years, the best way to use these technologies will become clearer. But for those willing to roll up their sleeves, the benefits of this exciting industry development are plentiful.