During a recent deployment, WHEN I was asked “What is API testing?” I was working with the client on an API testing strategy. That’s when I realized how challenging it is to describe API testing, and even when you do, it often sounds boring and complicated.

Well, let me tell you that API testing isn’t boring or complicated. It’s actually very interesting and powerful, and thinking about it in a different way frees up your ability to create testing strategies that really work. To really understand how to do API testing, read on.

What is an API and how is it used?

In service development, an application program interface (API) is a way for various applications to communicate with each other using a common language, usually defined by a protocol. These examples are Swagger documents for RESTful services or WSDL for SOAP services. Even databases have an interface language, SQL.

Apis enable efficient communication between machines in much the same way that UIs allow humans to interact with applications.

Apis are great because they represent building blocks that developers can use to easily assemble interactions without having to rewrite the interface every time they need a machine to communicate. Also, because apis have protocols, applications that want to talk to each other can be built in completely different ways as long as they talk to each other according to API protocols. This allows different developers from different organizations around the world to create highly distributed applications while using the same API over and over again.

When a user interacts with the front end of an application (i.e. a mobile application), the front end makes API calls to the back end system, simplifying the development process in two main ways:

  • Developers don’t have to worry about creating custom applications for every mobile device or browser.
  • You can update or modify different back-end systems without having to redeploy the entire application each time.

As a result, developers can save time by focusing individual services on discrete tasks rather than spending time writing logic to their applications.

A good example of standard API usage

The amazon Shopping service’s documented API enables developers to interact with Amazon Shopping when creating applications. Developers can use the Amazon API at appropriate times in their user experience to create a seamless customer journey.

For example, this might look like this:

The user experience Corresponding API call
1. Search for great video games 1. SearchItems
2. Amazon suggests Using Minecraft 2. GetItems
Add Minecraft to my shopping cart 3. AddToCart

The user interacts with the user interface, and the application interacts with the developer-defined back-end Amazon API. As long as the underlying API behaves as expected, everything works fine.

… But if it’s a large quantity.

Hence, the importance of testing these apis.

What are API tests?

So how do you perform API tests? What does it really mean? How do I do API testing? Unlike a user who interacts with the application only at the UI level, the developer/tester must ensure the reliability of any and all underlying apis. Without testing the API itself, developers and testers will be stuck with manual testing, much like users testing applications at the UI level until the entire application stack has been built.

Instead, you can perform automated API testing by testing your application at the API level, designing test cases that interact directly with the underlying API, and gaining many advantages (including the ability to test business logic in a stable manner on a layer that is easy to automate). Unlike manual testing, which is limited to verifying a specific user experience, API testing allows you to bulletproof your application against the unknown.

How do I do API testing?

Different types of API testing — where, why, and how

The best way to do API testing is to build solid testing practices from the bottom up. To this end, a good way to design a testing strategy is to follow Martin Fowler’s Testing Pyramid. The pyramid approach suggests that you build various API tests (for example, protocols, schemas, performance, etc.) on top of a solid foundation of unit tests with UI tests. API testing allows you to test application logic at a level that unit tests cannot.

These testing strategies are complementary. Testing earlier, in the lower layers of your application, helps you “fail fast and bug early,” finding defects at source sooner rather than in the SDLC. Unit testing is important, but for now we’re focusing on API testing. How do you test APIS? What tests can you run? Why are they important? How do I do API testing?

How do you use the API?

The next section covers the different types of API tests, including where/why/how to use them.

Protocol testing

An API represents a protocol between two or more applications. Protocols describe how to interact with interfaces, the services available, and how to invoke them. The agreement is important because it is the basis for communication. If there’s something wrong with the protocol, then nothing else matters.

The first and most basic type of API testing is protocol testing, which tests the service protocol itself (Swagger, PACT, WSDL, or RAML). This type of test verifies that the protocol is written correctly and can be used by customers. This test works by creating a series of tests that can pull in protocols and verify the following conditions:

  • The service agreement is written according to the specification
  • Message request and response are semantically correct (schema validation)
  • Endpoint valid (HTTP, MQ/JMS topic/queue, etc.)
  • The service agreement has not changed

I think these are your first “smoke tests”. If these tests fail, there is really no reason to continue testing that particular service. If these tests pass, you can move on to testing the actual functionality of the API.

Component test

Component testing is like unit testing of an API — you want to take the various methods available in the API and test each of them individually. These tests are created by performing test steps on each method or resource available in the service protocol.

The easiest way to create component tests is to consume the service agreement and let it create the client. You can then data-drive each individual test case using positive and negative data to verify that the response returned has the following characteristics:

  • Request payload format is correct (schema validation)
  • Correct response payload format (schema verification)
  • Response status as expected (200 OK, returned SQL result set, even an error if you want to do so)
  • The response error payload contains the correct error message
  • The response matches the expected baseline. This can take two forms:
    • Regression/variance — The response payload looks exactly the same from call to call (a top-down approach where you actually take a snapshot of the response and validate it each time). This can also be an important catalyst for identifying API changes (more on that later).
    • Assertion – Each element in the response matches what you expect (this is a more superficial, bottom-up approach to a particular value in the response).
  • The service responds in the expected time

These individual API tests are the most important tests you can build, because they will be utilized in all subsequent testing techniques. Why rebuild test cases when you can simply refer to these individual API calls in all different types of tests later? This not only improves consistency, but also simplifies the process of doing API testing.

Scenario testing

Most people think about scenario testing when they think about API testing. In this testing technique, you assemble individual component tests into a sequence, as in the example I described above for Amazon services.

There are two great techniques for getting a sequence:

  • View the user story to identify the individual API calls that are being made.
  • Practice the UI and capture traffic to the base API.

Scenario testing allows you to see if it is possible to introduce defects by combining different data points together.

I came across a very interesting example when working with clients. They use a range of services to call customers’ financial information, available accounts, credit cards and recent transactions. Each of these API calls works individually, but when you put them together in order, they start reporting errors. The reason for this is a simple timestamp that returns from an API call in a different format than expected in subsequent requests. They don’t realize this when they do unit tests or smoke tests because they assert that no format is specified when the timestamp is returned. It was not until the entire scenario was tested that it became clear that moving the timestamp from one call to another would cause a failure.

Another benefit of scenario testing is that you can verify expected behavior when you use the API in unexpected ways. When you publish an API, you offer the world a set of building blocks. You may have the prescribed technology to put these pieces together, but customers may have unexpected requirements and accidentally group apis together to expose defects in the application. To prevent this, you might want to create many schema tests using different combinations of apis to insulate your application from major failures.

Because component testing forms the basis for scenario testing, organizations often have a large number of scenario tests. They are built when new features are introduced to simulate the customer’s journey with the new features. This way, you can really cut down on testing time because you only need to build tests for new features, and you know you have a solid base of tests to catch any surprises.

The performance test

In performance-specific testing environments, performance testing is often relegated to the end of the testing process. This is because performance testing solutions are often expensive, require specialized skills, and require specific hardware and environments. This is a big problem because apis have service level agreements (SLAs) that must be met before applications can be published. If you wait until the last minute to perform your performance tests, failing to meet your SLA can cause huge release delays.

Performance testing early in the process allows you to find performance-related issues before running a full regression cycle. If you’ve followed the testing process so far, it’s actually quite easy because you have all the basic test cases you need to perform performance testing. You can simply test scenarios, load them into performance testing tools, and then run them with a larger number of users. If these tests fail, you can trace the failure to individual users and better understand the impact. Managers can then use this knowledge to decide whether to release the application.

Security testing

Security testing is important to all stakeholders in an organization. If security vulnerabilities are exposed and exploited, they can result in serious reputational and financial losses. Just as users may accidentally use your API, they may intentionally try to take advantage of it. Hackers can get hold of your API, find vulnerabilities and exploit them.

To prevent this kind of behavior, you need to build test cases to try to execute these types of malicious attacks. You can do this using existing test cases, because schema tests can provide attack vectors to the application. You can then reuse this attack medium to launch infiltration attacks. A good example is to combine different types of parameter fuzziness testing or SQL injection attacks with schema testing. This way, any changes propagated through the application will be selected by your security tests. To learn more about API security testing, check out this article and stay tuned.

Omni-channel testing

Because of the multiple interfaces (mobile, Web, API, database, etc.) with which your application interacts, if you test any of them individually, you will encounter a gap in test coverage, missing the subtleties of the complex interactions between these interfaces.

Omni-channel testing covers many interfaces of an application by interweaving API and database testing into validation of mobile and Web UI interactions to ensure thorough coverage of testing. This means running a test that is using one of the interfaces and coordinating with the other — performing UI tests like Web (Selenium) or Mobile (Appium), interweaving them with either of your API or database tests, and executing them from the system through tests. With effective omni-channel testing, you can create stable, reusable test cases that can be easily automated.

Managing change

Change is one of the most important indicators of application risk. Change can occur in many forms, including:

  • Protocol message format changes for the service
  • Elements added or removed from the API
  • Underlying code changes affect the format of the data returned
  • Rearchitecting services to decompose them into parts (this is very common as organizations move to microservices)

When changes occur, you need to build test cases to identify the changes and provide a remediation plan. Using a solution that provides analysis to resolve the impact of these changes will allow you to understand what has changed and target the specific tests that are affected.

Changes can then be captured as templates to update any underlying component or schema tests with new functionality. Because the remaining tests reference these tests, the impact of the change is reduced.

conclusion

Establishing a solid automated API testing strategy is the best way to ensure that your application “works today as it did yesterday” (it’s not just an easy phrase to understand). API testing allows you to build a reliable framework for identifying defects in multiple layers of your application. These tests can be fully automated and run continuously, so you can ensure that your application meets business expectations while maintaining functional accuracy. Since API testing works at a much lower level than UI testing, you know that you need to be consistent and that the tests you build will last a long time.

Come on, boy! Are you ready to start API testing? But first you need to figure out what tools to use. Stay tuned for more on how to choose the best API testing solution for your organization.