Find out what to test

Before writing any tests, it is important to understand the basics. What do you need to test?

If your goal is to extend an existing application, you should first write tests for any components you plan to change.

In general, tests should cover:

  • Core functionality: Model classes and methods and their interactions with controllers
  • The most common UI workflow
  • The boundary conditions
  • Bug fix

Testing best practices

A concise set of criteria for effective unit testing is described below. The criteria are:

  • fast: Tests should be done quickly.
  • Independence/isolation: Tests should not share state with each other.
  • repeatable: You should get the same results every time you run the test. Intermittent failures may result from external data providers or concurrent problems.
  • Self validation: Tests should be fully automated. The output should be “pass” or “fail” and not depend on the programmer’s interpretation of the log file.
  • In a timely manner: Ideally, you should write tests (test-driven development) before you write production code for the tests.

Following the guidelines above will make your tests clear and useful, and not a hindrance to your application.

An introduction to

There are two separate introductory projects: BullsEye and HalfTunes.

  • BullsEyeBased on the sample application from iOS Apprentice. Game logicBullsEyeGameIn this class, you will test it in this tutorial.
  • HalfTunesIs an updated version of the URLSession tutorial sample application. Users can query songs in the iTunes API, then download and play snippets of songs.

Unit tests in Xcode

The test navigation provides the easiest way to work with tests; You will use it to create test targets and run tests against your application.

Create a unit test target

Open the BullsEye project and press Command-6 to open the Test navigator.

Click the *+* button in the lower left corner and select New Unit Test Target from the menu:

Accept the default name BullsEyeTests. When the test bundle appears in the Tests navigator, click to open the bundle in the editor. If the bundled software is not displayed automatically, troubleshoot by clicking one of the other navigators, and then return to the Test navigator.

The default template imports the test framework, XCTest, and defines a BullsEyeTests subclass, XCTestCase, with setUp(), tearDown(), and for example test methods.

There are three ways to run tests:

  1. The Product ▸ Build ▸ TestorCommand-U. Both of these runallThe test class.
  2. Click the arrow button in the Test navigator.
  3. Click the diamond button in the binding line.

You can also run a single test method by clicking its diamond in the Test navigator or stapler.

Try different ways to run the tests to get an idea of the time and appearance required. The sample tests haven’t done anything yet, so they run very fast!

When all tests are successful, the diamond turns green and displays a check mark. You can click the gray diamond testPerformanceExample() at the end to open the performance results:

Test the model using XCTAssert

First, you’ll use the XCTAssert function to test the core functionality of the BullsEye model: Does the BullsEyeGame object calculate the score of the turn correctly?

In bullseyetests. swift, add the following line under the import statement:

@testable import BullsEye
Copy the code

This gives unit tests access to internal types and functions in BullsEye.

At the top of the BullsEyeTests class, add the following attributes:

varSut:BullsEyeGame!Copy the code

This creates a placeholder for the placeholder BullsEyeGame, which is an object associated with the system under test (SUT) or this test case class.

Next, replace its contents with setup() :

super.setUp()
sut = BullsEyeGame()
sut.startNewGame()
Copy the code

This creates an object at the class level for BullsEyeGame, so that all tests in that test class have access to the properties and methods of the SUT object.

Here, you also call the game’s startNewGame(), which initializes targetValue. Many tests use targetValue to see if the score is calculated correctly.

Before you forget, release your SUT object tearDown() in. Replace its contents with:

sut = nil 
superTearDown ()Copy the code

Note: It is better to create the SUT in setUp() and release it in tearDown() to ensure that each test starts in the original state.

Write your first test

Now you’re ready to write your first test!

Add the following code to BullsEyeTests at the end:

func testScoreIsComputed(a) {
  // 1. given
  let guess = sut.targetValue + 5

  // 2. when
  sut.check(guess: guess)

  // 3. then
  XCTAssertEqual(sut.scoreRound, 95."Score computed from guess is wrong")}Copy the code

Run the tests by clicking on the binding line or the diamond icon in the Tests navigator. This will build and run the application, and the diamond icon will change to a green check mark!

Note: For a complete list of XCTestAssertions, go to Apple’s Assertions by Category.

Commissioning test

There is a deliberate built-in error in BullsEyeGame, and you will practice finding it. To see errors in the run, you create a test that subtracts 5 from a given section and leaves everything else unchanged. targetValue

Add the following tests:

func testScoreIsComputedWhenGuessLTTarget(a) {
  // 1. given
  let guess = sut.targetValue - 5

  // 2. when
  sut.check(guess: guess)

  // 3. then
  XCTAssertEqual(sut.scoreRound, 95."Score computed from guess is wrong")}Copy the code

The difference between GUESS and targetValue is still 5, so the score should still be 95.

In the Breakpoint navigator, add Test Failure Breakpoint. This stops the test run when the test method has a fault assertion.

Run the test, which should stop on the line where the XCTAssertEqual test failed.

Check the SUT and guess in the debug console:

Guess is targetValue – 5 but scoreRound is 105, not 95!

To investigate further, use the normal debugging procedure: set a breakpoint and check(guess:) in a bullseyeGame.swift, where you create difference. Then run the test again and check the values in the difference application with the let difference statement:

The problem is, the difference is negative, so the score is 100 — (-5). To solve this problem, you should use absolute value difference. In check(guess:), uncomment the correct lines and delete the incorrect lines.

Delete both breakpoints, and then run the test again to confirm that it is now successful.

Test asynchronous operations using XCTestExpectation

Now that you know how to test your model and debug test failures, it’s time to move on to testing asynchronous code.

Open the HalfTunes project. It is used for URLSession to query the iTunes API and download song samples. Suppose you want to modify it to work with AlamoFire for networking. To see if there are any outages, you should write tests for network operations and run them before and after the code changes.

The URLSession methods are asynchronous: they return immediately, but wait until later to run. To test asynchronous methods, you XCTestExpectation can make the test wait for the asynchronous operation to complete.

Asynchronous tests are usually slow, so separate them from faster unit tests.

Create a new unit test target called HalfTunesSlowTests. Open the HalfTunesSlowTests class and import the HalfTunes application module below the existing import statement:

@testable import HalfTunes
Copy the code

All tests in this class use the default value URLSession to send requests to Apple’s servers, so declare a SUT object, create it in, setUp() and then release it tearDown() in.

Replace the contents of the HalfTunesSlowTests class with

var sut: URLSession!
  
override func setUp(a) {
    super.setUp()
    sut = URLSession(configuration: .default)}override func tearDown(a) {
    sut = nil
    super.tearDown()
}
Copy the code

Next, add this asynchronous test:

// Asynchronous test: success fast, failure slow
func testValidCallToiTunesGetsHTTPStatusCode200(a) {
  // given
  let url = URL(string: "https://itunes.apple.com/search?media=music&entity=song&term=abba")
  / / 1
  let promise = expectation(description: "Status code: 200")
  
  // when
  let dataTask = sut.dataTask(with: url!) { data, response, error in
    // then
    if let error = error {
      XCTFail("Error: \(error.localizedDescription)")
      return
    } else if let statusCode = (response as? HTTPURLResponse)? .statusCode {if statusCode == 200 {
        / / 2
        promise.fulfill()
      } else {
        XCTFail("Status code: \(statusCode)")
      }
    }
  }
  dataTask.resume()
  / / 3
  wait(for: [promise], timeout: 5)}Copy the code

This test checks whether sending a valid query to iTunes returns a 200 status code. Most of the code is the same as what you would write in your application, but with the following lines:

  1. Expectation（description 🙂Returns theXCTestExpectationObject stored inpromise. thedescriptionParameters describe what you expect to happen.
  2. Promise. Fulfill (): This function is called in the success condition closure of the completion handler of an asynchronous method to mark that the expectation has been met.
  3. Wait (for: timeout 🙂: Keep tests running until all expectations are met ortimeoutThe interval ends (whichever comes first).

Run the tests. If you are connected to the Internet, the test should take about a second to succeed after loading the application into the emulator.

You can improve this approach and make tests fail faster by changing the following assumptions: do not wait for the request to succeed, but wait for the completion handler of the asynchronous method to be invoked. This happens as soon as the application receives a response from the server that meets the expectations (normal or wrong). Your tests can then check whether the request was successful.

To see how this works, create a new test.

But first, fix the previous test URL by undoing the changes you made. Then, add the following tests to your class:

// Asynchronous test: faster fail
func testCallToiTunesCompletes(a) {
  // given
  let url = URL(string: "https://itunes.apple.com/search?media=music&entity=song&term=abba")
  / / 1
  let promise = expectation(description: "Completion handler invoked")
  var statusCode: Int?
  var responseError: Error?
  
  // when
  let dataTask = sut.dataTask(with: url!) { data, response, error in
    statusCode = (response as? HTTPURLResponse)? .statusCode responseError = error/ / 2
    promise.fulfill()
  }
  dataTask.resume()
  / / 3
  wait(for: [promise], timeout: 5)
  
  // then
  XCTAssertNil(responseError)
  XCTAssertEqual(statusCode, 200)}Copy the code

The key difference is that the expectation can be satisfied by typing the completion handler, which can be done in just a second. If the request fails, then the assertion fails.

Run the tests. Now it takes about a second to fail. It failed because the request failed, not because the test run exceeded timeout.

Fix the URL, and then run the test again to confirm that it can now proceed successfully.

Fake objects and interactions

Asynchronous testing gives you confidence that the code will generate the correct input for the asynchronous API. You may also want to test whether the code works properly with URLSession when it receives input from the user, or if it correctly updates the user’s default database or iCloud container.

Most applications interact with system or library objects (objects that you have no control over), and tests that interact with these objects can be slow and non-repeatable, which violates two of the principles of FIRST. Instead, you can fake the interaction by getting input from the stub or by updating the mock object.

Forge when your code has a dependency on a system or library object. You can play that role by creating a dummy object, do this and inject the dummy into your code. Jon Reid’s dependency injection describes several approaches.

Dummy input from the stub

In this test, you analyze whether the session downloaded data searchresults.count is correct by checking that the application updateSearchResults(_:) is correct. The SUT is the view controller, and you’ll fake the session using stubs and some pre-downloaded data.

Go to the Test navigator and add a new Unit Test target. Name it HalfTunesFakeTests. Open halftunesfaketests. swift and import the HalfTunes application module below the following import statement:

@testable import HalfTunes
Copy the code

Now replace the content of the HalfTunesFakeTests class with the following:

var sut: SearchViewController!

override func setUp(a) {
  super.setUp()
  sut = UIStoryboard(name: "Main", bundle: nil)
    .instantiateInitialViewController() as? SearchViewController
}

override func tearDown(a) {
  sut = nil
  super.tearDown()
}
Copy the code

This will be declared as SUT, and SearchViewController creates it in and releases it tearDown() in setUp() :

Note: the SUT is view controller, because HalfTunes has a great view controller problem – all work in SearchViewController. Swift. Moving the network code into a separate module will reduce this problem and make testing easier.

Next, you’ll need some sample JSON data that your bogus session feeds into the test. Just do the following, so to limit the results you download in iTunes, append &limit=3 to the URL string:

https://itunes.apple.com/search?media=music&entity=song&term=abba& limit = 3
Copy the code

Copy this URL and paste it into your browser. This will download a file named 1.txt, 1.txt.js or similar. Preview it to confirm that it is a JSON file, and rename it to abbadata.json.

Now, go back to Xcode and go to the project navigator. Add the file to the HalfTunesFakeTests group.

The HalfTunes project contains the support file dhurlsessionmock. swift. This defines a simple protocol called DHURLSession with methods (stubs) to create a data task URLRequest using a URL or A. It also defines URLSessionMock, which complies with this protocol using an initializer that allows your URLSession to select data, responses, and errors to create mock objects.

To setUp the forgery, go to halftunesfaketests.swift and add the following to the setUp() SUT statement:

let testBundle = Bundle(for: type(of: self))
let path = testBundle.path(forResource: "abbaData", ofType: "json")
let data = try? Data(contentsOf: URL(fileURLWithPath: path!) , options: .alwaysMapped)let url = URL(string: "https://itunes.apple.com/search?media=music&entity=song&term=abba")
let urlResponse = HTTPURLResponse(url: url! , statusCode:200, httpVersion: nil, headerFields: nil)

let sessionMock = URLSessionMock(data: data, response: urlResponse, error: nil)
sut.defaultSession = sessionMock
Copy the code

This sets up dummy data and responses and creates dummy session objects. Finally, finally, it injects the forged session as a property into the application SUT.

Now you can write tests to check whether the call to updateSearchResults(_:) parses the pseudo-data. Add the following tests:

func test_UpdateSearchResults_ParsesData(a) {
  // given
  let promise = expectation(description: "Status code: 200")
  
  // when
  XCTAssertEqual(sut.searchResults.count.0."searchResults should be empty before the data task runs")
  let url = URL(string: "https://itunes.apple.com/search?media=music&entity=song&term=abba")
  let dataTask = sut.defaultSession.dataTask(with: url!) {
    data, response, error in
    // if HTTP request is successful, call updateSearchResults(_:) which parses the response data into Tracks
    if let error = error {
      print(error.localizedDescription)
    } else if let httpResponse = response as? HTTPURLResponse,
      httpResponse.statusCode == 200 {
      self.sut.updateSearchResults(data)
    }
    promise.fulfill()
  }
  dataTask.resume()
  wait(for: [promise], timeout: 5)
  
  // then
  XCTAssertEqual(sut.searchResults.count.3."Didn't parse 3 items from fake response")}Copy the code

You still have to write it as an asynchronous test because the stub pretends to be an asynchronous method.

In this case, the searchResults is empty before the data task executes. This should be correct because you created a brand new SUT setUp() in.

The fake data contains three JSON Track objects, so it is then asserted that the view controller’s searchResults array contains three items.

Run the tests. It should be a quick success since there is no real Internet connection!

Fake updates to mock objects

Previous tests used stubs to provide input from dummy objects. Next, you’ll use mock objects to test that your code updates UserDefaults correctly.

Reopen the BullsEye project. The application has two play styles: the user either moves the slider to match the target value, or guesses the target value from the slider position. The segmented control in the lower right corner toggles the game style and saves it in the user default Settings.

Your next test will check that the application has saved the gameStyle properties correctly.

In the Tests navigator, click New Unit Test Class and name it BullsEyeMockTests. Add the following under the import statement:

@testable import BullsEye
class MockUserDefaults: UserDefaults {
  var gameStyleChanged = 0
  override func set(_ value: Int, forKey defaultName: String) {
    if defaultName == "gameStyle" {
      gameStyleChanged += 1}}}Copy the code

MockUserDefaults overrides set(_:forKey:) to add the gameStyleChanged flag. Typically, you’ll see similar tests set a Bool variable, but adding an Int gives you more flexibility — for example, your test can check that the method is called only once.

Declare SUT and mock object BullsEyeMockTests in:

var sut: ViewController!
var mockUserDefaults: MockUserDefaults!
Copy the code

Next, replace the default setUp() and tearDown() with this:

override func setUp(a) {
  super.setUp()
  
  sut = UIStoryboard(name: "Main", bundle: nil).instantiateInitialViewController() as? ViewController
  mockUserDefaults = MockUserDefaults(suiteName: "testing")
  sut.defaults = mockUserDefaults
}

override func tearDown(a) {
  sut = nil
  mockUserDefaults = nil
  super.tearDown()
}
Copy the code

This creates the SUT and mock objects, and the mock objects are injected as properties of the SUT.

Now replace the two default test methods in the template with:

func testGameStyleCanBeChanged(a) {
  // given
  let segmentedControl = UISegmentedControl(a)// when
  XCTAssertEqual(mockUserDefaults.gameStyleChanged, 0."gameStyleChanged should be 0 before sendActions")
  segmentedControl.addTarget(sut,
                             action: #selector(ViewController.chooseGameStyle(_:)), for: .valueChanged)
  segmentedControl.sendActions(for: .valueChanged)
  
  // then
  XCTAssertEqual(mockUserDefaults.gameStyleChanged, 1."gameStyle user default wasn't changed")}Copy the code

Before the test method changes the staging control when the assertion is that the gameStyleChanged flag is 0. Therefore, if the THEN assertion is also true, it means that set(_:forKey:) was called only once.

Run tests; It should succeed.

UI testing in Xcode

UI tests enable you to test interaction with the user interface. UI testing works by finding the application’s UI objects through queries, synthesizing events, and then sending events to those objects. The API lets you examine the properties and state of UI objects so that you can compare them to the expected state.

In the Test navigator of the BullsEye project, add a new UI Test Target. Check that the target to be tested is BullsEye, and then accept the default name BullsEyeUITests.

Open BullsEyeUITests. Swift and add this attribute to the top of the BullsEyeUITests class:

varThe app:XCUIApplication!Copy the code

In setUp(), replace the statement XCUIApplication().launch() with the following:

app = XCUIApplication()
app.launch()
Copy the code

Change the name testExample() to testGameStyleSwitch().

Open a new line in it, testGameStyleSwitch() and click the red “* Record” * button at the bottom of the editor window:

This opens the application in the emulator with your interaction recorded as a test command. Once the app is loaded, click the “* Slide” * section of the game style switch and the top TAB. Then, click the Xcode Record button to stop recording.

Now you have testGameStyleSwitch() in the following three lines:

let app = XCUIApplication()
app.buttons["Slide"].tap()
app.staticTexts["Get as close as you can to: "].tap()
Copy the code

The logger has created code to test the same action that you tested in the application. Send the faucet to the slider and label. You will create your own UI tests based on these. If you see any other statements, simply delete them.

The first line repeats setUp() with the property you created in, so delete that line. You don’t need to click anything, so please also delete.tap() at the end of lines 2 and 3. Now, open up next to the small menu, / “Slide”. Then select segmentedControls buttons (” Slide “).

You should be left with the following:

app.segmentedControls.buttons["Slide"]
app.staticTexts["Get as close as you can to: "]
Copy the code

Click on any other object and let the logger help you find the code you can access in the test. Now replace these lines with the following code to create the given section:

// given
let slideButton = app.segmentedControls.buttons["Slide"]
let typeButton = app.segmentedControls.buttons["Type"]
let slideLabel = app.staticTexts["Get as close as you can to: "]
let typeLabel = app.staticTexts["Guess where the slider is: "]
Copy the code

Now that you have the names of the two buttons in the segmented control and the two possible top labels, add the following code below:

// then
if slideButton.isSelected {
  XCTAssertTrue(slideLabel.exists)
  XCTAssertFalse(typeLabel.exists)
  
  typeButton.tap()
  XCTAssertTrue(typeLabel.exists)
  XCTAssertFalse(slideLabel.exists)
} else if typeButton.isSelected {
  XCTAssertTrue(typeLabel.exists)
  XCTAssertFalse(slideLabel.exists)
  
  slideButton.tap()
  XCTAssertTrue(slideLabel.exists)
  XCTAssertFalse(typeLabel.exists)
}
Copy the code

This checks that you tap() has the correct label on each button in the segmented control. Run the tests – all assertions should succeed.

The performance test

According to Apple’s documentation: Performance tests take the block of code you want to evaluate and run it ten times to collect the average execution time and standard deviation of the run. The average of these individual measurements forms the value of the test run, which can then be compared to the benchmark to assess success or failure.

Writing performance tests is simple: you just put the code you’re going to measure into a closure called Measure ().

To see this in action, reopen the HalfTunes project and add the following tests to HalfTunesFaketests.swift:

// Performance
func test_StartDownload_Performance(a) {
  let track = Track(name: "Waterloo", artist: "ABBA",
                    previewUrl: "http://a821.phobos.apple.com/us/r30/Music/d7/ba/ce/mzm.vsyjlsff.aac.p.m4a")
  measure {
    self.sut.startDownload(track)
  }
}
Copy the code

Run the test, and then click the icon next to the start of the measure() trailing closure to see the statistics.

Click Set Baseline to set the reference time. Then, run the performance test again and see the results – it could be better or worse than the benchmark. Use the * Edit * button to reset the baseline to this new result.

Benchmarks are stored by device configuration, so you can perform the same tests on multiple different devices and keep each benchmark different based on processor speed, memory, and so on for a particular configuration.

Every time you make changes to your application that could affect the performance of the methods under test, run the performance tests again to see how they compare to the benchmark.

Code coverage

The code coverage tool tells you what application code the test is actually running, so you know which parts of the application code are not (yet) tested.

To enable code coverage, edit the “* Test” * action for your schema, and then select the “* Collect Coverage” * check box under the “Options” TAB:

Run all tests (command-U), then open the Report navigator (command-9). Select Coverage under the item at the top of the list:

Click show triangles to view SearchViewController. Swift, functions and closures in the list:

Scroll down to updateSearchResults(_:) to see coverage at 87.9%.

Click the arrow button for this feature to open the source file for that feature. When you hover over the Coverage comment in the right-hand column, the code section is highlighted in green or red:

The override comment shows how many times the test hit each code snippet; Parts that are not called are highlighted in red. As you might expect, the for loop runs three times, but nothing in the error path executes.

To increase the coverage of this feature, copy abbadata.json and then edit it so that it does not cause different errors. For example, change “results” to “result” a test hit print(“Results key not found in dictionary”).

100% coverage?

Should you strive for 100% code coverage? Google’s “100% unit test coverage,” and you’ll find a range of arguments for and against it, as well as arguments over the definition of “100% coverage.” Opponents of it say the last 10-15% is not worth the effort. The argument about it is that the last 10-15% is the most important because it is hard to test. Google “makes it hard to unit test bad design” to find a convincing argument that untestable code is a sign of deeper design problems.