Introduction
GitHub Copilot can assist you in developing tests quickly and improving productivity. In this article, weโll demonstrate how you can use Copilot to write both unit and integration tests. While Copilot performs well when generating tests for basic functions, complex scenarios require more detailed prompts and strategies. This article will walk through practical examples of using Copilot to break down tasks and verify code correctness.
Prerequisites
Before getting started you must have the following:
- A GitHub Copilot subscription plan.
- Visual Studio, Visual Studio Code, or any JetBrains IDE.
- The GitHub Copilot extension installed in your IDE.
Writing unit tests with Copilot Chat
In this section, weโll explore how to use GitHub Copilot Chat to generate unit tests for a Python class. This example demonstrates how you can use Copilot to create unit tests for a class like BankAccount
. We will show you how to prompt Copilot to generate tests, execute them, and verify the results.
Example class: BankAccount
Letโs start with a class BankAccount
that contains methods for depositing, withdrawing, and getting the balance of an account. Create a new file bank_account.py
in a GitHub repository and add the following BankAccount
class in Python.
class BankAccount:
def __init__(self, initial_balance=0):
if initial_balance < 0:
raise ValueError("Initial balance cannot be negative.")
self.balance = initial_balance
def deposit(self, amount):
if amount <= 0:
raise ValueError("Deposit amount must be positive.")
self.balance += amount
def withdraw(self, amount):
if amount <= 0:
raise ValueError("Withdrawal amount must be positive.")
if amount > self.balance:
raise ValueError("Cannot withdraw more than the current balance.")
self.balance -= amount
def get_balance(self):
return self.balance
Generating unit tests with Copilot Chat
You can prompt Copilot with a request to write tests that cover a range of scenarios. Be specific with your test requirements to get the best results. The following is an example prompt you can use to generate unit tests for the BankAccount
class.
Prompt: Develop a comprehensive suite of unit tests for the BankAccount() function in Python. Write multiple test methods that cover a wide range of scenarios, including edge cases, exception handling, and data validation.
This prompt does a few important things:
- Scope: It explicitly asks for a comprehensive suite of unit tests, so that Copilot generates more than just basic tests, covering a variety of potential use cases.
- Scenarios: The prompt specifies that the tests should include:
- Edge cases: Tests for boundary conditions (e.g., negative values for deposits or withdrawals, withdrawing more than the available balance).
- Exception handling: Tests for errors that should be raised when certain invalid actions occur (e.g., withdrawing more than the available balance, depositing negative amounts).
- Data validation: Ensures that inputs are correctly validated and handled by the methods.
Copilot will give you a response like the following.
import unittest
from bank_account import BankAccount
class TestBankAccount(unittest.TestCase):
def setUp(self):
self.account = BankAccount()
def test_initial_balance(self):
self.assertEqual(self.account.get_balance(), 0)
def test_deposit_positive_amount(self):
self.account.deposit(100)
self.assertEqual(self.account.get_balance(), 100)
def test_withdraw_within_balance(self):
self.account.deposit(100)
self.account.withdraw(50)
self.assertEqual(self.account.get_balance(), 50)
def test_deposit_negative_amount_raises_error(self):
with self.assertRaises(ValueError):
self.account.deposit(-100)
def test_withdraw_negative_amount_raises_error(self):
with self.assertRaises(ValueError):
self.account.withdraw(-50)
def test_withdraw_more_than_balance_raises_error(self):
self.account.deposit(100)
with self.assertRaises(ValueError):
self.account.withdraw(200)
def test_initial_balance_negative_raises_error(self):
with self.assertRaises(ValueError):
BankAccount(-100)
if __name__ == '__main__':
unittest.main()
Running the tests cases
The test suite generated by Copilot includes a test case for each scenario specified in the prompt. Each function in the BankAccount
class is tested for a range of conditions, including valid operations like deposits and withdrawals, as well as edge cases such as negative amounts and attempts to withdraw more than the available balance.
Once Copilot has generated the test suite to your satisfaction, add the code to a new file test_bank_account.py
. You can ask it how to run the tests.
Prompt: "How do I run these unit tests in Python using the unittest framework?"
Copilot will give you the following bash command.
python -m unittest test_bank_account.py
After running the tests, you will see the output in your terminal or IDE. If all tests pass, you can be confident that your BankAccount
class is working as expected.
Slash command
Additionally, you can prompt Copilot to write a full suite of unit tests with the /tests
slash command. Ensure that you have the file open on the current tab of your IDE and Copilot will generate unit tests for that file. The tests that Copilot generates may not cover all scenarios, so you should always review the generated code and add any additional tests that may be necessary.
Tip
If you ask Copilot to write tests for a code file that is not already covered by unit tests, you can provide Copilot with useful context by opening one or more existing test files in adjacent tabs in your editor. Copilot will be able to see the testing framework you use and will be more likely to write a test that is consistent with your existing tests.
Copilot will generate a unit test suite such as the following.
import unittest
from bank_account import BankAccount
class TestBankAccount(unittest.TestCase):
def setUp(self):
self.account = BankAccount()
def test_initial_balance(self):
self.assertEqual(self.account.get_balance(), 0)
Writing integration tests with Copilot
Integration tests are essential for ensuring that the various components of your system work correctly when combined. In this section, weโll extend our BankAccount
class to include interactions with an external service NotificationSystem
and use mocks to test the systemโs behavior without needing real connections. The goal of the integration tests is to verify the interaction between the BankAccount
class and the NotificationSystem
services, ensuring that they work together correctly.
Example class: BankAccount
with notification services
Let's update the BankAccount
class to include interactions with an external service such as a NotificationSystem
that sends notifications to users. NotificationSystem
represents the integration that would need to be tested.
Update the BankAccount
class in the bank_account.py
file with the following code snippet.
class BankAccount:
def __init__(self, initial_balance=0, notification_system=None):
if initial_balance < 0:
raise ValueError("Initial balance cannot be negative.")
self.balance = initial_balance
self.notification_system = notification_system
def deposit(self, amount):
if amount <= 0:
raise ValueError("Deposit amount must be positive.")
self.balance += amount
if self.notification_system:
self.notification_system.notify(f"Deposited {amount}, new balance: {self.balance}")
def withdraw(self, amount):
if amount <= 0:
raise ValueError("Withdrawal amount must be positive.")
if amount > self.balance:
raise ValueError("Cannot withdraw more than the current balance.")
self.balance -= amount
if self.notification_system:
self.notification_system.notify(f"Withdrew {amount}, new balance: {self.balance}")
def get_balance(self):
return self.balance
Here we'll break down our request for Copilot to write integration tests for the BankAccount
class into smaller, more manageable pieces. This will help Copilot generate more accurate and relevant tests.
Prompt: "Write integration tests for the deposit
function in the BankAccount
class. Use mocks to simulate the NotificationSystem
and verify that it is called correctly after a deposit."
This prompt does a few important things:
- Scope: It specifies integration tests, focusing on the interaction between the
deposit
function and theNotificationSystem
, rather than just unit tests. - Mocks: It explicitly asks for the use of mocks to simulate the
NotificationSystem
, ensuring that the interaction with external systems is tested without relying on their actual implementation. - Verification: The prompt emphasizes verifying that the
NotificationSystem
is called correctly after a deposit, ensuring that the integration between the components works as expected. - Specificity: The prompt clearly states the method (
deposit
) and the class (BankAccount
) to be tested.
Tip
If Copilot is producing invalid tests, provide examples of inputs and outputs for the function you want to test. This will help Copilot evaluate the expected behavior of the function.
Copilot will generate a test suite like the following.
import unittest
from unittest.mock import Mock
from bank_account import BankAccount
class TestBankAccountIntegration(unittest.TestCase):
def setUp(self):
self.notification_system = Mock()
def test_deposit_with_notification(self):
account = BankAccount(initial_balance=100, notification_system=self.notification_system)
account.deposit(50)
self.assertEqual(account.get_balance(), 150)
self.notification_system.notify.assert_called_once_with("Deposited 50, new balance: 150")
if __name__ == '__main__':
unittest.main()
Add the generated code to a new file test_bank_account_integration.py
.
Improving on the test cases
The prompt above generated a single test case that verifies the NotificationSystem
is called when a valid deposit is made. However, it doesn't cover cases where an error is raised during the deposit. In those scenarios, the NotificationSystem
should not be called. We need to add a test case that handles invalid deposits and ensure the notification system is not triggered.
Prompt: "Add a test case for invalid deposit amounts to verify the function raises the correct exceptions and that the NotificationService
is not called."
Copilot will generate a test case such as the following.
def test_deposit_negative_amount_raises_error(self):
account = BankAccount(initial_balance=100, notification_system=self.notification_system)
with self.assertRaises(ValueError):
account.deposit(0)
self.notification_system.notify.assert_not_called()
Asking about areas of improvement
Now that we've written test cases to validate the integration functionality for deposits, this is a great opportunity to search for improvements within the test suite. While the current tests are functional, we can prompt Copilot to evalulate code coverage and suggest areas of improvement.
Prompt: "What additional tests should be included to ensure full coverage for the integration between the BankAccount
class and the NotificationSystem
?"
Prompting Copilot with this question can help you identify missing test cases that may have been overlooked. In this situation, while we tested valid and invalid deposits, we haven't yet covered the withdrawal functionality.
Copilot will generate an updated test suite such as the following.
Click to expand the full generated code example
import unittest
from unittest.mock import Mock
from bank_account import BankAccount
class TestBankAccountIntegration(unittest.TestCase):
def setUp(self):
self.notification_system = Mock()
def test_deposit_with_notification(self):
account = BankAccount(initial_balance=100, notification_system=self.notification_system)
account.deposit(50)
self.assertEqual(account.get_balance(), 150)
self.notification_system.notify.assert_called_once_with("Deposited 50, new balance: 150")
def test_deposit_negative_amount_raises_error(self):
account = BankAccount(initial_balance=100, notification_system=self.notification_system)
with self.assertRaises(ValueError):
account.deposit(-50)
self.notification_system.notify.assert_not_called()
def test_deposit_zero_amount_raises_error(self):
account = BankAccount(initial_balance=100, notification_system=self.notification_system)
with self.assertRaises(ValueError):
account.deposit(0)
self.notification_system.notify.assert_not_called()
def test_withdraw_with_notification(self):
account = BankAccount(initial_balance=100, notification_system=self.notification_system)
account.withdraw(30)
self.assertEqual(account.get_balance(), 70)
self.notification_system.notify.assert_called_once_with("Withdrew 30, new balance: 70")
def test_withdraw_exceeding_balance_raises_error(self):
account = BankAccount(initial_balance=100, notification_system=self.notification_system)
with self.assertRaises(ValueError):
account.withdraw(150)
self.notification_system.notify.assert_not_called()
def test_withdraw_negative_amount_raises_error(self):
account = BankAccount(initial_balance=100, notification_system=self.notification_system)
with self.assertRaises(ValueError):
account.withdraw(-30)
self.notification_system.notify.assert_not_called()
def test_withdraw_zero_amount_raises_error(self):
account = BankAccount(initial_balance=100, notification_system=self.notification_system)
with self.assertRaises(ValueError):
account.withdraw(0)
self.notification_system.notify.assert_not_called()
def test_initial_negative_balance_raises_error(self):
with self.assertRaises(ValueError):
BankAccount(initial_balance=-100, notification_system=self.notification_system)
if __name__ == '__main__':
unittest.main()
Once Copilot has generated the test suite to your satisfaction, run the tests with command below to verify the results.
python -m unittest test_bank_account_integration.py