使用 Python 进行测试（8）纯净测试

原文：Testing with Python (part 8): purity test

总结

如果你要使用综合测试（integrated tests）：

def test_add_new_item_to_cart(product, cart):new_product = Product.objects.create(name='New Product', price=15.00)new_cart_item = add_item_to_cart(cart, new_product, 1)assert new_cart_item.cart == cartassert new_cart_item.product == new_productassert new_cart_item.quantity == 1

或者你要隔离地测试，通过mock摆脱组合爆炸：

def test_add_to_cart_endpoint():product_mock = mock()cart_mock = mock()cart_item_mock = mock(quantity=1)user_mock = mock(cart=cart_mock)request_mock = mock(user=user_mock)(when(get_object_or_404).called_with(Product, id=1).thenReturn(product_mock))(when(add_item_to_cart).called_with(cart_mock, product_mock, 1)                    .thenReturn(cart_item_mock))response = add_to_cart(request_mock, CartSchema(id=1, quantity=1))assert response == {"id": 1, "quantity": 1}

两者都可以。

第一个选项容易编写和阅读，但时间长了会有很多累赘。

第二个可以构建一个假设所有依赖都已测试且正常运作的信任金字塔。如果你从测试基础开始，通过归纳可知，每个测试都是洁净的、独立的、快速的。但是，你真的确定你在测试这个功能，而不是被你的模拟数据一直返回“是”所欺骗了吗？毕竟，这些测试更难读写。

蓝队还是红队？

红蓝对决

IT领域也有不同的流派，通常有红队和蓝队，例如Mac VS Windows，Python VS Ruby, Vim VS Emacs，…
每个层次都有对立，制表符 VS 空格， 方法 VS函数，是否应该使用 early return,…
对于测试来说， 隔离测试 还是 综合测试（interated test）？

措辞

网上有时将“integration tests”和“integrated tests”混用，这会让人困惑。为了避免歧义，本文约定：

• Integrated test（综合测试）：对非隔离代码的测试
• Integration test（集成测试）：测试部件是否配合，Integration test 是 Integrated test，但反之不成立。
• e2e test（端到端）
• 持续集成：提交代码后， 打包并安装整个系统并进行测试。

什么隔离测试？隔离测试指测试代码的某一部分时，通过mock将其与依赖项隔离。

API 端点的测试

假设我们有一个使用友好的 django-ninja 库的 HTTP API 端点，用于在购物网站上将产品添加到购物车：

# WARNING: Since the project doesn't actually exist, I write 
# this code and the associated tests without running it, so read 
# it to get the general idea, don't take it as exactly what 
# you should writeimport loggingfrom django.shortcuts import get_object_or_404
from django.db import transaction
from .models import CartSchema, CartItem, Product
from .endpoints import apilog = logging.getLogger(__name__)@api.post("/cart")
def add_to_cart(request, payload: CartSchema):product = get_object_or_404(Product, id=payload.product_id)# Add an item to the cart if it doesn't exist, or update # the quantity if it does. We do that with locked rows in # a transaction to deal with the potential race condition.with transaction.atomic():locked_qs = CartItem.objects.select_for_update()cart_item = locked_qs.filter(cart=request.user.cart, product=product).first()if cart_item:cart_item.quantity += payload.quantitycart_item.save()else:cart_item = CartItem.objects.create(cart=request.user.cart,product=product,quantity=payload.quantity)log.info('Product %s added to user %s cart',product.id, request.user.id)return {"id": product_id, "quantity": cart_item.quantity}

将使用 Django 测试客户端进行综合测试。由于它将访问测试数据库，因此我们需要fixture：

import pytest
from django.contrib.auth.models import User
from ninja.testing import TestClient
from .models import Product, CartItem, Cart
from .endpoints import [e-mail]
def user(db): # db fixture assumed from pytest-djangouser = User.objects.create_user(username='testuser', password='testpass')Cart.objects.create(user=user)return [e-mail]
def client(user):return TestClient(api)@pytest.fixture
def auth_client(client, user):client.authenticate(user)return [e-mail]
def product(db):return Product.objects.create(name='Test Product', price=10.0)@pytest.fixture
def cart_item(db, user, product):return CartItem.objects.create(cart=user.cart, product=product, quantity=1)

然后我们需要测试：

• 认证
• 不存在的产品 ID 返回 404。
• 创建了一个不存在的商品购物车。
• 任何现有的商品购物车数量都会增加。
• API 返回格式符合我们的期望。

API端点的集成测试

def test_user_not_authenticated(product, client):payload = {'product_id': product.id, 'quantity': 1}response = client.post('/cart', json=payload)assert response.status_code == 401def test_non_existing_product_raises_404(auth_client):payload = {'product_id': 9999, 'quantity': 1}response = auth_client.post('/cart', json=payload)assert response.status_code == 404def test_cart_item_quantity_incremented(auth_client, product, cart_item):payload = {'product_id': product.id, 'quantity': 1}response = auth_client.post('/cart', json=payload)assert response.status_code == 200cart_item.refresh_from_db()assert cart_item.quantity == 2def test_cart_item_created_if_not_exist(auth_client, product):payload = {'product_id': product.id, 'quantity': 1}response = auth_client.post('/cart', json=payload)cart_item = CartItem.objects.get(cart=user.cart, product=product)assert response.status_code == 200assert cart_item.quantity == 1assert response.json() == {"id": product.id, "quantity": 1}# You may want to write more tests, like if your schema rejects 
# properly badly formatted requests, or to check what happens with 
# congestions on locked rows but this is enough to make our point.

它们很好，可以测试功能，并且还能测试函数之间的相互配合。
另一方面：

• 由于它需要处理测试数据库，所以速度较慢。
• 每个测试都依赖于整体。如果一个部分出问题，整个测试套件就会崩溃。
• 改变代码意味着改变所有这些测试，系统的所有部分彼此依赖。
• 测试的内容并不是一眼就能看出来的，因为存在大量干扰。

我喜欢这些测试，因为它们便宜且相对容易实施。测试通常是一次性的，特别是在项目早期阶段，代码库经常变动的时候。但如果我们不想使用一次性测试呢？如果我们想要一个强大、快速且能够证明整个系统现在以及未来几年都是健全的测试套件怎么办？孤立测试可以帮助解决这个问题。

同样的端点，但是隔离测试

隔离测试推动设计，因为要创建良好的隔离，你需要清晰、明确的部分，实际上你可以隔离这些部分。在我们的例子中，如果我们稍作停顿，我们可以看到购物车项的创建可以移到一个单独的函数中：

def add_item_to_cart(cart, product, quantity):"""Create a cart item, or update the quantity an existing one.We do that with locked rows in a transaction to deal with the potential race condition."""with transaction.atomic():locked_qs = CartItem.objects.select_for_update()cart_item = locked_qs.filter(cart=cart, product=product).first()if cart_item:cart_item.quantity += quantitycart_item.save()return cart_itemreturn CartItem.objects.create(cart=request.user.cart,product=product,quantity=quantity)@api.post("/cart")
def add_to_cart(request, payload: CartSchema):product = get_object_or_404(Product, id=product_id)cart_item = add_item_to_cart(request.user.cart, product, quantity)log.info('Product %s added to user %s cart', product_id, request.user.id)return {"id": product_id, "quantity": cart_item.quantity}

add_item_to_cart 成为我们可以模拟的对象，并且隔离数据库的副作用。它还具有增强关注点分离的良好特性，代码的每个部分都更加清晰。

我们可以联合测试改写后的add_item_to_cart，这也变得更加简洁、清晰了：

@pytest.fixture
def cart(db):user = User.objects.create_user(username='testuser', password='testpass')return Cart.objects.create(user=user)@pytest.fixture
def product(db):return Product.objects.create(name='Test Product', price=10.00)def test_add_existing_item_to_cart(cart, product, cart_item):cart_item = CartItem.objects.create(cart=cart, product=product, quantity=1)updated_cart_item = add_item_to_cart(cart, product, 2)cart_item.refresh_from_db()assert cart_item.quantity == 3assert updated_cart_item == cart_itemdef test_add_new_item_to_cart(product, cart):new_product = Product.objects.create(name='New Product', price=15.00)new_cart_item = add_item_to_cart(cart, new_product, 1)assert new_cart_item.cart == cartassert new_cart_item.product == new_productassert new_cart_item.quantity == 1

然后我们转向测试端点。这就是有趣的地方。因为我们在隔离环境中进行测试，所以我们不需要测试所有可能性的笛卡尔积，我们只需要检查它是否以正确的参数调用了正确的内容。

import pytest
from mockito import when, mock
from django.shortcuts import get_object_or_404
from .views import add_to_cart
from .models import Product, CartItem, User
from .models import CartSchema, CartItem, Productdef test_add_to_cart_endpoint():product_mock = mock()cart_mock = mock()cart_item_mock = mock(quantity=1)user_mock = mock(cart=cart_mock)request_mock = mock(user=user_mock)(when(get_object_or_404).called_with(Product, id=1).thenReturn(product_mock))(when(add_item_to_cart).called_with(cart_mock, product_mock, 1)                    .thenReturn(cart_item_mock))response = add_to_cart(request_mock, CartSchema(id=1, quantity=1))assert response == {"id": 1, "quantity": 1}

确实，我们不需要测试如果产品不存在，端点是否返回 404 状态码。 get_object_or_404 已经由 Django 测试套件测试过，所以我们可以假设它能正常工作。我们只需要检查我们如何使用它。同样，既然我们已经测试过add_item_to_cart ，我们就不需要检查项目是否已经正确添加。我们可以假设它有效，只需检查我们是否使用它即可。这意味着测试端点是一个单一的测试。它也独立于购物车管理、路由、身份验证或请求/响应序列化的细节。

由于 django-ninja 允许您转储 API 的模式，我们还可以通过一次性检查整个 API 签名来提高我们的弹性

import json
import pytest
from django.urls import reverse
from ninja.testing import TestClient
from .endpoints import api
from pathlib import Pathclient = TestClient(api)
schema_file = Path('api_schema.json')# This is naive and will fail even if you just add an enpoint, 
# but you get the idea
def test_api_schema(get_api_schema):if not schema_file.exists():# This command doesn't really exist, but you can code it.pytest.fail(f"API schema missing. Run ./manage.py dump_schema.")current_schema = client.get(reverse("api:schema")).json()saved_schema = json.loads(schema_file.read_text())if current_schema != saved_schema:pytest.fail("API schema has changed. Follow stability policy.")

所以，红还是蓝?

光谱就是光谱，当然你可以把光标放在任何你喜欢的地方。我写这篇文章假装有两种做法，但我们都知道我们可以走得更极端，或者在中间，混合两者。我知道我知道，我用“这要看情况”来逃避做出明确结论。欢迎来到现实生活，事情很糟糕，绝对真理不存在。我在等待那一天，当我们发现物理定律在整个宇宙中并不是均匀的时候。那将会很有趣。
所以：

• 这是集成测试示例
• 这是隔离测试示例
  你更喜欢哪一个？

集成测试目前较短，但这种情况不会持续下去。一旦我们添加更多功能，可能性图将增加，测试数量要么激增，要么忽略边缘情况。

But they are much easier to read because they are a collection of basic concepts.
但它们更容易阅读，因为它们是基本概念的集合。

On the other hand, the isolated tests are more correct: they are actually testing the business logic directly, and not just what the API returns. They can also already warn us when we break our public API in any way, and forever. Since only a fraction touches the DB, they are going to be fast, meaning we have a quick feedback loop.
另一方面，孤立测试更加准确：它们实际上直接测试业务逻辑，而不仅仅是 API 返回的内容。它们还可以在我们以任何方式破坏公共 API 时发出警告，而且是永久的。由于只有一小部分接触数据库，它们将会很快，这意味着我们有一个快速的反馈循环。

But the mock magic makes you stop. Do you really know what happens in there? Is it saving your butt, or are you testing your mocks instead of your code base?
但是模拟魔术让你停下来。你真的知道里面发生了什么吗？它是在帮你摆脱困境，还是你在测试模拟而不是代码库？

Plus, isolated tests are harder to write, require more experience, and more abstract thinking. They are less code in the long run but will take longer to bootstrap. You can’t delegate them to ChatGPT (or an intern), it will get them completely wrong.
另外，隔离测试更难编写，需要更丰富的经验和更抽象的思维。从长远来看，它们包含的代码更少，但需要更长的启动时间。你不能将它们委托给 ChatGPT（或实习生），否则它会完全错误地执行。

Then again, by nature, you won’t get the combinatorial explosions of all the possible things to test mixing with each other. Your code base will get exponentially more robust and more tested. You won’t lie to yourself about what correctness you cover. As much.
再说一遍，从本质上讲，你不会得到所有可能要测试的事物的组合爆炸混合在一起。你的代码库将变得指数级更加健壮和经过更多测试。你不会对自己所覆盖的正确性撒谎。就这样。

This builds trust incrementally since each test can assume the underlying functions are tested, recursively until you reach the ones actually performing the side effects. You know the ground you cover, and you can rely on this coverage progression.
这样逐步建立信任，因为每个测试都可以假设底层函数已经被测试，递归地进行，直到达到实际执行副作用的函数。你知道你所覆盖的范围，可以依赖这种覆盖进展。

However, and I’m going to repeat myself, I rarely go full isolated testing.
不过，我要重复一遍，我很少进行完全隔离的测试。

Most projects are just not that critical. For the typical app, change rates are going to be so fast that having dirty but disposable tests is more important than proving the system is very reliable. Bugs in production will happen, and the boss is likely ok with it although he won’t admit it.
大多数项目并非如此关键。对于典型的应用程序来说，变化速度会非常快，因此拥有脏但是可丢弃的测试比证明系统非常可靠更重要。生产中的错误是不可避免的，老板可能会接受，尽管他不会承认。

Or rather, his bank account is.
或者说，他的银行账户是。

Because the hard truth is very reliable software is super expensive. It takes time, it takes skill, it takes care. Plus, the typical users are quite tolerant of bugs but want features, yet probably don’t want to pay much for them.
因为残酷的事实是非常可靠的软件非常昂贵。它需要时间，需要技术，需要关怀。此外，典型的用户对错误有相当高的容忍度，但他们希望有更多功能，然而可能不愿为此支付太多费用。

If you ever wondered why software is so buggy in general, and the advertising industry so powerful, there it is.
如果你曾经 wondered 为什么软件普遍如此 buggy，广告行业如此强大，这就是原因所在。

Since this is a zero-sum game, spending resources on one thing means you don’t spend it on another. Design is expensive. Ergonomics testing is expensive, marketing is expensive.
由于这是一个零和游戏，将资源花费在一件事情上意味着你不会在另一件事情上花费。设计是昂贵的。人体工程学测试是昂贵的，市场营销也是昂贵的。

If you ever wondered why a super robust FOSS app is very ugly or hard to use, there it is.
如果你

On top of that, unlike in many industries, the consequences of user-facing issues are not as high as in the typical nerd mind. Most companies pay little price for a security breach or messages delivered to the wrong user. But don’t tweet the wrong thing, you could lose your head!

If you ever wondered why a bug has not been addressed for years, but the icon pack has been updated 3 times to reflect the current trend, there it is.
如果你曾经想过为什么一个漏洞多年来都没有得到解决，但图标包已经更新了 3 次以反映当前的趋势，那就是原因。

Anyway, a few projects reach a huge user count and have very few bugs at the same time.
无论如何，一些项目能够同时达到巨大的用户数量并且很少出现 bug。