导言
实际测试过程中,测试用例可能需要支持多种场景,可以把场景强相关的部分抽象为参数,通过对参数赋值来驱动用例的执行。参数化对的行为表现在不同的层级上:
- fixture的参数化
- 测试用例的参数化:使用@pytest.mark.parametrize可以在测试用例、测试类甚至是测试模块中标记多个参数或者fixture的组合
- 也可以通过pytest_generate_tests这个钩子方法自定义参数化的方案。源码:
#_pytest/python.py
def parametrize(self, argnames, argvalues, indirect=False, ids = None,scope = None)
参数分析如下:
argnames:一个逗号分隔的字符串或者一个元组、列表,表明指定对的参数名。对于argnames,还是有一些限制:
只能是被标记对象入参的子集
@pytest.mark.parametrize('input, expected', [(1,2)])
def test_sample(input):
assert input + 1 =1不能是被标记对象入参中,定义了默认值的参数:
@pytest.mark.parametrize('innput, expected' , [(1,2)])
def test_sample(input, expected = 2)
assert input + 1 == expected
虽然test_sample声明了expected参数,但是也赋予了一个默认值,如果我们在标记中强行声明,会得到错误:
In test_sample: function already takes an argument 'expected' with a default value
- 会覆盖同名的fixture
import pytest
@pytest.fixture()
def excepted():
return 1
@pytest.mark.parametrize('input, excepted', [(1,2)])
def test_sample(input , excepted):
assert input + 1 == excepted
测试结果:
D:\pytest\exercise\chapter11>pytest test_mark.py
================================================================= test session starts ==================================================================
platform win32 -- Python 3.8.5, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: D:\pytest\exercise\chapter11
plugins: allure-pytest-2.8.36
collected 1 item
test_mark.py . [100%]
================================================================== 1 passed in 0.04s ===================================================================
test_sample标记中对的excepted(2)覆盖了同名对的fixture excepted(1),所以测试用例通过。
- argvalues:一个可迭代对象,表明对argnames参数的赋值,具体有以下几种情况:
- 如果argnames包含多个参数,那么argvalues的迭代返回元素必须是可度量的(即支持len()方法),并且长度和argnames声明参数对的个数相等,所以它可以是元组、列表、集合等,表明所有入参的实参:
@pytest.mark.parametrize('input,excepted',[(1,2),[2,3]],set([3,4]))
def test_sample(input,excepted):
assert input + 1 == excepted
测试结果:
D:\pytest\exercise\chapter11>pytest test_mark.py
================================================================= test session starts ==================================================================
platform win32 -- Python 3.8.5, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: D:\pytest\exercise\chapter11
plugins: allure-pytest-2.8.36
collected 0 items / 1 error
======================================================================== ERRORS ========================================================================
____________________________________________________________ ERROR collecting test_mark.py _____________________________________________________________
In test_sample: expected Sequence or boolean for indirect, got set
=============================================================== short test summary info ================================================================
ERROR test_mark.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=================================================================== 1 error in 0.14s ===================================================================
注意:考虑到集合的去重性,我们并不建议使用它
- 如果argnames只包含一个参数,那么argvalues的迭代返回元素可以是具体的值:
@pytest.mark.parametrize('input',[1,2,3])
def test_sample(input):
assert input + 1
- 之前提到,argvalues是一个可迭代对象,我们可以实现更加复杂的场景。例如:从excel中读取实参
import pytest
def read_excel():
for dev in ['dev1','dev2','dev3']:
yield dev
@pytest.mark.parametrize('dev',read_excel())
def test_sample(dev):
assert dev
注意:实现这个场景有多种方法,可以在fixture这种加载excel中的数据,但是测试报告有区别。
- 我们使用pytest.param为argvalues参数赋值:
@pytest.mark.parametrize(
('n','excepted'),
[(2,1),
pytest.param(2,1,marks=pytest.mark.xfail(),id='XPASS')
])
def test_params(n, excepted):
assert 2 / n == excepted
测试结果:
D:\pytest\exercise\chapter11>pytest test_excel.py
================================================================= test session starts ==================================================================
platform win32 -- Python 3.8.5, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: D:\pytest\exercise\chapter11
plugins: allure-pytest-2.8.36
collected 2 items
test_excel.py .X [100%]
============================================================= 1 passed, 1 xpassed in 0.07s =============================================================
分析:无论argvalues中传递的是可度量的对象(列表、元组、集合)还是具体的值,在源码中我们都会将其封装成一个ParameterSet对象,是一个具名元组(nameedtuple),包含values、marks、id三个元素:
from _pytest.mark.structures import ParameterSet as PS, ParameterSet
PS._make([(1,2),[],None])
ParameterSet(values=(1,2),marks=[],id = None)
如果直接传递一个ParameterSet对象会发生什么呢?
# _pytest/mark/structures.py
class ParameterSet(namedtuple("ParameterSet", "values, marks, id")):
...
@classmethod
def extract_from(cls, parameterset, force_tuple=False):
"""
:param parameterset:
a legacy style parameterset that may or may not be a tuple,
and may or may not be wrapped into a mess of mark objects
:param force_tuple:
enforce tuple wrapping so single argument tuple values
don't get decomposed and break tests
"""
if isinstance(parameterset, cls):
return parameterset
if force_tuple:
return cls.param(parameterset)
else:
return cls(parameterset, marks=[], id=None)
可以看出,如果直接传递一个ParameterSet对象,那么返回的就是它本身(return parameterset)
import pytest
from _pytest.mark.structures import ParameterSet
@pytest.mark.parametrize(
'input,excepted',
[(1,2), ParameterSet(values=(1,2), marks=[], id=None)])
def test_sample(input,excepted):
assert input + 1 == excepted
pytest.param方法对的作用就是封装一个ParameterSet对象。
# _pytest/mark/__init__.py
def param(*values, **kw):
"""Specify a parameter in `pytest.mark.parametrize`_ calls or
:ref:`parametrized fixtures <fixture-parametrize-marks>`.
.. code-block:: python
@pytest.mark.parametrize("test_input,expected", [
("3+5", 8),
pytest.param("6*9", 42, marks=pytest.mark.xfail),
])
def test_eval(test_input, expected):
assert eval(test_input) == expected
:param values: variable args of the values of the parameter set, in order.
:keyword marks: a single mark or a list of marks to be applied to this parameter set.
:keyword str id: the id to attribute to this parameter set.
"""
return ParameterSet.param(*values, **kw)
- indirect:argnames的子集或者一个布尔值,将指定参数的实参通过request.param重定向到和参数同名的fixture中,以此满足更加复杂的场景。
import pytest
@pytest.fixture()
def max(request):
return request.param - 1
@pytest.fixture()
def min(request):
return request.param + 1
#默认indirect为False
@pytest.mark.parametrize('min,max',[(1,2),(3,4)])
def test_indirecct(min,max):
assert min <= max
# min max对应对的实参重定向到同名的fixture中
@pytest.mark.parametrize('min,max',[(1,2),(3,4)],indirect=True)
def test_indirect_indirect(min,max):
assert min >= max
# 只将max对应的实参重定向到fixture中
@pytest.mark.parametrize('min,max',[(1,2),(3,4)], indirect=['max'])
def test_indirecct_part_indirect(min,max):
assert min == max
测试结果:
D:\pytest\exercise\chapter11>pytest test_excel.py
================================================ test session starts =================================================
platform win32 -- Python 3.8.5, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: D:\pytest\exercise\chapter11
plugins: allure-pytest-2.8.36
collected 6 items
test_excel.py ...... [100%]
================================================= 6 passed in 0.05s ==================================================
ids:一个可执行对象,用于生成测试ID,或者一个列表/元组,知名所有新增测试用例的用例ID
如果使用列表、元组直接指明测试ID,那么长度要等于argvalues的长度
@pytest.mark.parametrize('input,excepted',[(1,2),(3,4)],
ids=['first','second'])
def test_ids_with_ids(input,excepted):
pass
collected 2 items
<Module test_ids.py>
<Function test_ids_with_ids[first]>
<Function test_ids_with_ids[second]>
如果指定了相同的测试ID,pytest会在后面自动添加索引:
@pytest.mark.parametrize('input,excepted',[(1,2),(3,4)],
ids=['num','num'])
def test_ids_with_ids(input,excepted):
pass
collected 2 items
<Module test_ids.py>
<Function test_ids_with_ids[num0]>
<Function test_ids_with_ids[num1]>
如果在指定的测试ID这种使用了非ASCII的值,默认显示的是字节序列:
@pytest.mark.parametrize('input, expected', [(1, 2), (3, 4)],
ids=['num', '中文'])
def test_ids_with_ids(input, expected):
pass
collected 2 items
<Module test_ids.py>
<Function test_ids_with_ids[num]>
<Function test_ids_with_ids[\u4e2d\u6587]>
可以看出,“中文”实际上显示的是**\u4e2d\u6587**
如果我们想要得到期望的显示,应该如何处理?参考源码:
# _pytest/python.py
def _ascii_escaped_by_config(val, config):
if config is None:
escape_option = False
else:
escape_option = config.getini(
"disable_test_id_escaping_and_forfeit_all_rights_to_community_support"
)
return val if escape_option else ascii_escaped(val)
我们可以通过在pytest.ini中使用disable_test_id_escaping_and_forfeit_all_rights_to_community_support选项来避免这种情况:
[pytest]
disable_test_id_escaping_and_forfeit_all_rights_to_community_support = True
结果如下:
<Module test_ids.py>
<Function test_ids_with_ids[num]>
<Function test_ids_with_ids[中文]>
如果通过一个可执行对象生成测试ID:
def idfn(val):
return val + 1
@pytest.mark.parametrize('input,exceptes',[(1,2),(3,4)],ids=idfn)
def test_ids_with_ids(input,excepted):
pass
结果如下:
collected 2 items
<Module test_ids.py>
<Function test_ids_with_ids[2-3]>
<Function test_ids_with_ids[4-5]>
通过例子可以看出,对于一个具体的argvalues参数(1,2),被拆分为1和2分别作为参数传给idfn函数,通过-符号链接在一起作为一个测试ID返回,而不是作为一个整体传入的。源码如下:
# _pytest/python.py
def _idvalset(idx, parameterset, argnames, idfn, ids, item, config):
if parameterset.id is not None:
return parameterset.id
if ids is None or (idx >= len(ids) or ids[idx] is None):
this_id = [
_idval(val, argname, idx, idfn, item=item, config=config)
for val, argname in zip(parameterset.values, argnames)
]
return "-".join(this_id)
else:
return _ascii_escaped_by_config(ids[idx], config)
函数首先通过zip(parameterset.values, argnames)将argnames和argvalues的值一一对应,在通过“-”.join(this_id)链接;
此外, 假设已经通过pytest.param指定了id属性,那么会覆盖ids中指定的测试ID…
@pytest.mark.parametrize(
'input,excepted',
[(1,2), pytest.param(3,4, id='id_via_pytest_param')],
ids=['first','second'])
def test_ids_with_ids(input,excepted):
pass
测试结果:
collected 2 items
<Module test_ids.py>
<Function test_ids_with_ids[first]>
<Function test_ids_with_ids[id_via_pytest_param]>
ids的用法主要作用就是更进一步细化测试用例,区分不同的测试场景,为有针对性的执行测试用例提供一种新的约束方式。
例如:对于以下测试用例,可以通过-k 'Window and not Non’选项,只执行和windows相关的场景。
import pytest
@pytest.mark.parametrize(
'inpout,excepted',[
pytest.param(1,2,id='Windows'),
pytest.param(3,4,id='Windows'),
pytest.param(5,6,id='Non-Windows'),
])
def test_ids_with_ids(input,excepted):
pass
- scope:声明argnames中参数的作用域,并通过对应的argvalues实例划分
次测试用例,控制测试用例的收集顺序。
例如:显示声明scope参数。将参数的作用域声明为模块级别:
# @Time : 2021/5/26 10:00
import pytest
@pytest.mark.parametrize('test_input,excepted',[(1,2),(3,4)],scope='module')
def test_scope1(test_input,excepted):
pass
@pytest.mark.parametrize('test_input,excepted',[(1,2),(3,4)],scope='module')
def test_scope2(test_input,excepted):
pass
收集到的测试用例如下:
collected 4 items
<Module test_scope.py>
<Function test_scope1[1-2]>
<Function test_scope2[1-2]>
<Function test_scope1[3-4]>
<Function test_scope2[3-4]>
以下是默认的收集顺序:
collected 4 items
<Module test_scope.py>
<Function test_scope1[1-2]>
<Function test_scope1[3-4]>
<Function test_scope2[1-2]>
<Function test_scope2[3-4]>
scope未指定的情况下(或者scope=None),当indirect等于True或者包含所有的argnames参数时,作用域为所有fixture作用域的最小范围;否则,其永远为function;
@pytest.fixture(scope='module')
def test_input(request):
pass
@pytest.fixture(scope='module')
def expected(request):
pass
@pytest.mark.parametrize('test_input, expected', [(1, 2), (3, 4)],
indirect=True)
def test_scope1(test_input, expected):
pass
@pytest.mark.parametrize('test_input, expected', [(1, 2), (3, 4)],
indirect=True)
def test_scope2(test_input, expected):
pass
test_input和expected的作用域都是module,所以参数的作用域也是module,用例的收集顺序和上一节相同:
collected 4 items
<Module test_scope.py>
<Function test_scope1[1-2]>
<Function test_scope2[1-2]>
<Function test_scope1[3-4]>
<Function test_scope2[3-4]>
1.1 empty_parameter_set_mark选项
默认情况下,如果@pytest.mark.parametrize的argnames中参数没有接收到任何的实参,用例的结果将会被置位为SKIPPED
例如:当python版本小于3.8时返回一个空的列表
# src/chapter-11/test_empty.py
import pytest
import sys
def read_value():
if sys.version_info >= (3, 8):
return [1, 2, 3]
else:
return []
@pytest.mark.parametrize('test_input', read_value())
def test_empty(test_input):
assert test_input
测试结果:
D:\pytest\exercise\chapter11>pytest -s test_empty.py
================================================ test session starts =================================================
platform win32 -- Python 3.8.5, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: D:\pytest\exercise\chapter11
plugins: allure-pytest-2.8.36
collected 3 items
test_empty.py ...
================================================= 3 passed in 0.04s ==================================================
我们可以在pytest.ini中设置empty_parameter_set_mark选项来改变这种行为,可能的值为:
- skip:默认值
- xfail:跳过执行直接将用例标记为XFAIL,等价于xfail(run=False)
- fail_at_collect:上报上一个CollectError异常
1.2 多个标记组合
如果一个用例标记了多个@pytest.mark.parametrie标记
import pytest
@pytest.mark.parametrize('test_input',[1,2,3])
@pytest.mark.parametrize('test_output,excepted',[(1, 2), (3, 4)])
def test_multi(test_input,test_output,excepted):
pass
实际收集到的用例:是它们所有可能的组合。
collected 6 items
<Module test_multi.py>
<Function test_multi[1-2-1]>
<Function test_multi[1-2-2]>
<Function test_multi[1-2-3]>
<Function test_multi[3-4-1]>
<Function test_multi[3-4-2]>
<Function test_multi[3-4-3]>
1.3 标记测试模块
可以通过对pytestmark赋值,参数化一个测试模块:
import pytest
pytestmark = pytest.mark.parametrize('test_input, expected', [(1, 2), (3, 4)])
def test_module(test_input, expected):
assert test_input + 1 == expected
2. 钩子方法
pytest_generate_tests方法在测试用例的收集过程中被调用,它接收一个metafunc对象,我们可以通过其访问测试请求的上下文,更重要的是,可以使用metafunc.parametrize方法自定义参数化的行为;
源码如下:
# _pytest/python.py
def pytest_generate_tests(metafunc):
# those alternative spellings are common - raise a specific error to alert
# the user
alt_spellings = ["parameterize", "parametrise", "parameterise"]
for mark_name in alt_spellings:
if metafunc.definition.get_closest_marker(mark_name):
msg = "{0} has '{1}' mark, spelling should be 'parametrize'"
fail(msg.format(metafunc.function.__name__, mark_name), pytrace=False)
for marker in metafunc.definition.iter_markers(name="parametrize"):
metafunc.parametrize(*marker.args, **marker.kwargs)
首先,它检查了parametrize的拼写错误,如果你不小心写成了[“parameterize”, “parametrise”, “parameterise”]中的一个,pytest会返回一个异常,并提示正确的单词;然后,循环遍历所有的parametrize的标记,并调用metafunc.parametrize方法;
现在,我们来定义一个自己的参数化方案:
在下面这个用例中,我们检查给定的stringinput是否只由字母组成,但是我们并没有为其打上parametrize标记,所以stringinput被认为是一个fixture
def test_valid_string(stringinput):
assert stringinput.isalpha()
现在,我们期望把stringinput当成一个普通的参数,并且从命令行赋值:
首先,我们定义一个命令行选项:
def pytest_addoption(parser):
parser.addoption(
"--stringinput",
action="append",
default=[],
help="list of stringinputs to pass to test functions",
)
然后,我们通过pytest_generate_tests方法,将stringinput的行为由fixtrue改成parametrize:
#
def pytest_generate_tests(metafunc):
if "stringinput" in metafunc.fixturenames:
metafunc.parametrize("stringinput", metafunc.config.getoption("stringinput"))
我们就可以通过–stringinput命令行选项来为stringinput参数赋值了:
λ pipenv run pytest -q --stringinput='hello' --stringinput='world' test_strings.py
.. [100%]
2 passed in 0.02s
如果我们不加–stringinput选项,相当于parametrize的argnames中的参数没有接收到任何的实参,那么测试用例的结果将会置为SKIPPED
λ pipenv run pytest -q src/chapter-11/test_strings.py
s [100%]
1 skipped in 0.02s
注意:不管是metafunc.parametrize方法还是@pytest.mark.parametrize标记,它们的参数(argnames)不能是重复的,否则会产生一个错误:ValueError: duplicate ‘stringinput’;