SpecFlow SpecRun.exe+MSBuild+Report
最近想要使用SpecFlow 使用 SpecRun.exe+MSBuild 去產生報表 並且使用排成器來產生測試報表
一開始要先安裝MSBuild
PS.可以參考我寫的安裝 MSBuild 2017 Tools Intall
<specFlow>
<!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config-->
<!--<unitTestProvider name="MsTest" />-->
<!--<add name="SpecRun" path="../packages/SpecRun.SpecFlow.2-3-0.1.7.1/lib/net45" />-->
<!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config-->
<!-- use unit test provider SpecRun+NUnit or SpecRun+MsTest for being able to execute the tests with SpecRun and another provider -->
<unitTestProvider name="SpecRun" />
<plugins>
<!--<add name="SpecRun" path="../packages/SpecRun.Runner.1.7.2/tools/"/>-->
<add name="SpecRun" path="../packages/SpecRun.SpecFlow.2-3-0.1.7.2/lib/net45/"/>
</plugins>
</specFlow>
接下來開始到web.config or app.config 去設定使用SpecRun執行
<?xml version="1.0" encoding="utf-8"?>
<TestProfile xmlns="http://www.specflow.org/schemas/plus/TestProfile/1.5">
<Settings projectName="SpecFlowTestProject.Test" projectId="{b2ada03f-013d-4569-a618-f4bfb5df6594}" outputFolder="/Report" />
<!--<Report disable="false" copyAlsoToBaseFolder="true">
<Template name="CustomReportTemplate_1.cshtml" outputName="Report1.html"/>
</Report>-->
<Execution stopAfterFailures="-1" testThreadCount="1" testSchedulingMode="Sequential" retryFor="None"/>
<!-- For collecting by a SpecRun server update and enable the following element. For using the
collected statistics, set testSchedulingMode="Adaptive" attribute on the <Execution> element.
<Server serverUrl="http://specrunserver:6365" publishResults="true" />
-->
<TestAssemblyPaths>
<TestAssemblyPath>SpecFlowTestProject.Test.dll</TestAssemblyPath>
</TestAssemblyPaths>
<DeploymentTransformation>
<Steps>
<!-- sample config transform to change the connection string-->
<!--<ConfigFileTransformation configFile="App.config">
<Transformation>
<![CDATA[<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
<connectionStrings>
<add name="MyDatabase" connectionString="Data Source=.;Initial Catalog=MyDatabaseForTesting;Integrated Security=True"
xdt:Locator="Match(name)" xdt:Transform="SetAttributes(connectionString)" />
</connectionStrings>
</configuration>
]]>
</Transformation>
</ConfigFileTransformation>-->
</Steps>
</DeploymentTransformation>
</TestProfile>
然後去設定Default.srprofile
參數說明(參照官方文件說明+SpecRunTestProfile.xsd)
| 參數名稱 |
說明
|
projectName
| 測試專案名稱 |
outputFolder
| 輸出結果的資料夾 |
stopAfterFailures
|
失敗測試的次數，之後停止執行所有測試。將此值設置為0（零）以執行所有測試，而不管故障的數量。如果未指定，則默認為10.
注意：.srprofile NuGet包附帶的默認值為3。
|
testThreadCount
|
用於執行測試的測試線程數（默認值= 1）。
雖然沒有最大線程數，但將此值設置得太高會降低性能，因為各個線程會爭用處理能力。建議將Process測試線程隔離（testThreadIsolation設置）的線程數設置為小於核心數的一個。您可以使用一般使用較高的數字隔離AppDomain和SharedAppDomains。
注意：僅增加線程數可能會導致問題，例如，如果線程訪問同一個數據庫。因此，使用{TestThreadId}新文件和數據庫實例名稱中的佔位符將其與.config文件的重定位和轉換相結合是有意義的（請參閱下面）。
注意：每個線程執行鉤子之前和之後（例如BeforeTestRun，AfterTestRun等）。
|
testSchedulingMode
|
確定SpecRun執行測試的順序
Sequential::測試始終以相同的順序執行（默認）。執行測試的順序是可預測的並且始終相同，但不能手動影響。
Random：測試順序是隨機的。這可用於檢測可能通過/失敗的閃爍場景，具體取決於執行其他測試區域的順序。
Adaptive：SpecRun使用通過/失敗歷史記錄中可用的統計信息來確定首先運行哪些測試。以前失敗的測試和新測試在成功和穩定的測試之前執行。
注意：如果設置testSchedulingMode為Adaptive如果沒有以前的測試執行結果可用（例如，沒有配置服務器，或服務器無法訪問），則使用該Random選項執行測試。
|
retryFor
|
確定應根據其狀態重試哪些測試。可用選項::
None不重試測試：重試
Failing失敗測試（默認）
All：重試所有測試，包括通過的測試
注意：在返回相應狀態之後和執行任何後續測試之前，將立即重試測試。
|
TestAssemblyPath
|
所述<TestAssemblyPaths>元件是一種用於容器<TestAssemblyPath>的元件，從所述命令行執行測試時，其定義組件的路徑。<TestAssemblyPath>元素中的路徑相對於基礎文件夾。
確保在自己的<TestAssemblyPath>元素中指定所有程序集文件的正確名稱，否則將不會找到任何測試！
|
也可以參考官方文件
PS.我在實際進行時發現設定檔跟官方文件有些落差，所以我建議參考官方文件然後到SpecRunTestProfile.xsd裡面去看可以設定資訊的位置
SpecRunTestProfile.xsd文件內容
<?xml version="1.0" encoding="utf-8"?>
<xs:schema id="SpecRunTestProfile"
targetNamespace="http://www.specflow.org/schemas/plus/TestProfile/1.5"
elementFormDefault="qualified"
xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="TestProfile" type="TestProfile" />
<xs:complexType name="TestProfile">
<xs:choice minOccurs="0" maxOccurs="unbounded">
<xs:element minOccurs="0" maxOccurs="1" name="Settings" type="TestProfileSettings" />
<xs:element minOccurs="0" maxOccurs="1" name="Server" type="ServerSettings" />
<xs:element minOccurs="0" maxOccurs="1" name="Execution" type="ExecutionSettings" />
<xs:element minOccurs="0" maxOccurs="1" name="Environment" type="EnvironmentSettings" />
<xs:element minOccurs="0" maxOccurs="1" name="TestAssemblyPaths" type="TestAssemblyPaths" />
<xs:element minOccurs="0" maxOccurs="1" name="Report" type="ReportSettings" />
<xs:element minOccurs="0" maxOccurs="1" name="Filter" type="xs:string" />
<xs:element minOccurs="0" maxOccurs="1" name="Targets" type="Targets" />
<xs:element minOccurs="0" maxOccurs="1" name="DeploymentTransformation" type="DeploymentTransformation" />
<xs:element minOccurs="0" maxOccurs="1" name="TestThreads" type="TestThreads" />
<xs:element minOccurs="0" maxOccurs="1" name="VSTest" type="VSTest" />
</xs:choice>
</xs:complexType>
<xs:complexType name="TestProfileSettings">
<xs:attribute name="name" type="xs:string" use="optional" />
<xs:attribute name="projectName" type="xs:string" use="required" />
<xs:attribute name="licenseKey" type="xs:string" use="optional" />
<xs:attribute name="baseFolder" type="xs:string" use="optional" />
<xs:attribute name="outputFolder" type="xs:string" use="optional" />
<xs:attribute name="reportTemplate" type="xs:string" use="optional" />
<xs:attribute name="projectId" type="guid" use="optional" />
</xs:complexType>
<xs:complexType name="ServerSettings">
<xs:attribute name="serverUrl" type="xs:string" use="optional" />
<xs:attribute name="publishResults" type="xs:boolean" use="optional" default="false" />
</xs:complexType>
<xs:complexType name="ExecutionSettings">
<xs:attribute name="stopAfterFailures" type="xs:int" use="optional" default="10" />
<xs:attribute name="testSchedulingMode" type="TestSchedulingMode" use="optional" default="Sequential" />
<xs:attribute name="testThreadCount" type="TestThreadCount" use="optional" default="1" />
<xs:attribute name="retryFor" type="RetryMode" use="optional" default="Failing" />
<xs:attribute name="retryCount" type="xs:int" use="optional" default="2" />
<xs:attribute name="apartmentState" type="ApartmentState" use="optional" default="Unknown" />
<xs:attribute name="isDryRun" type="xs:boolean" use="optional" default="false" />
</xs:complexType>
<xs:complexType name="EnvironmentSettings">
<xs:attribute name="framework" type="FrameworkVersion" use="optional" default="Default" />
<xs:attribute name="platform" type="Platform" use="optional" default="Default" />
<xs:attribute name="testThreadIsolation" type="TestThreadIsolation" use="optional" default="AppDomain" />
<xs:attribute name="apartmentState" type="ApartmentState" use="optional" default="Unknown" />
</xs:complexType>
<xs:complexType name="TestAssemblyPaths">
<xs:sequence>
<xs:element minOccurs="0" maxOccurs="unbounded" name="TestAssemblyPath" type="xs:string" />
</xs:sequence>
</xs:complexType>
<!-- Report settings -->
<xs:complexType name="Template">
<xs:attribute name="name" type="xs:string" use="required" />
<xs:attribute name="outputName" type="xs:string" use="optional" />
<xs:attribute name="outputCulture" type="xs:string" use="optional" />
</xs:complexType>
<xs:complexType name="ReportSettings">
<xs:sequence>
<xs:element minOccurs="0" maxOccurs="unbounded" name="Template" type="Template" />
</xs:sequence>
<xs:attribute name="copyAlsoToBaseFolder" type="xs:boolean" use="optional" default="false" />
<xs:attribute name="disable" type="xs:boolean" use="optional" default="false" />
<xs:attribute name="existingFileHandlingStrategy" type="ExistingFileHandlingStrategy" use="optional" default="Default" />
</xs:complexType>
<!-- test targets -->
<xs:complexType name="Targets">
<xs:sequence>
<xs:element minOccurs="0" maxOccurs="unbounded" name="Target" type="Target" />
</xs:sequence>
</xs:complexType>
<xs:complexType name="Target">
<xs:choice minOccurs="0" maxOccurs="unbounded">
<xs:element minOccurs="0" maxOccurs="1" name="Filter" type="xs:string" />
<xs:element minOccurs="0" maxOccurs="1" name="Environment" type="EnvironmentSettings" />
<xs:element minOccurs="0" maxOccurs="1" name="DeploymentTransformationSteps" type="DeploymentTransformationSteps" />
</xs:choice>
<xs:attribute name="name" type="xs:string" use="required" />
</xs:complexType>
<!-- test threads -->
<xs:complexType name="TestThreads">
<xs:sequence>
<xs:element minOccurs="0" maxOccurs="unbounded" name="TestThread" type="TestThread" />
</xs:sequence>
</xs:complexType>
<xs:complexType name="TestThread">
<xs:sequence>
<xs:element minOccurs="0" maxOccurs="1" name="TestAffinity" type="xs:string" />
</xs:sequence>
<xs:attribute name="id" type="xs:int" use="required" />
</xs:complexType>
<!-- deployment transformations -->
<xs:complexType name="DeploymentTransformation">
<xs:sequence>
<xs:element minOccurs="0" maxOccurs="1" name="GlobalSteps" type="DeploymentTransformationSteps" />
<xs:element minOccurs="0" maxOccurs="1" name="Steps" type="DeploymentTransformationSteps" />
</xs:sequence>
</xs:complexType>
<xs:complexType name="DeploymentTransformationSteps">
<xs:choice minOccurs="0" maxOccurs="unbounded">
<xs:element minOccurs="1" maxOccurs="1" name="CopyFolder" type="CopyFolder" />
<xs:element minOccurs="1" maxOccurs="1" name="ConfigFileTransformation" type="ConfigFileTransformation" />
<xs:element minOccurs="1" maxOccurs="1" name="Relocate" type="Relocate" />
<xs:element minOccurs="1" maxOccurs="1" name="RelocateConfigurationFile" type="RelocateConfigurationFile" />
<xs:element minOccurs="1" maxOccurs="1" name="Custom" type="CustomStep" />
<xs:element minOccurs="1" maxOccurs="1" name="IISExpress" type="IISExpressStep" />
<xs:element minOccurs="1" maxOccurs="1" name="CopyFile" type="CopyFile" />
</xs:choice>
</xs:complexType>
<xs:complexType name="CopyFolder">
<xs:attribute name="source" type="xs:string" use="required" />
<xs:attribute name="target" type="xs:string" use="required" />
<xs:attribute name="deleteFolderOnRestore" type="xs:boolean" use="optional" default="true" />
<xs:attribute name="deleteFolderOnApply" type="xs:boolean" use="optional" default="true" />
</xs:complexType>
<xs:complexType name="Relocate">
<xs:attribute name="targetFolder" type="xs:string" use="required" />
<xs:attribute name="deleteFolderOnRestore" type="xs:boolean" use="optional" default="true" />
</xs:complexType>
<xs:complexType name="ConfigFileTransformation">
<xs:sequence>
<xs:element minOccurs="1" maxOccurs="1" name="Transformation" type="xs:string" />
</xs:sequence>
<xs:attribute name="configFile" type="xs:string" use="optional" />
</xs:complexType>
<xs:complexType name="RelocateConfigurationFile">
<xs:attribute name="target" type="xs:string" use="required" />
</xs:complexType>
<xs:complexType name="CustomStep">
<xs:attribute name="type" type="xs:string" use="required" />
<xs:attribute name="arguments" type="xs:string" use="optional" />
</xs:complexType>
<xs:complexType name="IISExpressStep">
<xs:attribute name="webAppFolder" type="xs:string" use="required" />
<xs:attribute name="iisExpressPath" type="xs:string" use="optional" />
<xs:attribute name="port" type="xs:string" use="optional" default="8080" />
<xs:attribute name="useShellExecute" type="xs:boolean" use="optional" default="true" />
</xs:complexType>
<xs:complexType name="CopyFile">
<xs:attribute name="source" type="xs:string" use="required" />
<xs:attribute name="target" type="xs:string" use="optional" />
<xs:attribute name="targetFolder" type="xs:string" use="optional" />
<xs:attribute name="deleteOnRestore" type="xs:boolean" use="optional" default="true" />
</xs:complexType>
<!-- VSTest -->
<xs:complexType name="VSTest">
<xs:attribute name="testRetryResults" type="TestRetryResultsMode" use="optional" default="Separate" />
<xs:attribute name="passRateRelative" type="RelativePassRate" use="optional" default="1"/>
<xs:attribute name="passRateAbsolute" type="xs:int" use="optional"/>
</xs:complexType>
<!-- simple types-->
<xs:simpleType name="RelativePassRate">
<xs:union memberTypes="floatPassRate percentagePassRate" />
</xs:simpleType>
<xs:simpleType name="floatPassRate">
<xs:restriction base="xs:float">
<xs:minInclusive value="0"/>
<xs:maxInclusive value="1"/>
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="percentagePassRate">
<xs:restriction base="xs:string">
<xs:pattern value="(100(\.0*)?|[0-9][0-9](\.\d*)?)%" />
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="TestThreadCount">
<xs:union memberTypes="testThreadCountName testThreadCountNumber" />
</xs:simpleType>
<xs:simpleType name="testThreadCountNumber">
<xs:restriction base="xs:int">
<xs:minInclusive value="0" />
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="testThreadCountName">
<xs:restriction base="xs:string">
<xs:enumeration value="Max" />
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="TestSchedulingMode">
<xs:restriction base="xs:string">
<xs:enumeration value="Sequential" />
<xs:enumeration value="Random" />
<xs:enumeration value="Adaptive" />
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="RetryMode">
<xs:restriction base="xs:string">
<xs:enumeration value="None" />
<xs:enumeration value="Failing" />
<xs:enumeration value="All" />
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="guid">
<xs:restriction base="xs:string">
<xs:pattern value="{[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}}" />
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="ApartmentState">
<xs:restriction base="xs:string">
<xs:enumeration value="Unknown" />
<xs:enumeration value="STA" />
<xs:enumeration value="MTA" />
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="FrameworkVersion">
<xs:restriction base="xs:string">
<xs:enumeration value="Default" />
<xs:enumeration value="Net35" />
<xs:enumeration value="Net40" />
<xs:enumeration value="Net45" />
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="Platform">
<xs:restriction base="xs:string">
<xs:enumeration value="Default" />
<xs:enumeration value="x86" />
<xs:enumeration value="x64" />
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="TestThreadIsolation">
<xs:restriction base="xs:string">
<xs:enumeration value="SharedAppDomain" />
<xs:enumeration value="AppDomain" />
<xs:enumeration value="Process" />
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="ExistingFileHandlingStrategy">
<xs:restriction base="xs:string">
<xs:enumeration value="Default"/>
<xs:enumeration value="Overwrite"/>
<xs:enumeration value="IncrementFilename"/>
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="TestRetryResultsMode">
<xs:restriction base="xs:string">
<xs:enumeration value="Separate" />
<xs:enumeration value="Unified" />
</xs:restriction>
</xs:simpleType>
</xs:schema>
為什麼會這樣說呢?因為我一開始要設定報表輸出的位置
官方文件這樣寫
<TestProfile xmlns="http://www.specrun.com/schemas/2011/09/TestProfile"> <Settings name="Multiple Reports" projectName="SpecRun Test Project" /> <Report> <Template name="CustomReportTemplate_1.cshtml" outputName="Report1.html" copyAlsoToBaseFolder="true"/> <Template name="CustomReportTemplate_2.cshtml" outputName="Report2.html" copyAlsoToBaseFolder="true"/> </Report> <TestAssemblyPaths> <TestAssemblyPath>SpecRun.TestProject.dll</TestAssemblyPath> </TestAssemblyPaths> </TestProfile>
然後就不過不過不過不過不過不過
所以我自己去參考SpecRunTestProfile.xsd 然後去設定outputFolder才解決的問題
都設定完成就可以用cmd去執行runtests.cmd~如果你的人品還不錯的話應該就會跑開始跑測試並且產出報告囉~
沒有留言:
張貼留言