LoadRunner性能数据结果分析连载(引言)

HP Mercury LoadRunner 是一款功能相当强大的性能测试工具,由三个部分构成, VUGen, Controller以及Analysis. 其中VUGen负责进行脚本录制, Controller是一个总控中心,负责场景的配置,监控器的选取和监控,并选择合适的负载生成器进行执行, Analysis是一个分析模块,主要负责所有执行数据的分析以及报告的生成.
之所以说LoadRunner是强大的性能测试工具,主要是因为VUGen支持大概好几十种主流的协议. 因此支持的被测对象相当广泛,另外Analysis也有超强的功能,提供非常丰富的图表,供测试结束之后分析和定位问题.
我使用Mercury LoadRunner有一年左右,2006年6月通过了LoadRunner的SP考试,并于12月底参加了CPC考试,以2分之差失败. 在这一年之内对Controller和Analysis的使用有一些心得和体会,大概能看懂一些主要的分析图表,包括事务相关的一些图表,所以希望作一个总结,对希望学习和了解LoadRunner的朋友有一些帮助. 因为时间有限,所以初步打算以系列的形式发表. 下次打算介绍Analysis提供图表的概述.

--to be continued

LoadRunner的工作原理(第二部分)

LoadRunner Internal Architecture This Visio 2002 file is based on LR8.0

application servers under test are placed under stress by driver processes mdrv.exe (the Multi-threaded Driver Process) and r3vuser.exe which emulate application clients such as Internet Explorer web brower. It performs 3 main actions: Kli> cpp (C language pre-processor) cci (C pre-compiling) which creaes a file with ci file, and execute using the driver for the protocol technology being tested. Runs can be invoked to run "silently" by invoking Mdrv.exe from a Windows batch script. Mdrv can automatically stop loading Vusers because they communicate with Vusers and monitor CPU usage on Windows Load Generator machines. A separate JVM is instantiated by each Java-based Vuser on Windows-based machines. Java Vusers are not supported on Unix platforms. virtual Vusers are invoked as groups (logical collection of virtual users running the same script on a specific load generator machine) by agents (3,900K magentproc.exe) running as a service or as a process on load generator client machines. Each machine hosting agents maintains an Execution Log in a .qtp file. When logging is enabled, the agent also creates within the results folder a sequential log file for each Vuser (segregated by Vuser group). During execution, this file is displayed in the view > Show Output window on the LoadRunner Controller machine. Upon a pre-set delay, the Scheduler running on a Controller machine instructs agents (via Windows port 54345 or dynamic Unix port) to initiate test session scenarios. The Controller (wlrun.exe) sends a copy of scenario files along with the request. Agents are launched by the Remote Agent Dispatcher process (formerly called Remote Command Launcher (RCL)) on each load generator machine. Each agent refer to scenario (.lrs) definition files to determine which Vuser groups and scripts to run on host machines. This means the Controller can be started from a DOS batch (.bat) file (preferrably with a short name on a root drive): REM Start Controller: SET M_ROOT=C:\Program Files\Mercury Interactive\LoadRunner\bin cd %M_ROOT% wlrun.exe -TestPath D:\Dev\Dev1.lrs -port 8080 -Run -DontClose pause Press Ctrl-Z to keep this window or Including the -Run parameter is the same as manually pressing the "Start Scenario" automatically upon invocation. This is not a good idea because you may have to decide about collating the file from a previous run or want to change the output folder. This assumes that the system's environment PATH variable was updated to include where LoadRunner is installed.

The Controller is invoked using parameter values within files in the Windows OS folder (WINNT for Windows 2000 and WINDOWS for Windows XP). The Windows folder is used because LoadRunner is designed to have only one instance of Controller running at a time on a machine. To quickly switch among several applications, save a copy of LoadRunner's ini files after working on it within the Controller, then use Notepad to craft a batch file to copy application-specific versions of ini files before executing wlrun. An example of file copy actions for application XXX: copy %WinDir%/wlrun7-XXX.ini %WinDir%/wlrun7.ini copy %WinDir%/wlrun7-XXX.dft %WinDir%/wlrun7.dft Some defaults you might want to change: In the wlrun7.ini file file [output] section, MaxNumberOfOutputMessages= from 10000 to 100000 for long runs. This limits the number of output messages stored in the database. MaxOutputUIRowsToShow limits the amount of messages/errors (lines) displayed in the Controller's Output window. In the QTWeb.lrp file within the LoadRunner Program Files dat\protocols folder section [Vugen], add entry MaxThreadPerProcess=5 to limit the number of threads managed by each load generator mdrv.exe process. Values for DefaultScenarioDir, DefaultScriptDir, DefaultResultDir, and [Recent File List] stored in the wlrun5.ini and wlrun7.dft files updated whenever values are changed within the Controller. The blocks of actions taken by each Vuser are defined in Vu scripts created using Loadrunner's VuGen.exe. When this program is invoked, it stores in the Windows folder a comparamui.INI file to save under "[LastTablesUsed]" a history of files and [ParamDialogDates] specified using menu option Insert > New Parameter > Dates. VuGen stores and retrieves a vugen.ini file in the Windows folder. When using Java, enable additional debug options: [DynaDlg] JavaLevel=3 When using 8.0 scripts within VuGen 8.1, add to Vugen.ini: [Editor] OLDEDITOR = 1 VuGen opens in LR folder template/qtweb default.cfg and script files. Vu scripts can be coded to use variable values obtained from parameter files external to the script. I have a lot more on VuGen here During a run, execution results are stored to a results folder. I prefer to set Results Settings to "Automatically create a results directory for each scenario execution." which means that LR will increment the name of the Results Name when I start a scenario runs. For example, a value of "Res11" will be automatically incremented to "Res12" or sometimes "Res11-1". Errors are written to the output.mdb MS Access database. See the ASP page I have written to access this database Within each results folder, a "Log" folder is automatically created to contain a log file for each group. After a run, to view a log file from within the Controller, click then right-click on a group to select "Show Vuser Log". As a scenario is run, monitors maintain counters locally on each host. After a run, the "collate" process takes .eve and .lrr result files and creates in the results folder a temporary .mdb (MS-Access) database. To prevent errors when processing large result files, use MSDE (Microsoft SQL Desktop Engine). Don't install it from the Add-in folder on the LoadRunner 7.8 CD, which is obsolete SQL7. Download MSDE 2000 Release A which includes MSDE 2000 Service Pack 3a and MDAC 2.7 SP1a for use by Analysis on any Windows machine. Extract the file and share that folder. Open a command window to run a command such as: setup SAPWD="StrongPassword" INSTANCENAME="LR" SECURITYMODE=SQL DISABLENETWORKPROTOCOLS=0 /L*v path to log file Using Windows Explorer, share the Data folder. Then in Analysis Options > Database tab, use 8.3 names without spaces (indentified with DOS command DIR /X): Input the SAPWD password specified above. Logical Storage location: \\loadclient02\Data (the folder you shared) Physical Storage Location: C:\PROGRA~1\MICROS~1\MSSQL\Data (not C:\Program Files\Microsoft SQL Server\MSSQL\Data) Click "Test parameters". (This takes a few seconds) The Analysis Module (8,320K analysisu.exe) generates analysis graphs and reports using data from the .mdb database. The LoadRunner Results file results_name.lrr from each scenario run -- also called an Analysis document file -- is read by the Analysis program to display Percentile graphs. By default, the LRReport folder is created in the test analyst's local machine My Documents folder to store Analysis Session files. They can optionally be formated in HTML. Their format are controlled by a .tem template file. Optionally, Mercury's Remote Performance Monitoring (RPM) MS-IIS/ASP web server for LoadRunner 7.8 can be installed on a Windows 2000 server ( but not on a Windows 2003 server) so that load test results to be viewed using a web browser. Not pictured is the LoadRunner Tuning Module (a separate $50,000 product).

LoadRunner的工作原理(第一部分)

Architecture Overview LoadRunner works by creating virtual users who take the place of real users operating client software, such as Internet Explorer sending requests using the HTTP protocol to IIS or Apache web servers. Requests from many virtual user clients are generated by "Load Generators" in order to create a load on various servers under test These load generator agents are started and stopped by Mercury's "Controller" program. The Controller controls load test runs based on "Scenarios" invoking compiled "Scripts" and associated "Run-time Settings". Scripts are crafted using Mercury's "Virtual user script Generator" (named "V U Gen"), It generates C-language script code to be executed by virtual users by capturing network traffic between Internet application clients and servers. With Java clients, VuGen captures calls by hooking within the client JVM. During runs, the status of each machine is monitored by the Controller. At the end of each run, the Controller combines its monitoring logs with logs obtained from load generators, and makes them available to the "Analysis" program, which can then create run result reports and graphs for Microsoft Word, Crystal Reports, or an HTML webpage browser. Each HTML report page generated by Analysis includes a link to results in a text file which Microsoft Excel can open to perform additional analysis. Errors during each run are stored in a database which can be read using Microsoft Access

Virtual Users (Vusers) Unlike a WinRunner workstation which emulates a single user's use of a client, LoadRunner can emulate thousands of Virtual Users. Load generators are controlled by VuGen scripts which issue non-GUI API calls using the same protocols as the client under test. But WinRunner GUI Vusers emulate keystrokes, mouse clicks, and other User Interface actions on the client being tested Only one GUI user can run from a machine unless LoadRunner Terminal Services Manager manages remote machines with Terminal Server Agent enabled and logged into a Terminal Services Client session. During run-time, threaded vusers share a common memory pool. So threading supports more Vusers per load generator. The Status of Vusers on all load generators start from "Running", then go to "Ready" after going through the init section of the script. Vusers are "Finished" in passed or failed end status. Vusers are automatically "Stopped" when the Load Generator is overloaded. No additional license is needed to monitor standard web (HTTP) servers (Apache, IIS, and Netscape). To use Web Services Monitors for SOAP and XML, a separate license is needed, and vUsers require the Web Services add-in installed with Feature Pack (FP1)

Product Versions Version 8.1 Feature Pack 4 patch installer LR81FP4P136.exe at 7,786,800 bytes, was signed on January 2, 2007 as file 8.1.4.0 (Build: 1735) is Recorder Version: 1290. Version 8.1 Feature Pack 4 installer LR81FP4.exe, at 194,644,720 bytes, was signed on December 15, 2006 as file version 8.1.4.0 (Build: 2249) is Recorder Version: 1289. This requires an upgrade to MS.NET 2.0 clients. Version 8.1 Feature Pack 3 installer LR81FP3.exe, at 116,601,240 bytes, was signed on June 18, 2006 as file version 8.1.3.0 (Build 2085). It installs (as an item on your Start > Program Files) Microsoft WSE (Web Services Enhancements) 2.0 SP3 to deploy security policies for sytems running .NET Framework 1.1. Version 8.1 became available October 2005. In VuGen it adds a "Workflow View", "Workflow Wizard", and a memory leak which is fixed with a patch downloadable since Dec. 2005. It renames the VuGen "Execution Log" the "Replay Log". Version 8.0 became available August 2004. It adds "Additional Attributes" to Runtime Settings. It also adds (for additional fee) diagnostics and tuning capabilities, allowing Transaction Breakdown to breakdown transaction times across different servers servicing various transaction layers (web server, Oracle 11i & Peoplesoft 8 app server, database) layers. It separates SQL time in execute, parse, and fetch times. Version 7.8 Feature Pack 1 added support for Windows XP. Version 7.8 became available September 2003. Version 6.5 available June 2000 offered new "TurboLoad" technology -- a completely new replay engine that runs thousands of vusers using a single operating system thread. Version 6.0 used a separate thread per user, which required almost 10 times more i/o and CPU cycles than 6.5.

Note: Links to documents that used to be here were removed after Mercury Interactive, Inc. lawyers demanded their removal. Page numbers in online pdf files are different (have more pages) than page numbers in the paper document of the same title.

Application Components' Requirements Loadrunner makes use of four executables with different system requirements Application Product Process Image Name V8.0 Img KB File Size - LRLauncherApp.exe 16,288 n/a Virtual User Generator VuGen.exe 12,436 2,334,769 Controller with On-Line Monitors wlrun.exe 13,076 5,681,215 Load Generator Agent magentproc.exe 3,236 magentservice.exe 65,536 mdrv.exe Analysis Analysisui.exe 13,132 6,058,496 Tuning Console protune.exe 3,403,833

Console programs perl5.8.0.exe Interpreter 20,535 regtlb.exe registers the batch automation type library 30,720 sed.exe 55,296 wdiff.exe Compares text files 197,632 Alex Arbitman's LR 7.8 Footprints.xls reports that to run Web requires __ per process and __ per thread.

Using Windows Remote Desktop Connection Remote Desktop Connection (part of the Terminal Services that comes with Winodows XP) is not as reliable with LoadRunner as Remote Administrator. To keep Windows Remote Desktop Connection sessions from timing out during a test, the Terminal Services on each machine should be configured as follows: Click Start, point to Programs (or Control Panel), Administrative Tools and choose Terminal Services Configuration. Open the Connections folder in tree by clicking it once. Right-click RDP-Tcp and select Properties. Click the Sessions tab. Make sure "Override user settings" is checked. Set Idle session limit to the maximum of 2 days instead of the default 2 hours. Click Apply. Click OK to confirm message "Configuration changes have been made to the system registry; however, the user session now active on the RDP-Tcp connection will not be changed." Make sure that when you do this you're not violating one of your corporation's security policies. Terminal Server only allows two simultaneous connections. To disconnect from a session, do not click "X" on the remote desktop window but click Start and Log Off.

Quality Web Systems: Performance, Security, and Usability (Addison-Wesley Professional; Aug. 2001) by Elfriede Dustin, Jeff Rashka, Douglas McDiarmid The Web Testing Handbook (S T Q E Pub, Jan. 2001) by Steven Splaine, Stefan P. Jaskiel, Alberto Savoia is a good overview introduction, but does not make you an expert.

Among Common problems installing LoadRunner: Windows 2003 & XP SP2 have a DEP (Data Execution Prevention) feature which prevents VuGen recording. Go to Control Panel, System -> Advanced tab, Performance section "Settings" button Data Execution Prevention tab and add the client program or choose "Turn on DEP for essential windows programs and services only." A reboot is required. LR Installation and Configuration Download [after required registration] the 10 day trial from here or here. I recommend that you put downloaded LoadRunner installation files and patches to a separate media such as a CD or USB drive. Then mark those files as read-only. Disable your anti-virus software (Symantec, McAfee, etc.) before invoking on installers. Virus Detection engines (such as Hauri versions since May 18) may find that program regtlb.exe (which registers/unregisters type libraries) to contain a "virus" they call "Backdoor.Win32.PoeBot.15872". Automatic repair by the virus remover will break those files. The LR box comes with two CD's and this installation manual. Separate installation manuals are available for the Controller and Analysis modules. The Windows CD autostarts to this initial screen for v7.8 and this initial screen for v8.0 You can install just a single component (such as VuGen) by (ironically) selecting "Full install" and then the "Custom" option to check the specific components to install. However, due to a strange bug with v8.0, before you do that, first install the Load Generator, then return to install "custom" components. The UNIX CD installs only the Load Generator (not the Controller or VuGen) on UNIX machines because the Controller and VuGen only run on Windows machines. Zero fill machine names to t001, ... t010, etc. The LR Controller sorts machines named t1, t2, ... t10 as t1, t10, t2. Location of Program Files The LoadRunner installation program adds files in Program Files, Windows folder, and the Windows Registry that are NOT removed during un-installation. If you get a "License violation" message, you need to get from Mercury support a one-day license key to install with. By default, different versions of LoadRunner are installed to different parts of your hard disk: LoadRunner 8.1 executable files are installed to file path "C:\Program Files\Mercury\LoadRunner\bin" LoadRunner 8.0 "stutters" when it installs to its default file path "C:\Program Files\Mercury Interactive\Mercury LoadRunner\bin" Note: Even though 8.0 uses a different folder, folders created by previous versions still need to be removed before its installation. LoadRunner 7.8 executable files are installed to file path "C:\Program Files\Mercury Interactive\LoadRunner\bin" When working with Java, instead of overridding these default installation folder to a path without spaces (such as C:\LR78) as recommended by KB article 11878, just use the equivalent DOS 8.3 file names: For LoadRunner 8.1: C:\PROGRA~\MERCUR~1\LoadRunner\lib For LoadRunner 8.0: C:\PROGRA~\MERCUR~1\MERCUR~1\lib To quickly get at this LoadRunner installation folder, create an environment variable named "LR81" so you can use a quick command such as cd %lr81% To get at this folder quickly, I created a batch file named "L.bat" in the cmd's default C:\ root folder containing this: cd \Program Files\Mercury\LoadRunner\bin pause I created a shortcut to this file on my desktop and dragged it over Windows so that I can click into that folder from anywhere. The pause command ensures that the command window does not disappear automatically. Alternately, from within a command window I can just type "L" and press Enter. Files in Windows Folders This path specified is stored under [ProductEnv] as M_ROOT in file wlrun.ini in the C:\WINNT (or C:\Windows) folder. The C:\WINNT (or C:\Windows) folder also holds the Maintenance Number (MPN) specified during installation, stored as a parameter named "LoadRunner_SerialNumber" (such as 1234-1234567890) in the mercury.ini file. Start Menu The LR installer adds links to the most used programs in > Programs > Mercury Interactive > LoadRunner However, some programs are installed which are not conveniently listed there, such as WDiff.exe v1.49 to compare differences between two ASCII text files. It has an accompanying help file Beginning with v7, LoadRunner prevents software piracy (much like Microsoft began doing with Windows XP) by requiring that a license key be provided within 10 days of installation. Mercury generates its license key based on a host ID generated on each computer. CPT12784.doc With v7.x, to generate Generate a HostID key (such as "XCCWJU-APBE-BYDS") click down Programs > Mercury LoadRunner > LoadRunner > License tab The key can be obtained before installation from program licidgenerator.exe and (after registration) its lm70.dll from the installation CD folder \lrunner\lm70.nt\bin or \lrunner\setup\lm70.nt\bin. Sample Apps / Protocols Running the samples install program populates folder C:\Program Files\Mercury Interactive\Mercury LoadRunner\ Copy the link location below into the "Program to record:" field: Protocol Server Client Program Parameter Notes Web WebTours\StartServer.bat http://localhost:1080/mercuryWebTours COM/DCOM (Operating System) samples\bin\frsui.exe Winsock sockfrs.exe samples\bin\flights.exe Winsock WinSockWeb ODBC (MS Access) samples\bin\flights.exe ODBC_Access CORBA samples\CorbaSamples\server.cmd & samples\CorbaSamples\server.bat samples\CorbaSamples\client.cmd & samples\Corbasamples\clientrecord.cmd Stuart Moncrieff's article on CORBA RMI samples\RMISamples\server.cmd & samples\RMISamples\server.bat samples\RMISamples\client.cmd & samples\RMISamples\clientrecord.cmd According to CPT11877.doc, JDK 1.5 users need to contact Mercury Support for a patch to each specific LoadRunner version (7.6, 7.8 FP1 or 8.0). Otherwise, you'll get these messages: Error: Failed to find javac.exe Java Compiler in Path and JDK installation folder in registry. [MsgId: MERR-22981] Error: Failed to compile the Actions.java file. Please add the \bin to the path and try again. [MsgId: MERR-22996] Warning: Extension java_int.dll reports error -1 on call to function ExtPerProcessInitialize [MsgId: MWAR-10485] Error: Thread Context: Call to service of the driver failed, reason - thread context wasn't initialized on this thread. [MsgId: MERR-10176] The Java sample apps use the "flight32lr" User Data Source with Microsoft Access driver(*.mdb) in the USER DNS table in Data Sources(ODBC) of the VuGen's local machine. Additionally, the sample Java servers must be operational prior to starting the client. This is done with the "samples\RMISamples\server.cmd": set lrpath=C:\PROGRA~1\Java\jre1.5.0_02\bin;C:\PROGRA~1\MERCUR~1\MERCUR~1\classes set lrclasspath=C:\PROGRA~1\MERCUR~1\MERCUR~1\classes;C:\PROGRA~1\MERCUR~1\MERCUR~1\classes\srv;C:\PROGRA~1\Java\jre1.5.0_02\lib\rt.jar set flightRmi=%~dp0;%~dp0RmiSamples.zip set classpath=%lrclasspath%;%flightRmi%;C:\PROGRA~1\Java\lib\rt.jar;.;%classpath% set path=%lrpath%;.;%path% cd %~dp0 start java -Djava.security.policy="%~dp0RmiFlights.policy" RmiFlights.Server Note the location of loadrunner class files I added to the default sample. They are pre-pended to the existing classpath. Note that there are no spaces in the file path. The Zip file is equivalent to a JAR file in Unix systems. Do not delete the black command window because the Java server runs within it. CORBA and RMI Java clients are invoked with a command for Windows to start the java.exe program. This "samples\RMISamples\client.cmd" file contains: set lrpath=C:\PROGRA~1\Java\jdk1.5.0_02\bin;C:\PROGRA~1\MERCUR~1\MERCUR~1\classes set lrclasspath=C:\PROGRA~1\MERCUR~1\MERCUR~1\classes;C:\PROGRA~1\MERCUR~1\MERCUR~1\classes\srv;C:\PROGRA~1\Java\jdk1.5.0_02\lib\rt.jar set flightRmi=%~dp0;%~dp0RmiSamples.zip set classpath=%lrclasspath%;%flightRmi%;C:\PROGRA~1\Java\jdk1.5.0_02\lib\rt.jar;.;%classpath% set path=%lrpath%;.;%path% cd %~dp0 start java RmiFlights.main Note that the RmiFlights.main class file name is passed into java for it to load. When recording Java with VuGen, a different command — such as the sample clientRecord.cmd — needs to be invoked because VuGen needs to be invoked within the JVM sandbox: set flightRmi=%~dp0;%~dp0RmiSamples.zip set classpath=%flightRmi%;%classpath% cd %~dp0 start InvokeVugen.exe The location of the JDK needs to be specified in the Windows PATH environment variable PATH to avoid this message: Error: Failed to find javac.exe Java Compiler in Path and JDK installation folder in registry. [MsgId: MERR-22981] VuGen "Java Vusers" can only operate as Single Vuser mode (not multi-vuser). Instead of web "Start recording", Java VuGen scripts invoke Java functions within the Actions section. "vuser_init" and "vuser_end" actions are not relevant within Java VuScripts. Internally, the cjhook.ini file specifies which Java classes can hook in its [EXC_SYSTEM_CL] section. Java classes specified in the [SYSTEM_CL] section are not hooked. The user.hooks file in LR \bin folder is a general format and cannot be used as-in. It needs to be copied. Uninstall To uninstall LoadRunner, you must be logged in with the same Windows userID as was used during original installation! If you use a different userid, the uninstall will delete only the dat folder which contains the "miuninst" file. Unlike Microsoft Office applications, Mercury has not programmed invididual components to be selectively uninstalled on its own.