Customer Case Study
The SwiftTest 3000 enables scalability testing for QA team at F5 Networks
The F5® team responsible for QA of the company’s BIG-IP® WAN acceleration technology has
deployed the SwiftTest 3000 in their lab, cutting the time spent on testing in half and enabling the
scaling of test scenarios to accurately simulate customer environments. No more writing homegrown tools
or reconfiguring fragile scripts for each test case. The SwiftTest 3000 generates CIFS traffic that simulates
actual Windows®-based applications, with the flexibility to customize at will.
About F5 Networks
F5 provides the means for businesses to maximize IT infrastructure agility and effectiveness. F5 sells
a variety of storage, networking, and application solutions that require their product teams to develop
for the broadest imaginable range of customers. One of the significant capabilities of the F5 BIG-IP
product line is WAN acceleration, including Microsoft® CIFS (Common Internet File System) protocol.
And this testing lands in the lap of Terry Swartz, QA manager at F5. Terry is charged with implementing
the most efficient test processes possible. As with all competitive businesses, he needs to do more with
less each day. Terry says his goal is to automate as much as he can to relieve his team of labor-intensive,
low-value tasks. So he is persistent in his hunt for the newest equipment and tools to do just that.
The Trouble with CIFS
The particular challenges faced by the F5 test team highlighted in this case study revolve around testing
their WAN accelerator product capability for accelerating CIFS traffic. Dealing with CIFS presents a
number of difficulties, including setting up your QA environment to accurately simulate Microsoft applications.
This requires a deep understanding of the CIFS protocol. CIFS is a chatty protocol, generating a lot
of traffic for every request. This chattiness provides an opportunity for companies like F5 to develop
products that speed up file access.
F5 WAN Acceleration Basics
To greatly simplify the topology of the WAN acceleration testing configuration, one F5 BIG-IP appliance
sits between the client machines and the network in a remote location. Another appliance sits in the
datacenter between the network and the servers. This two-device testing setup leverages F5’s iSessions
technology to secure and accelerate data traveling over the WAN. With iSessions, two BIG-IP devices can
be deployed symmetrically to create a site-to-site secure connection to improve transfer rates, reduce
bandwidth, and offload applications for more efficient WAN communication. The two appliances inspect
CIFS traffic and respond to requests that are redundant or deemed unnecessary without sending them to
the other end. This reduces the number of packets sent and improves overall network performance. To effectively
test F5 BIG-IP’s WAN acceleration capabilities, the team must generate traffic on the client side
that is Microsoft application specific, as well as accurately simulate application responses on the server
side.
Before SwiftTest
The F5 QA team was manually setting up both client and server side, and writing test tools from scratch.
Terry says they were writing a lot of complicated scripts and spending too much time configuring whitebox
servers and the network. And with every new hardware platform and feature set that F5 delivers to its
customers, the testing matrix gets bigger and the challenges compounded.
One Customer Scenario
A typical test for F5 is to simulate a customer with 200 employees using Windows desktops and sharing
a filesystem. Testing a client load of a dozen desktops is easy, but to create realistic traffic for
200+ Windows clients is a daunting effort. And ideally, F5 would want to test a scenario with 1000 desktops.
Such scale is practically impossible with manually-constructed testing environments, whether you’re
using real servers or virtual machines. It’s no wonder that Terry Swartz is constantly on the look-out
for tools and technologies to make his capabilities broader, his team faster, and results better.
Enter SwiftTest
SwiftTest offers a high-performance appliance featuring 8 ports that are each configurable as either
a CIFS client or CIFS server. The SwiftTest 3000 includes the SwiftTest Test Development Environment,
a graphical application for configuring and running tests, and analyzing results. Terry provided invaluable
input into the feature set and helped identify functionality critical to a company like F5. “SwiftTest
jumped all the hurdles,” says Terry, “they were responsive, receptive, and we got everything
we asked for.”
“SwiftTest has provided a flexible feature set for effective testing. The SwiftTest 3000 enables
us to create complex tests, scalable tests, and of course repeatable tests very efficiently through the
user interface. And the automation capability plugs into our existing automation framework.”
- Terry Swartz, QA Manager at F5 Networks
SwiftTest 3000 at F5
F5 is using the SwiftTest 3000 to simulate multiple CIFS clients and servers in a WAN acceleration test
environment, with clients writing and reading F5-specific content to the servers. The machine is providing
the automation Terry was looking for and has enabled him to reduce by half the manpower required to run
the CIFS test. He has now moved engineering talent to more valuable tasks. And he’s realizing better
results faster, with scaling potential for the long term.
The F5 Growth Path
For Terry’s QA team, the capacity to scale aligns with the company’s growth path. Instead
of struggling with added requirements or playing catch-up with the company’s technology advances
and increasing customer needs, they’re ready for whatever F5 developers come up with.
Isolation and Resolution
The SwiftTest 3000 isn’t only about traffic. SwiftTest Test Development Environment software features
a graphical interface providing granular configuration options and results analysis. Testers can now
find problems, modify parameters, and quickly see results using the SwiftTest interface. Problem isolation
is fast and accurate. This helps Terry’s test team work extremely effectively with F5 engineers
who can now address any potential issues more quickly.
Replacing Physical Machines
Terry’s mandate is to create efficient test configurations that can be used for as many scenarios
as possible. Using a dozen whitebox servers to simulate average customer environments was one standard
set-up. Those servers have now been replaced by a single SwiftTest 3000 that enables Terry to easily
simulate 1000s of concurrent users. The bottom line is that the lab’s power limitations and maintenance
costs make replacing racks of servers with a single appliance a huge win.
The Last Word
By using the SwiftTest 3000, F5 not only realizes the physical benefits of eliminating hardware, they
now reliably test application-level protocols in scalable, real-world environments, using fewer engineers
with better results.
“The SwiftTest 3000 has improved our testing coverage, reduced the number of engineers required
to run our test suites, and eliminated the need to buy additional servers.”
- Terry Swartz, QA Manager
SwiftTest 3000 Advantages
- Cost Savings: Costs less than the servers and software required to generate the same volume of traffic.
- Real Content: User-specified content can be written to the server, read from the server, and verified.
- Granular Control and Reporting: CIFS, NFS, and TCP/IP configuration options far exceed homegrown tools.
- Quicker Test Cycles: Wizards and canned tests minimize test development time. Automation reduces test
suite execution time.
Download a printable version
