Customer Case Study
Exinda gets results using the SwiftTest 3000 to test their WAN optimization systems
About Exinda
Exinda is a global supplier of next generation WAN optimization products, delivering solutions to over
2,000 organizations in 80 countries that reduce network operating costs and ensure consistent application
performance over the WAN. The Exinda Unified Performance Management solution provides application visibility,
control, optimization, and intelligent acceleration — all within a single network appliance that
is affordable and easy to manage. Its main features help to optimize bandwidth utilization, set policies
to control peer-to-peer and recreational Internet traffic, and ensure application performance over the
WAN.
Exinda QA Challenges
The network management functions of the Exinda appliance cover network optimization, application visibility,
traffic control, and application acceleration. Exinda has separate licensing for different software offerings
in one product, so QA must test components separately — QoS, application visibility, and acceleration
— and together in a highly integrated environment.
Exinda provides the best of all worlds for its customers, with a high level of interaction between components.
This tight integration is a key factor for testing, and presents a number of challenges for the Exinda
QA team.
Exinda Test Environment
The test lab at Exinda supports product QA as well as development. The hardware setup includes dozens
of cabinets of equipment—servers, product hardware, test equipment, and a variety of devices —
all onsite next to the company's development area. The lab is designed to need minimum intervention,
with dedicated machines and very little configuration redo. It is automation heavy, with several racks
devoted to automated setups. In addition, Exinda has dedicated testing environments for acceleration,
requiring multiple appliances to emulate WANS, and simulate customer traffic patterns and topologies.
John Bothe, chief architect for Exinda, tells us that QA includes a team of engineers who work closely
with Exinda developers. They have resources dedicated to automation and some who are general staff, but
all are responsible for test case development.
Exinda Testing Capabilities
With homegrown test tools, Exinda needed improved reporting, automation, and scalability. With a rapidly
expanding customer base, their capabilities wouldn't fully support the depth and breadth of customer
use cases. John says that the test team's main pain point was in testing CIFS, with two problems front
and center. One, they needed to generate more TCP connections. Two, they didn't have sophisticated, comprehensive
test results reporting. Often, there was no indication of what might have gone wrong during the test,
so getting to the root cause of the problems was costly. In addition, the interactions of Exinda product
features were difficult and time consuming to test. John didn't have sufficiently detailed reporting
to measure the impact on throughput of various combinations of features, which led to longer manual test
cycles.
"One of the biggest benefits is the ability to automate, with error detection and flagging. And,
overall, for a single test case, we find at least 10x improvement — now 10-15 minutes compared
to half a day or more."
- John Bothe, Chief Architect
Enter SwiftTest
Exinda discovered the SwiftTest 3000 and quickly got an evaluation unit into the lab to put it through
its paces. John says that their setup experience was good. The QA guys had a few questions for SwiftTest,
but a web-based walkthrough beforehand enabled the team to use the interface and start testing on their
own, out of the gate. In the buying decision, Exinda looked at other ways to solve their problems. With
SwiftTest, the cost per port wasn't variable and there were no add-ons or licenses required. Exinda appreciated
that the SwiftTest 3000 was purpose-built and dedicated to high-capacity, performance-driven testing
without hidden costs.
About the SwiftTest 3000
A high-performance load-generating system, the SwiftTest 3000 is designed for comprehensive functional
and performance testing of IP storage. It provides protocol-level control and semantic definition that
enables the broadest possible range of test cases. The SwiftTest 3000 features 8 singly configurable
1GE ports, each capable of a million TCP connections. It also has a 1GE management port and supports
multiple authentication types, originates NFS, CIFS, iSCSI, and HTTP traffic, and easily handles multiple
test scenarios with varying lists of commands — all running concurrently. The SwiftTest 3000 is
the storage industry's elite testing tool due to its storage-specific features, high-volume traffic handling,
customer-centric graphic interface, and an automation framework.
Test Development Environment
SwiftTest products include a graphical application, the SwiftTest Test Development Environment, that
provides flexible, granular configuration options and test results analysis. Set up tests, find problems,
modify parameters, and quickly see results via an intuitive drag-and-drop interface.
The Results
While Exinda's homegrown tests did generate CIFS connections and do basic acceleration testing, the
number of connections and throughput were limited and error reporting was weak, making troubleshooting
difficult. John had servers and a console to run scripts driving one testbed at a time, but just one
test at a time. With SwiftTest, not only can John plug in and have CIFS traffic flowing in minutes, he
can run multiple tests at once. He saves test settings, takes advantage of statistics and reporting,
and accurately benchmarks against claims. For a single test case, what formerly took half a day or more
— configuring servers and software to run the test, actually running it and then generating reports
— now takes 10-15 minutes.
Beyond dramatic time savings, the single biggest benefit for the team is the ability to automate all
of this and integrate high performance testing into their automation setups. For example, say you have
continuous integration systems detecting changes to the source tree that kick off automated tests. Check
that if you send 10MB, the box reports 10MB. Or if a normal 30-second sequence of traffic goes up to
60 seconds, it's automatically flagged. Previously this was not at all simple to do with homegrown tools.
In the Exinda lab, SwiftTest 3000 ports are dedicated permanently to continuous automated testing. With
rapid customer
growth, Exinda will continue to add ports to scale with the ongoing expansion of the QA lab. The SwiftTest
3000 has become an integral part of Exinda's QA environment.
John is also using the SwiftTest 3000 for general testing. Not only can they test CIFS, they can do
a general verification of QoS under load. In addition, the product developers who work in the lab take
advantage of the system to test feature sets and work in a more-accurately-simulated customer environment.
All in all, SwiftTest has made a significant difference in the productivity of the Exinda QA lab.
"We're growing fast, and need test tools beyond what we can do inhouse. We found SwiftTest to be
better than any other solution we looked at and the SwiftTest 3000 is more than filling our needs. Particularly
with the depth and maturity of their CIFS tools, we can expand into new markets and support bigger customers."
Download a printable version
