Large Scale AS/400 Load Testing
||In November of 1995, Network Pathways received a request
from a large utility looking for help to reduce the risk in implementing its first AS/400
application. Our client was concerned the application might be too slow or the AS/400
might not "fit" in its existing TCP/IP environment. The utilitys
executive had to know with certainty that the new system would work when their 500 users
signed on for the first time! The project implementation strategy was unforgiving in
this regard -- a 20 year old legacy system would to be "turned off" on a
Friday evening and replaced by an AS/400 system that would "go live" two days
latter. Added to this, the utility was about to become the application
vendors largest installed site by a factor of two.
Pathways was asked to lead the acceptance test effort for this project and immediately
began looking for automated test tools. After an extensive search of available
products, three software vendors were selected for proof-of-concept demonstrations and,
based on these evaluations, Performix/TTY
from Rational Software Corporation was selected.
Over the next 6 months Network Pathways developed a test capability
which at that time was likely unparalleled anywhere. Today we believe it may only be
equaled by IBM's Rochester facility.
Using Rational Software's Performix/TTY test tools and StarQuest Connectivity's TN5250 terminal emulator,
Network Pathways was able to generate accurate workloads consisting of 100s of
"virtual" users all running telnet 5250 sessions on the target AS/400 system.
The Performix/TTY test harness was found to be extremely scalable and capable of
generating 1000's of virtual users. Users that operate real AS/400 applications and
that look and feel exactly like real users to the AS/400 system under test.
The Test Environment
||The test bed used for this project consisted of three
Pentium-166Mhz test drivers all running SCO UNIX. Each test driver was used to
generate between 140 and 170 virtual users depending on the requirements of the specific
workload. The Performix/TTY tools made it easy to coordinate complex workloads
across multiple test drivers and provided the ability to scale workloads to 1000s of
Workloads were developed to model the business activity expected with
300 users operating the new system under different seasonal conditions. These models
included 5 different classes of users (call center, field staff, etc.) and 13 different
business events (service request, trouble report, account inquiry, etc.). Classes
and events were easily combined to create a variety of workloads which simulated the
expected worse case scenarios at different times of the year.
These same workloads were then scaled from 50 to 500 users to develop
capacity curves, which predicted the performance brick walls!
Stress tests were devised to subject the application to unrealistic
workloads to determine the weakest points. In these tests, users performed the same
operations in lock step and committed their transactions at the precisely the same
time. Tests were run with 5 to 100 users, adding 5 more each time, to see when/if
the system would break. In each test one virtual user was appointed to
"supervise" the others. Each time the users reached a rendezvous point,
the "supervisor" would signal them to all click [OK] at exactly the same
time. This is a punishing test that could not have been duplicated manually.
Also, Sign-On tests were run to ensure the system would be able to
handle all 500 users signing on to the application during a half-hour period in the
||Performix/TTY allowed us to measure the AS/400
applications response times with pinpoint accuracy. It provided detailed
statistics and identified application bottlenecks. The test results were easy to
understand, well organized, and easily summarized by class of user, business event,
individual screen, and groups of screens.
||In 1995, Performix/TTY was known as Empower/TTY. The
product was originally developed by Performix, Inc. of McLean, Virginia. In 1996,
Performix, Inc. was acquired by Pure Software, Inc. which latter merged with Atria
Software Inc. to form Pure Atria, Inc.. In 1997, Pure Atria was acquired by
Rational Software Corporation.