Skytap Provides Platform for Ellie Mae Enterprise Applications
Ellie Mae is a provider of software and services for the mortgage industry. Each day, loan originators such as lenders, credit unions, banks and mortgage brokers depend on our solutions. They use our software for residential loan origination and mortgage management, to order and process mortgage loan closing documents, to manage mortgage documents electronically, to create and manage mortgage company websites, and to connect via Internet to partners and vendors throughout the mortgage industry.
To keep Ellie Mae and its customers at the forefront of the mortgage business and technology, we have a 50-person IT team working to deliver a full spectrum of applications that enable business growth. My role as a Quality Assurance Director ensures the smooth integration of these applications and business processes.
Like most IT-centric organizations, our testing infrastructure was becoming a bottleneck for releasing new software. We had recently expanded our team to include offshore testing resources to help us reduce cost and add extra capacity to deliver new software on time. However, the offshore test team also brought some new challenges to Ellie Mae. The offshore and US infrastructures were physically separate and distinct. Neither of our infrastructures were as adaptive or agile as they needed to be to keep pace with the changing and increasing testing demands. Having two separate and distinct environments did not lend itself to good team collaboration. In addition, both US and offshore groups only received limited support for the set-up and tear-down of test environments because IT operations were often focused on production and other tasks. We quickly found that it was difficult for our overseas counterpart to collaborate with the team in the US on activities and testing scenarios given their different locations and environments.
So, we decided to look for a solution that would enable our global team to:
• Employ a cloud that is easy to build and maintain, is as maintenance free as possible, and that could be shared by the US and offshore teams.
• Scale up testing resources during periods when testing demands are heavy.
• Scale down testing resources during periods when testing demands are low.
• Automate the set-up and tear-down of new environments during the development process.
• Complete complex test matrices (including cross platform testing) without requiring significant manual effort.
• Gain self-service access to cloud environments, thus avoiding IT operations delays.
• Enable distributed teams to work together more effectively, including the sharing of datacenter resources.
• Deploy new applications seamlessly to Ellie Mae’s production systems located in a SAVVIS managed data center.
Exploring Our Options
During our evaluation, we were reluctant to invest in an on-site virtualization infrastructure because of the hardware, storage, virtualization software, implementation costs, and technical knowledge requirements. Plus, our IT operations team was unlikely to have the bandwidth to quickly build out a virtual environment given they were focused on more pressing production activities. Given the industry’s shift to cloud computing, we decided to explore this option. It seemed like a logical solution as we faced an urgent need to scale resources for upcoming projects, and needed testing resources to ensure that the company was fully compatible with the new Microsoft Windows 7 release.
After a review of several cloud service providers, we quickly discovered that many of these clouds lacked support for the complex software stacks Ellie Mae needed to test. For example, we needed the ability to test a range of client and server configurations, including testing of the upcoming Windows 7 release. It appeared that most cloud providers are designed for Web hosting rather than software testing. We also wanted our cloud solution to provide rich team collaboration, so that groups in the US and offshore sites could work more effectively together and leverage the various time zones. To accommodate the different skill levels of our developers and test engineers, we needed an easy-to-use software user interface to mask the complexity of virtual environments, and provide the ability to share IT environments in real time between developers and testers.
Heading to the Cloud
After a thorough evaluation, we decided to adopt a cloud-based development and test solution. Through our relationship with SAVVIS, we learned about Skytap and it met our criteria to create a collaborative global development process and a maintenance free cloud environment with role-based access for team members. As part of our due diligence, we needed the solution to meet a list of criteria such as ‘pay-as-you-go’ billing, scalability, and on-demand access to data centers. We also wanted a self-service cloud that would enable us to easily build and maintain a lab that would continually meet our ever changing development and testing needs. The best part about moving to the cloud was that we finally had access to a complete cloud solution that our team could be productive with immediately.
One of the first tasks we used the cloud for was a pilot project to evaluate the compatibility of Ellie Mae’s Encompass mortgage software with Windows 7. This was a priority, as we were fairly certain that some of our customers were planning to run Windows 7 when it was released, so we had to support it out of the gate. With the on-demand cloud model, we had our software test environments created in less than one day. After this, we were able to quickly execute test scenarios that would likely have been delayed 4-5 weeks if we’d waited for new hardware to come online using our standard internal process. Our quality assurance engineers learned how to use the application quickly, and can now create their own test environments instead of waiting in line or being reliant on IT. We can now deliver software more predictably and with better test coverage because we’ve eliminated manual set-up and delays due to IT provisioning requests. With the cloud, virtual machines can be provisioned and discarded as testing needs rise and diminish, depending on the size and phase of software releases.
After successfully completing our pilot project, we expanded our use of the cloud to our facilities offshore. Our teams in the US and offshore now have the ability to access the same virtual machines. This offers both teams greater visibility into each other’s activities, and allows for easy sharing of test and issue scenarios. Our development team can now push builds to share with offshore test teams in a matter of minutes instead of the 3-4 hours it used to take. Another benefit we’ve achieved by moving to the cloud is that bug reproduction times have decreased by over 80% because our development teams can spin up a snapshot of a datacenter in exactly the state the bug was found. For complex bugs, this saves hours of time trying to replicate the issue.
We also found that the cloud offers better manageability and cost control capabilities than our physical lab machines. For instance, using a web interface, I have a full view of all the data center configurations running in our account, and the ability to audit activity of our offshore team. In addition, we can set policies to reduce costs. This includes setting quotas on specific individuals or teams to manage usage and even set an ‘auto suspend’ option to suspend machines that are left running unintentionally to save costs. Overall, we’ve estimated our Total Cost of Ownership (TCO) is about 60% lower than running internal IT environment by adopting a ‘pay-by-the-hour’ utility model and ensuring we have good cost control policies enforced by the system.
Accelerating Application Delivery with the Cloud
Since our initial foray into the cloud, we’ve made these resources available to Ellie Mae’s entire application delivery team. We now use Skytap to test our full portfolio of products, from the applications we deliver to our customers’ desktops, to the applications that run in our datacenter. And, because their cloud infrastructure is based on VMware virtualization technology, we can easily deploy our applications from our cloud test environments to our SAVVIS managed production environment. That’s a big bonus when it comes to using the cloud for test and development activities; otherwise we’d have to spend additional time and resources reworking our applications for production.
Our move to the cloud has also helped us solve our communication challenges by enabling highly productive collaboration across 50 team members located offshore and in the US. Guaranteeing the delivery of high quality software in a timely manner is something our IT organization can be proud of – from our software quality team all the way up to our CTO.