NTL International is a leading player in the world telecommunications market and a multi billion dollar corporation. Project Harmony is a bold, highly visible internal project to unify all of NTLs 29+ legacy systems into one coherent, easy to use platform and is the largest internal software implementation ever undertaken in the company's history.
In a spokespersons words “it’s a bet-the-company project”.
There are more than 15 functionally different databases to build and deploy, some with 800 tables, 3000 stored procedures and storage requirements in the terabytes region (as you would expect in a telco). Add to this the fact that over 70 staff can make changes to the database code and that 20+ testing/training/trialing environments exist and you have the potential for things to go drastically wrong.
All in all the management of a diverse set of databases, huge data volumes, multiple environments and thousands of changes presented a significant change management challenge to the project team. With this in mind it was clear that an automated, repeatable approach was needed to bring the databases change process up to the same level of process quality that the other components in the system enjoyed. What follows is a statement from the senior management team at NTL who are tasked with delivering the Harmony project within tight timescales and on budget…
How the DB Ghost Process helped our developers
After our developers started to use DB Ghost to verify their changes before checking them into the Source Management System we immediately noticed an amazing increase in quality and almost zero build breaks. To help reinforce the process we have dedicated a build server to DB Ghost and build our databases from scratch and run a test upgrade every hour of every day. If any errors are found our DBAs and Configuration Management team are notified immediately and take remedial action whilst the changes are still fresh in the developer’s mind.
It also means that developers are only responsible for changing the code. How that code is deployed is no longer an issue that they need to deal with. Having worked with the DB Ghost Process for nearly a year now, our developers never want to go back to the bad old days of coding mundane scripts or hand picking which updates to apply from the output of a diff tool.
How the DB Ghost Process helped our configuration, deployment and DBA teams
Keeping track of scripts and verifying deployments used to be a nightmare! It has become so easy that, at first, our deployment team doubted the target database could actually be correct!
Typically, deployment used to involve many experts carefully examining scripts and applying the changes to the databases, this is now no longer the case. Databases can be built or upgraded from scratch within minutes, not days. With version 3.0 we can now capture an SQL script of the upgrade that can perform, via Query Analyzer, the same changes in seconds.
Our DBAs now produce code and schema audit reports for senior management by simply examining the changes between labels/baselines in the Source Management System. Now that they are off the critical path for the day-to-day change propagation they schedule their time much better and can really get stuck into the changes without being interrupted every five minutes by a developer needing to push their changes out. Initially there was some natural resistance to DB Ghost from the DBAs as nobody likes ‘change’, but we were sure it was the way to go and now, having seen such a dramatic improvement in the stability of the databases, they haven’t looked back. Coupled with the amazing support that we’ve had from the guys at Innovartis, we’re thrilled to bits with the product.
Using DB Ghost has made the whole task simple, repeatable and rock solid.
How the DB Ghost Process helped our test team
We used to spend vast amounts of time raising defects due to database build and deployment errors that had to be chased around the various departments before anyone took responsibility. The developers told us it was a deployment issue; the deployment people said a DBA needed to fix it and the DBA said that it was the developer’s problem!
With the DB Ghost Process the above problems have disappeared as everyone is now 100% sure of the schema level and it can be traced right back to the source repository. Any functional issues we raise are also resolved in a much shorter timescale as the developers know exactly where to start from with the database code. In fact, our only errors now are performance related or functional. Also, due to the low cost of deployment and the fast turn around, errors are now quickly redeployed and re-tested by the test team.
The team responsible for the migration of the data from legacy systems now has the ability to test at any time on the new schema. This means there are no surprises for them when the release date arrives.
When we first looked at DB Ghost we were skeptical to say the least as the problems of controlling changes to database code have been around for so long. Our thought train was "surely every avenue has been explored to solve these problems, what more can this tool possibly offer?"
However, as we're open minded we gave it a chance and not only has it paid back our investment many times over I can't imagine ever working any other way now.
Tim Carman, Director of Software Architecture, Technology Division, firstname.lastname@example.org