Friday, 2 November 2012

How Much Is Enough? (Data conversions that is...)

If you do the Twitter thing, follow me at @hotpixUK

After being asked this question for the third time in the last week by different organisations moving HMS, I felt moved to write a post about it. A true potential minefield in a project. An area that sometimes drastically over runs, struggles to produce even a single clean conversion until the eleventh hour or is not given the emphasis required and causes a disaster further down the line. If you are doing one at the moment or ever been savaged by a data conversion, read on....

I have been involved in many implementations and conversions. Some from the contract side, many at the coal face. For me the magic number is four. To the inexperienced it might appear that just one conversion at go-live should be needed. Why should you need more, wont that just push costs up? Won’t we just extract exactly what we need to transfer and import it smoothly into our new system? It’s just data, how could that go wrong? These points explored here are just as valid if converting and implementing a single application like HR or a big bang of a multi-module Housing Management System. The latter of which, I have experienced over a dozen, many first hand.

Generally it will go wrong and many elements of tailoring will be needed along the way. Audit checks to ensure everything needed is being extracted from the current system (and nothing extra – like those car-ports from your old Mayfair system in 1992). By audit checks I mean reconciliations to the penny (yes the penny!), number of items, count of live contacts, works orders, whatever. Fail to have that in your plan and you plan to fail. Transformations will then be needed to use new codes or match new formats. Maybe (very wide, I kid you not!) fixed length ones will be needed, certainly different table import formats. Same audit checks on those numbers need to occur on the new system when the data lands there. Every risk or issue to be conscientiously recorded. Miss this step, fudge or hide any issues and they will bit you and your project team later, bigger and badder.


In successful projects, the first conversion in my experience should be completed within the first 3 months (or third of the project). Possibly 65% or more of all data should be successfully converted and be good enough for preliminary project champion or stakeholder training, setup and other activities. The whole process should be built to be repeatable from the very start. This is a very important point. A hand-crafted conversion that takes 20 days, could not be of any use on a go-live, unless we are happy for a few hundred staff to have no systems for 20 days.

This first conversion should product a rich and valuable list of information, issues and risks that should be aimed to be pretty much mopped up by the second conversion. At the second, 100% of data or close to it should go through this one, with the usual reconciliation's to the penny and the number of items etc. Multiple databases obviously needed for these purposes, pointing to possibly 4 times or more of the estimated live working disk/database quota needed to conveniently support these functions. A virtualised environment makes this much easier these days, providing a bit of planning ahead has already been done. It’s ok to not have a 100% audit signoff, providing the mis-matched elements can be explained. Don’t be tempted to hide or bury these, they will come back to haunt you if not tackled.

The third conversion should be close to six weeks or so of the planned go-live and works best if treated like a serious dummy go-live. Timings should be collected, and if successful should validate that from a conversion viewpoint at least, that the go-live conversion will be ready to roll. 
If the outcome indicates anything other than that, resist the temptation to bury the bad news and press on. Honesty is the best policy, honest! Be prepared to inform management that you are not ready, the go-live date will need to be provisionally moved until a successful third conversion can be done and 100% reconciled. Your supplier may not be happy as they might not have a slot for you, if your project moves 6 weeks back. Most will do their best to accommodate in my experience, but don’t be surprised if your project will be delayed for a few months rather than a few weeks.   

It is very likely that at initial proposal stages of a contract, a supplier will include some implementation days allocated to data conversion. Note though generally the onus is on the customer to extract the data and provide in specific formats for provided loaders. Do not underestimate the size of this task. Specific or specialist tools, skills or software may be needed. Most systems come with standard reports that can aid reconciliation, once data is loaded. Some suppliers will offer additional chargeable days to take on more of the extraction process, depending on their experience of source systems where data is coming from. Some customers will have an in-house IT team that understands the original data or a ‘tame IT bod’ they have come in on a regular basis, who might be good candidates for extracting data. These approaches can have different outcomes dependant on personnel, culture, depth of existing system data structure knowledge etc.

Whoever extracts/converts the data it is the responsibility of both supplier and customer to agree the reconciliation of the data, pulling out and imported in. Ultimately, the sign-off will be down to the customer, as a final user acceptance. Personally I like to see a proper signature, a bit old school I know, but reassuring, that everyone understands the gravity of what is being signed off. Only certain data issues then generally fall under the suppliers ownership. For example if perfectly converted data becomes corrupted later by the new system. Missing or unconverted data found later, after signoff (and any associated costs etc) is generally the responsibility of the customer.

The data sign off milestone is crucially important for customer and supplier. It protects both and they give it low priority at their peril. Both need to take appropriate responsibility. An article in Inside Housing magazine in August 2012 ‘A Costly IT Error’ , was pointed out to me recently, where shortly after implementing a shiny new housing system, many thousands of residents were under billed by £100,000 after a rent review. For once a little windfall for residents!

That InsideHousing article (see http://www.insidehousing.co.uk/accent-in-costly-it-error/6523173.article ) was quite vague as to the real issues of how the data got in that state, it’s certainly not you might be looking for in your project, a few months after go-live. It does however underline the importance of the robustness of data conversion, audit, reconciliation and sign-off. I have always stood my ground and insisted in all of my projects to see that signoff signature in blood. It saves everyone’s bottoms later and goes as far as feasible to reduce the chance of things like the example above.

Read on, to how communication with your residents can break down:
http://tonysmiththathousingitguy.blogspot.com/2014/07/communication-breakdown.html

The Fixx - How much is enough?.


(c) Tony Smith, Acutance Consulting www.acutanceconsulting.co.uk

File Under: 360,1stTouch,4Js,Aareon,Academy,ActiveH,Alignment,ALMO,Anite,Apex,ArchHouse,Archouse,asbestos,Asprey e-state pro,Asset Management,Aurora,Average IT Costs,BO,BPR,Browser Applications,Business Objects,Business Process Review,Business social networking,Castle,CBL,Cedar Open Accounts,Change,Cheaper Housing IT,Chics,
CHR,Citrix,Civica,Clearview,CMS Template,COA,Coactiva,Codeman,Comino,Competitive Dialogue process,complex IT procurements,Component Accounting,Consilium,Consolidation,Consultancy,Consultant,Contact Manager,Context, MD , Director , Housing software, UK, United Kingdom,Contractor Systems,CORE,CorVu,Cost Reductions,Covalent,CRM,Crystal Reports,CTI,CTX,Customer Relationship Management,Deeplake,Development Systems,Document Management,Documotive,ECMK,EDRMS,England, English,EnterpriseBI,ERP Systems,Finance,Financial Systems,Getting best from,Grasp,Grip,hardware,HG,Housemark survey,Housing Blueprint,Housing Group,Impact Response,implementations,In House,In4,Infoflow,Information Technology,Informix,inHouse,in-house workforce, HousingIT, Innovation,Inside Housing,Internet Portal,Invu,Ireland,Irish,IT Budget,IT Training,iWorld,Keylogic,Keystone,Kirona,Kypera,management reporting,Measuring Change,Mebus,Miracle,MISCS, mobile functionality,Monopoly board images and pictures, Montal,MS Dynamics,MS Dynamics CRM2011,Neighbourhoods and Communities,Northgate,Notice,NROSH,OA,OGC Buying Solutions,Ohms,OJEU Limits,OmFax,Omniledger,Open source software,open tender,OpenHousing,Opti-Time,Oracle,Orchard,Outsource and outsourcing,Paloma,performance management systems,PfH,Pick,PIMMS,pimms4communities,Planned maintenance,Plus,PM,Progress,Promaster,Proval,Pyramid, QL,QLX,Registered Providers,Registered Social Landlord,Rent Increase Freezes,Repairfinder,Reporting,Reports,ROCC, NINTEX , NINTEX workflow , Rocket,RPs,RSL,Saffron,SAP,Scots, Scottish,Scotland,SDM,sector,Server Virtualisation,servicing, Universalcredit, universal credit, universal credits, Servitor,Servitor,Sharepoint,Simdel,Simdell,Slash and Burn,Social Media,Software,SQL Open Housing,SQL Reporting Services,SQL reporting services,SQL Server,Star rating,Stores and Stock,Strategic Asset Management,Sunguard ,Surveys,Sx3,System alignment, Systemwise, Factorwise, Blockwise, Block wise, Task,Terminal Services,Three Star,Today,Total repairs,Tribal,Twitter,U2, 07854-655009 , 07854655009 , 07854 655009 , "07854 655009" , tonysmith , tonysmiththat , thathousing , thathousingIT , thathousingITguy ,UniClass Enterprise,Unidata,Universal Housing,Universe,Unrest,Van Stock,Voice and data,Wales,Welsh,Windows Server,Workflow and tasking,Xmbrace,XML

1 comment:

  1. The major factors for this kind is Data Conversion services having more features and aspects like in the field of more benefits for having profit and quality work.
    Data Conversion Companies | Web Research Services

    ReplyDelete