or LinkedIn here https://uk.linkedin.com/in/tonysmiththathousingitguy
During my first few years working with Social Housing Systems, I was involved in a lot of full integrated solution implementations. I still surprise suppliers when pulling them up on implementation and data extract/transformation processes. You soon learn how to locate trouble-some and hidden data and use tried and tested approaches to clean it up and present it in the best form possible, for a brand new system.
Strangely, lately I have been involved in a raft of data conversion work, for Housing Groups merging systems or new properties in, and for a few organisations, taking their first housing solution.
That all covers the relatively structured stuff. Even Excel spreadsheets have some structure, however possibly haphazard. I am used to telephone numbers, containing comments, like (work) or (Daughter – Mon-Thu only) etc. Notes and scratch pads are another matter. Free text is of course notorious and using conventional database languages, a bugger to report from. I often come across sensitive data in text and note fields. Historic account references, person details, credit card data, sort codes and others. Some of it innocently blows PCI compliance right out of the window.
The above of course refer to database info. IE in SQL Server, Progress, Oracle, Ingress or Informix etc. Every organisation also has Terabytes of data held on local, network and cloud drives, as well as local Exchange Server storage. Once you start looking, unfortunately you start to see sensitive data in the strangest of places. Fascinating that all this is, within the recent GDPR rules, we should be managing every single byte of sensitive data that we own. Much of it is historic, and a lot down to poor user training and poorly managed habits.
|Innocent recording of card/payment details in free text notes are a major Achiles Heal for GDPR compliance|
Searching just a small corner of your unstructured data, can take several days and then you only just scratch the surface. You would need a small army (well maybe a platoon), to do a proper job, then keep on top of it too. Data quality groups are something I always promote, wherever I can, but they have a limit to their vigilance. For a few years I have been actively seeking some more intelligent solutions, to help crack this one.
Last month I think I spotted a pretty comprehensive, proven tool to do the job, called InfoBoss. You don’t need an army of people either, to feed and water it. Just a few pioneering Social Housing landlords are currently using it, in addition to at least one credit rating agency in the Midlands.
It pretty much sniffs out the problems by itself with a set of inbuilt rules, that can be activated to detect data in the wrong place, suspect data problems and breaches of GDPR policy, such as ‘Right To Be Forgotten’. Integral white and black lists enable checking for what should have been long since archived ID’s, that potentially are breaking the law.
The tool has been built on Elastic Search, the platform is scalable, cloud based and quick to setup and very easy to use. Once it has been pointed at a multitude of data sources, within an organisation, it will collate details into a single reportable repository. A messaging API allows set up of automation. Stakeholders can be created that are automatically alerted on new and newly discovered exceptions.
I’m techie, but it’s clear this is more of an end-user, than an ICT team tool, all in a private cloud too. Check it out by contacting Mark Hobart of InfoBoss or download the InfoBoss data management whitepaper.
|Download the InfoBoss white paper here|
If you have or suspect you may have data quality issues that possibly contravene GDPR, this is clearly a new tool worth a look.
Do mention you read it first on this blog and please let me know what you thought of it too.
|Catch the blog monthly archive at
I would be pleased to connect with you on LinkedIn - http://uk.linkedin.com/in/tonysmiththathousingitguy Message me with any issues or queries, you would like to be explored in this blog. We generally receive a couple of suggestions each month.
The Shamen – Boss Drum.
(c) Tony Smith, Acutance Consulting www.acutanceconsulting.co.uk 07854-655009
Access a quick list of our Social Housing ICT blog posts here
Could we help you or your organisation? Our contact details are here , get in touch we will be pleased to chat about your problems and help with your organisation issues.
File Under: 360,1stTouch,4Js. , 07854655009 ,365Agile," """" ",,#UKHousing,#HousingDAY, #HomesForBritain,#InternetOfTenants, #Shout,1st Touch,3squared,Accuserv,Acutence,Aareon,Academy,ActiveH,AirWatch,Alignment,ALMO,Anite,Apex,ArchHouse,Archouse,asbestos, Asprey e-state pro,ActiveHousing,Asset Management,Aurora,Average IT Costs,App,Associates,ACL,Abritas,AMS,AX,aspireview,Agile365, BO,BPR,Browser Applications,Business Objects,Business Planning, Business Process Review,Business social networking, Blockwise, Block wise,BI,BancTec,BluTek,Bluebox, Castle,CBL,Cedar Open Accounts,Cx,Change,Cheaper Housing IT,Chics, CHR,Citrix,Civica,Clearview,CMS , CCS IT Keystone CCSIT,Civica Genero,Contractor Systems,CORE,CorVu,Cost Reductions,Covalent,CRM,Crystal Reports,CTI,CTX,Customer Relationship Management,Cashflow, COA,Coactiva,Codeman,Comino,Competitive Dialogue process,complex IT procurements,Component Accounting, Community Reward Services, Consilium, Consolidation, Consultancy,Consultant,Contact Manager,Context,CIH,Chartered Institute Of Housing,Cadcorp,Cerrus Saturn,convergeOne,CACI,CPL, CRS Director, Deeplake,Development Systems,Accuserve,Document Management,Documotive,datasystems , Microsoft dynamics GP 2013, DynamicAI, Dynamic AI, ECMK,EDRMS,England, English,EnterpriseBI,ERP Systems,ERP,Elmhurst,Estatecraft,EDM,ESRI,Exponential-e, Finance, Financial Systems, Financials, Factorwise,Facebook ,Footprint,Fusion, Getting best from,gotonysmith, GasTag, Grasp,Grip,GeoSolveIT,GIS,GGP,Genero,GCloud,G-Cloud,HouSys,Housing software,HFI (Housing Financials interface),HG,Housemark survey,Housing Blueprint,Housemark,Housing Portfolio Management System, HRA,housingtITguy,Housing Support Pro,HousingSupportPro,Housing Options,Hometeam, Housing Group,hotpixuk, HousingIT,hotpix,Hardware,Hitex,Housing Contact Company,housing predictive analytics, Impact Response,iPayPro,implementations,iRobot,In House,InHouse,In4,InMotion, InMotion2016, InMotion2015, InMotion2016 , Exhibition, conference,Housing Management Consultant,Housing Management Consultants, InfoBoss, Infoflow,Information Technology,Informix,inHouse,in-house workforce,Information@Work components, Innovation,Inside Housing,Internet Portal,Internetalia,Information Management Server,IMS, Invu,Ireland,Irish,Inphase , IT Budget,IT Training,iWorld, Innovation Group Apex,ITIL, In4Systems Promaster,IT,Insight, iSupport,iSupportPro, Keylogic,Keystone,Kirona,Kypera,Keypera Hometeam,Keyfax,Ledgers,Linkedin,Locality,Liquid Voice,MD ,Mr Void , MrVoid, Measuring Change,Mebus,Miracle,MISCS,mDesign,mobile functionality,mobile platform,Monopoly board images and pictures, Montal,MS Dynamics,MS Dynamics CRM2011,Miracle Software,MDM,Management,CRM2013,Reporting,MI,M3,MIS,MIS-AMS,Mobysoft, Northgate Codeman,Neighbourhoods and Communities,NPS,NPS Housing,NPS CRM,NPS Job Manager Mobile, Northgate,Notice,NROSH,NINTEX,NINTEX workflow,Natural Forms,Nintexworkflow,NPS ASSIST, OA,OGC Buying Solutions,Ohms,OJEU Limits,OmFax,Omniledger,Open source software,open tender,OpenHousing,Opti-time,Oracle,Orchard,Optitime DRS,Outsource and outsourcing,OpenContractor,ORS,OneServe, PIMMS Data Systems, Paloma,performance management systems,PfH, Pick,PIMMS,pimms4communities, Planned maintenance,People Value,Plus, Peoplevalue,Portfolio Management System, PM,Progress,Promaster,Proval,Pyramid,PRINCE2,prince,Pamwin,PlanForm, Propoints, Pro-Points,predictive analytics for housing, QL,QLX,Quiss,Qlikview, QueryView,Queryview reporting,Registered Providers,Registered Social Landlord,Rent Increase Freezes,Repairfinder,Reporting,Reports,ROCC,RobotAutomation,Rave Technologies (India) PVT Limited, Rocket,RPs,RSL,Reality,ReAct,reidmark,RM865, Saffron,SAP,Scots, "Social Housing IT",Social Housing ICT, Scottish,Scenario, Scotland,SDM,sector,TonySmithHou,Safron,Server Virtualisation,servicing, Servitor,Sharepoint, Simdel,Simdell,SimPro,Sim Pro,Slash and Burn,Social Media,Software,SQL Open Housing,SQL Reporting Services,SQL reporting services,SQL Server,Star rating,Stores and Stock,Strategic Asset Management,Sunguard ,Surveys,Sx3,System alignment,Symatrix Human Capital Management, TonySmithHousing, Systemwise,SM,Sequoa,Serros,SP,Scout,score,Swordfish,Serengeti,SOTI,southview,Sunguard,Service Charges,Spotlight Service, Spotlightservice, SunAccounts, Social Housing Software Applications, Template,Task,Terminal Services, tonys , tonysm , tonysmi , tonysmith, tonysmitht, gotonysmith, tableau Business Dashboards, tonysmithth, tonysmiththa, tonysmiththat, tonysmiththath, tonysmiththatho, tonysmiththathou, tonysmiththathous, tonysmiththathousi, tonysmiththathousin, tonysmiththathousing,Tony Smith That,Tony Smith That Housing IT Guy,TonySmith that housing IT, guy,tonysmiththathousing, tonysmiththathousinguy,tony smith housing guy,sharepoint, share point, SHBVN, Three Star,Today,Total repairs,Tribal,Twitter,U2, tonysmith , tonysmiththat , thathousing , thathousingIT , thathousingITguy, TED,Tagish Casework,TP Tracker,Telecetera,Tagra,Trace, Totalmobile, Total Mobile, UniClass Enterprise,Unidata,Universal Housing,Universalcredit, universal credit, universal credits,UC,Universe,Unrest, UK, United Kingdom ,Ukhousing,UK Housing, Van Stock,Voice and data,Vantage Sentinel,verse one,verseone,vWork,Version One,Visualmetrics,Valueworks,Voluntas, Wales,Welsh,Windows Server,Workflow and tasking,Wheatley,Works Connect,worksConnect, Xmbrace,XML ,XenApp,Xen App ...