Data Integration

Posted Leave a commentPosted in Uncategorized

Data Integration

an underappreciated discipline


Often, we find ourselves heavily focused on one area of Knowledge Systems development and lately it’s been Data Integration. Having just come from assisting a global agribusiness with BusinessObjects Data Services work and now enhancing our internal ABBIE technologies SSIS scripting for new augmentation sources and visualisation tool data preparation, we find some common challenges to all tools.

There are many insights we can share with this, so we will split them into parts over the next weeks, posting more as we can squeeze out the time. This post will start with the one closest to mind, security.

Invariably the security needed to “write” or “create” is not natively available, and needs to be requested more than once. This is because DBA’s often cannot test the level needed by the tool. In an ideal dev environment, all developers have all access to all systems and databases they need, so security becomes a testing consideration.

However, in many organisations, especially outsourced ones, that’s not the case, and Service Request after Service Request must be put through to get the access needed. It’s not uncommon for more than a week to pass before a developer can even start coding.

It’s doubtful the outsource partner minds lots of service requests as each one leads to someone being engaged, but the developer has now got less time and thus needs to find new shortcuts to get his or her code out in time for the test cycle. Shortcuts invariably are more expensive in the long run as they don’t conform to standards as closely, can be filled with coded logic to circumvent slower predefined transforms and often will lack documentation.

The takeaway is not so much for the BODS developer, as you will come across this when you start so it’s not something you can prevent, but for Business Analysts and planners, we suspect in most cases the security requirement for the developer does not feature high on your scheduling agenda.

May we suggest that in all future projects you change that, and you remove a major impediment to a quality, on time result?