Does Data Science Replace BI?

Buck’s first laptop

When a new technology comes around, there’s a temptation to think that it replaces a previous one. But most of the time, that’s not really true.

I started working with computers in the Mainframe days. We still had them when I worked for NASA. Minicomputers came out, then Personal Computers – and in large part, the Mainframes were replaced.

But not completely…they are still out there. Working. Doing things they do well. And that’s the key.

The vocabulary around data is changing. We started with “Data Banks”, moved on to “Databases”, then “Data Stores”, then “Analysis” and on to “Business Intelligence”. It seems that a new pretender to the data throne is “Data Science”. It’s cool, it’s useful, it’s techie, geeky, and hard to understand. Tailor-made for us data professionals to aspire to.

At the Data Scientist’s meeting, where they are plotting to kill the WHERE clause

But does Data Science replace Business Intelligence? No. It doesn’t even replace reports, or even queries for that matter. It  complements them. You still need to focus on really good, clean, authoritative, trust-able data in your IT systems. Quality data is the life-blood of any kind of analysis. So yes, data storage and processing techniques (like a Relational Database Management System or it’s snarky upstart COBOL Flat-Files, I mean NoSQL) are essential, more important now than ever. You simply can’t do analysis over data that you can’t trust.

And the first step in analysis is always a query. I don’t care what language you use (Yes I do, you should be using SQL and maybe some R, but that’s another blog entry) you have to be good at understanding accurate, well-performing queries.

And I hate to be the one to break it to you: we still need reports. Always will. You’ll never be out of a job if you know Excel, Power BI, Reporting Services and the like. It’s the front-line of using (I think “Operationalizing” is the new buzzword) your data.

Yes, Virginia, Business Intelligence is still a thing. If you want to explore data to find patterns and trends, then BI is your best bet.  BI should make you say “Hmmm…that’s interesting – why is that data telling me that?” Which leads to more of those questions.

And Data Mining is separate from, but complimentary to, BI. Data Mining is where we start using the statistics and predictive techniques to not just ask questions of our data, but to let it tell us what will happen.

Enter Data Science. If 1970’s AI and Data Mining had a baby, and it went to stats classes, got a Computer Science Degree, and re-re-re-wrote the scale-out technologies we used on mainframes, it would call itself Data Science. It would then need some business knowledge and experience (Domain Knowledge) and then it can tell us not only what would happen, but what we should do about it.

Learn the ways of the SELECT. Oh, and then pick up a Double-tall for Buck…he’s almost out, and the last time….just get a double tall.

So it’s like I’ve always said: Use What Works ™. Start with good base data and data hygiene – which goes all the way back to validating fields on an entry screen. Learn to query like a Ninja (aka Itzik Ben-Gan). Make a report like a pro. Make cubes and snowflakes that would be so compelling Michael Bay wants the rights to make it into a movie. And then, by all means, learn Data Science. It’s a ton of fun in here.

(Oh, and we’ve moved on. The cool kids call all of this “Advanced Analytics” now. You probably haven’t heard of it.)