When a new technology comes around, there’s a temptation to think that it replaces a previous one. But most of the time, that’s not really true.
I started working with computers in the Mainframe days. We still had them when I worked for NASA. Minicomputers came out, then Personal Computers – and in large part, the Mainframes were replaced.
But not completely…they are still out there. Working. Doing things they do well. And that’s the key.
The vocabulary around data is changing. We started with “Data Banks”, moved on to “Databases”, then “Data Stores”, then “Analysis” and on to “Business Intelligence”. It seems that a new pretender to the data throne is “Data Science”. It’s cool, it’s useful, it’s techie, geeky, and hard to understand. Tailor-made for us data professionals to aspire to.
But does Data Science replace Business Intelligence? No. It doesn’t even replace reports, or even queries for that matter. It complements them. You still need to focus on really good, clean, authoritative, trust-able data in your IT systems. Quality data is the life-blood of any kind of analysis. So yes, data storage and processing techniques (like a Relational Database Management System or it’s snarky upstart COBOL Flat-Files, I mean NoSQL) are essential, more important now than ever. You simply can’t do analysis over data that you can’t trust.
And the first step in analysis is always a query. I don’t care what language you use (Yes I do, you should be using SQL and maybe some R, but that’s another blog entry) you have to be good at understanding accurate, well-performing queries.
And I hate to be the one to break it to you: we still need reports. Always will. You’ll never be out of a job if you know Excel, Power BI, Reporting Services and the like. It’s the front-line of using (I think “Operationalizing” is the new buzzword) your data.
Yes, Virginia, Business Intelligence is still a thing. If you want to explore data to find patterns and trends, then BI is your best bet. BI should make you say “Hmmm…that’s interesting – why is that data telling me that?” Which leads to more of those questions.
And Data Mining is separate from, but complimentary to, BI. Data Mining is where we start using the statistics and predictive techniques to not just ask questions of our data, but to let it tell us what will happen.
Enter Data Science. If 1970’s AI and Data Mining had a baby, and it went to stats classes, got a Computer Science Degree, and re-re-re-wrote the scale-out technologies we used on mainframes, it would call itself Data Science. It would then need some business knowledge and experience (Domain Knowledge) and then it can tell us not only what would happen, but what we should do about it.
So it’s like I’ve always said: Use What Works ™. Start with good base data and data hygiene – which goes all the way back to validating fields on an entry screen. Learn to query like a Ninja (aka Itzik Ben-Gan). Make a report like a pro. Make cubes and snowflakes that would be so compelling Michael Bay wants the rights to make it into a movie. And then, by all means, learn Data Science. It’s a ton of fun in here.
(Oh, and we’ve moved on. The cool kids call all of this “Advanced Analytics” now. You probably haven’t heard of it.)
8 thoughts on “Does Data Science Replace BI?”
Fantastic post, I could not agree more.
Still laughing about “COBOL flat files”/NoSQL…
LikeLiked by 2 people
Yep, as an old COBOL programmer, the comment was LOL funny, luckily a bit short of “spit-take”, I have new 4K monitors to protect!
There is a lot of truth behind the comment. A few years ago I was dealing with a CRM design where polymorphism would have been nice. You know, a customer may also be a supplier, and in a SQL Server database it can be messy. While exploring a “supertyping” solution, all I could think of was how much easier it would have been in COBOL with “occurs depending on” and “redefines” giving you a cohesive “document”.
“You’ll never be out of a job if you know Excel, Power BI, Reporting Services and the like.”
Yeah, but would I ever like my job?
Call me an idealist, but I want to work in order to use information to enable more effective decision making. These tools do constitute a great deal of the front lines; however, I think Data Science is asking for more folks to approach the “Always will” with healthy skepticism, curiosity, and storytelling. There certainly is a “next” step change in our understanding of information for decision making — settling (career-wise) in this field seems like the least helpful advice.
John Rauser covers the topic nicely: https://www.youtube.com/watch?v=0tuEEnL61HM&t=11s
Hello Bill – you’re exactly right. We need to aspire to the next levels of data analysis to be sure. Excel, R, SQL – these are important foundational steps, all part of the toolkit to do the prep-work, exploration and other steps needed prior to deeper analysis with things like Machine Learning and the like. And one should never settle – agree there – and it is SO important to get those foundations right. I can’t tell you how many models I’ve helped customers evaluate that turned out to be based on the wrong data – something that could have been avoided with a quick summary() in R. 🙂
Thanks for reading, and for commenting!