Handyman vs Contractor

You’ve got a home project to do this summer and you need to hire; would you hire a handyman (or, more inclusively, a handyperson) or a contractor? If you had to choose one, which one would you pick? The short answer is: it depends! If you are interested in my long answer, read on.

How does this relate to careers in Data Science? Imagine you have recently hired a Machine Learning code ninja who can implement the latest pose-estimation AI model into your live stream videos and crush it, but cannot tell the difference between mean and median in his data. Is that OK? Should you be worried about the future a little? Before answering this question, let’s pause for a moment here and think about the big picture. To make my point, I would like to tell you a little story: an interesting metaphor I heard for the first time from my Cosmology professor, Prof. Ram Cowsik, when I was a young grad student at WashU.

Vertical Ladder vs Horizontal Ladder

In one of his classes Ram talked about the history of Cosmology and I clearly remember he beautifully pointed out a fundamental difference between the nature of different branches of knowledge.

He used the term “Vertical Ladder” as a metaphor for the branches of knowledge such as Mathematics, Physics, and Biology to some extent. And here is what he meant by that: in order to reach step #10 of the ladder you need to climb from the first step all the way up to step 9. You cannot skip steps when the latter is in a vertical position!

However, in some other branches of knowledge such as History or Geography you don’t necessarily have to go through step 1 to 9 to reach 10. You can just walk straight to step 10 and you will be just fine. These are examples of the cases that Prof. Cowsik was calling the “Horizontal Ladder” models. You can reach any step of the ladder anytime you want when the ladder is in a horizontal position.

What About Data Science?

Now let’s go back to our main branch of thought: what about Data Science? Is it a Vertical Ladder or a Horizontal Ladder? Can you reach step 10 without climbing step 1 to 9? Can you implement advanced AI models without necessarily knowing the Math and Statistics behind the Machine Learning algorithms?

The short answer is: yes, you can! It is a Horizontal Ladder. You don’t need to know anything about how the Random Forest algorithm works in order to build, test, and implement a Classification AI model. However, you should keep in mind that it hasn’t always been like that.

For example, the answer would be different if you would ask me the same question 30 years ago when the Data Science tools were not mature enough yet. Data Science used to be a Vertical Ladder (like any other quantitative field of study), not much anymore! Thanks to Open Source and its decent documentation and big active communities online.

Despite the earlier days of Data Science in which it was traditionally a Vertical Letter, nowadays, it is considered more of a Horizontal Ladder. Nowadays with knowing just a little bit about the generic frameworks of the ML problems and being familiar with Python libraries like Pandas and Sklearn, anybody can implement the most advanced algorithms on their data. [now you might ask: “Are data scientists becoming data operators?”; well, yes and no! That’s a good topic for another blog post though.]

Which Type of Data Scientist You Need?

Now let’s go back to the original question we asked earlier in this note: “Would you hire a handyperson or a contractor?”

A contractor is licensed in a specific task (electrician, AC repair, lawn care, etc.) He will most probably do ok on your generic home project for which he is certified. He might have even got some prior experience on that specific job.

A handyman, however, has got life experiences on things he can do well. He may be able to even connect his past experiences to come up with some creative, unique, new solutions for your custom home projects.

Summary

Would you hire a handyperson or a contractor? It really depends! It depends on what you are trying to achieve, the nature of the work, your future plans and the scope of the project. If you want to get the job done, a contractor might be the best choice. On the other hand, if you are thinking more long term and might need some custom solutions or need to build totally new products then hiring a handyman would make much more sense.

It is not very different when it comes to hiring for Data Science jobs. A true Data Scientist (handyperson) with his own ladder under his arm ready to climb pretty much anything or a Data Science expert (contractor) who can jump up to the Horizontal Ladder of your data science projects. Which type of Data Scientist would YOU hire?