There's one skill that prevents data science from certain failure! 📉
Data science has too many possible solutions, in the end, you need domain experts to guide you through.
Whether it's checking against year-long expertise or making sure the application is interesting.
🔃 Don't build something no one needs!
The first and biggest failure is building an application no one wants. However, getting input from SMEs is the easiest way to prevent this.
Any project should start with a conversation or some reading not with code.
🧠 Read up and listen
Most experts have worked with the data for years. You gain intuitions and shortcuts by doing that work.
These intuitions and shortcuts will always help a data scientist but you need to read and listen to what domain experts put out!
🔙 Return the favour
Communication does not have to be a one-way street.
When you build a first prototype, use data viz and explainable machine learning to update the experts and ask for their opinion.
My faves:
- Simple models
- Correlation matrix
- Feature Importances
⁉️ Address Concerns
Experts will usually have concerns about your project. Never feel annoyed at these, they're the most honest expression of experience you can encounter
Always address them thoroughly and in earnest.
I talked about this at pydata
Conclusion
Avoid immediate failure in data science projects and:
- Listen to experts
- Use data viz and XAI to communicate
- Build baselines to compare against
- Address concerns, it's pure expertise