Assessing Organizational Readiness
Managing data science teams requires a skillset very different from what may be required when leading other teams in the technology industry. This is because, unlike other areas, the scientific process that drives artificial intelligence (AI) innovation can introduce a whole new level of uncertainty and perceived chaos, which very few organizations are prepared to manage. Project managers, product owners, or Scrum masters who have many years of experience working with software developers are being asked to quickly shift their mindsets when it comes to managing AI projects. They must refocus their priorities and adapt to a new way of approaching their roles.
Assessing Organizational Readiness
Before initiating any AI project, however, it’s important that the data science manager assess an organization’s readiness to incorporate AI into daily practices. A recent McKinsey survey found that while AI adoption is steadily increasing, few companies are implementing “the foundational practices needed to generate value at scale.” According to the survey, thirty percent of organizations that were not classified as AI high performers did not have an AI strategy aligned with a corporate strategy. A data science manager and his/her team therefore, must be supported by an organization that implements a commitment to core AI practices. The data science manager should determine this by asking:
- Are senior leaders committed to AI initiatives?
- Are business leaders educated in the potential and limitations of AI?
- Have the problems to be solved and questions to be answered been clearly stated with success measures and acceptable margins of error?
- Is data easily accessible to the data science team?
- Does the organization have the internal talent required to execute AI work?
- Does the organization have external partnerships to complement internal teams?
- Do business units trust insights generated by AI models?
Identifying and removing barriers will be paramount in order to lead a high-performing data science team that is set up for success.
The Evolving Role of Managers
Artificial Intelligence is disrupting just about every industry known to man. It is changing the way we work, and how managers manage. Traditionally, most managers’ primary responsibility has been on execution, turning a vision into reality through leadership and example. Today, data science managers’ main responsibility is innovation, creating a better vision through leadership and experimentation. The following are some of the ways data science managers can accomplish this:
Break Organizational Barriers
To drive AI innovation, managers should put their Scrum master skills to good use and eliminate any impediments their teams may encounter. When managing data science teams, strategic planning, combined with good old-fashioned persuading, negotiating, collaboration and persistence will be needed to break organizational barriers and pave the way for innovation.
Build Bridges with Business Units
Since AI is an emerging technology, business units are usually not equipped to understand its full potential. Data science managers need to keep in constant communication with business units to understand their pain points, unique domain knowledge and strategic goals to identify potential AI opportunities across the organization.
Build Bridges with Other Technical Units
Some organizations report that lack of defined integration processes can cause miscommunication between data science teams and software engineering teams who are tasked with integrating an R or Python model into an existing .Net or Java application. Bridging the gap between research and implementation is a process in and of itself and data science managers will be at the center of it.
Choose the Right Problems to Work On
Assessing AI opportunities within the organization based on available data and resources and potential value is critical for data science leaders. Data science teams have extensive technical knowledge but may not have the domain knowledge and business acumen to identify the most impactful problems to work on.
Manage Uncertainty
Managing uncertainty in data science projects often takes on a life of its own. Experiments not only sometimes fail but often fail. Managers need to quantify the acceptable margin of error for any given model (e.g. an 80% accurate model for churn prediction may be acceptable while an 80% accurate model for diagnosing cancer may not be.)
Extract Understandable Insights
Data scientists can apply complex mathematical algorithms and create sophisticated models to extract valuable insights for an organization. Managers must ensure the insights are clearly communicated in plain English and can be understood by all business users. They must make sure it is “not all Greek“ to key stakeholders.
Implementing an Innovation Culture
In a very insightful HBR article, Gary Pisano summarizes the behaviors that must be put in place in order to create and sustain innovative cultures. Promoting creativity and experimentation does not imply lowering expectations on performance, since innovators will always be held to a higher standard. Leading data science teams requires cementing an innovation culture in the organization. According to Pisano, here are some of the characteristics required of an innovator:
- “Willingness to Experiment but highly disciplined”- Experiments are well-thought out, planned, and follow experimental design best practices.
- “Tolerance for failure but no tolerance for incompetence”- Experiments fail more often than not. Failure due to lack of hard work and conducting due diligence processes should not be tolerated in any AI team.
- “Collaboration but with individual accountability”- As each team member needs to take ownership and responsibility for the quality of his/her work, the results of true collaboration will be maximized.
- “Psychologically safe but brutally candid”- Innovation is only cultivated in an open environment where ideas can be safely shared, and criticism delivered candidly and respectfully.
- “Flat but strong leadership”- Doing away with structured hierarchies helps remove unconscious emotional or mental constraints that hinder openly sharing ideas. But every team will need a single leader to set goals, give direction and be empowered to make decisions.
- These behaviors may seem contradictory in some cases so leaders driving AI innovation must constantly make the right judgement calls to create a balance of creativity, openness, experimentation and discipline, accountability and structure.
Identifying and Retaining Talent
If you do a Google search for “Best Jobs of 2019”, chances are that data scientist, software developer and statistician will rank in the top five. Data scientists are in extremely high demand and managers need to identify, hire and retain qualified talent. Yet, Google, Facebook and now Walmart (which hired 1,500 data scientists last year) may be shrinking the pool, luring away new graduates and promising talent.
But despite this, not all data scientists need to be PhD level. To build a well-balanced team you should identify and provide opportunities for your internal resources to up-skill and engage in lifelong learning. Identify quick learners, driven and hungry critical thinkers who can perform just as well if not better than other formally educated data scientists.
The most crucial and important role of a data science manager is retaining talent. Given all the AI hype, data scientists expect to have real impact on organizations and the world. If your organization poses barriers and does not live up to expectations, your data scientists will look for bigger and better jobs.
Here at Wovenware, we strive to keep data science teams challenged and motivated. Our thinking is that the best way to keep a data scientist engaged and motivated is by always providing new questions to answer and new problems to solve that result in extracting business insights and expanding our intellectual property. Data scientists are passionate and curious with heightened critical thinking skills. The curious mind needs to be constantly fed with new unanswered questions. Otherwise, they will get bored and may go out to search for new opportunities.
Identifying External Talent
To offset the challenges of hiring and retaining data scientists, managers should reach out and create partnerships with external providers and build an extended team. Many organizations are turning to AI outsourcing and nearshoring as a cost-effective and faster approach to initiating AI projects.
Autonomy in Managing Data Science Teams
Successfully leading data science teams requires empowering every individual with autonomy and responsibility, encouraging the team to experiment in the pursuit of continuous improvement and embracing lifelong learning. These are the foundations of building an innovative culture in a data science team. In a recent Ted Talk, former Marine Corps lieutenant Drew Humphreys shares a very thoughtful and surprising connection between the way we implement machine learning models and the way leaders should be agents of innovation. Humphreys invites everyone to “think differently about leadership…and empower people, the way we empower machines.”
Driving AI innovation and managing data science teams requires shifting the focus to empowering talent, embracing uncertainty and finding new ways of communicating. Now more than ever, managers have become facilitators for unlocking the creativity that will generate insight-driven transformation in organizations.
Sign up for our monthly newsletter:
