Job Market Realities for Data Professionals in 2026 | Career Panel | Data Science Hangout

Transcript#

This transcript was generated automatically and may contain errors.

Hey there, welcome to the Paws at Data Science Hangout. I'm Libby Herron, and this is a recording of our weekly community call that happens every Thursday at 12pm US Eastern Time. If you are not joining us live, you miss out on the amazing chat that's going on. So find the link in the description where you can add our call to your calendar and come hang out with the most supportive, friendly, and funny data community you'll ever experience.

Today we are having a career panel. This is a little bit of a shift. We often have just one or two data science leaders here that we get to ask questions to and hang out with for an hour. And today we have three. We have Gabriela de Quiroz, Dan Wauver, and Makarand Malu. And because they all have very varied titles, hats they wear, roles they play, and institutions that they've worked at, I'm going to let them introduce themselves.

Be careful when you hear and listen what people are saying about this high individual contributor, because it's very specific to the type of company. It's not going to be applicable for every company.

AI's impact on the job market

They're fantastic. And I have one from Gregory Power that is fantastic to go next. And I'd love to direct this one to Dan first to see what Dan thinks, because when I said AI, Dan was nodding his head.

My question is, like, what are your thoughts on generative AI and kind of the whole idea of agentic workflows and we must embrace the machine spirits and their effect on the job market? I don't know. I've used AI here and there, but I don't feel good using it when I do, because, you know, read about how it affects, you know, retention and your ability to actually grow skills or debug the code that you write or write whenever you're using AI. But, yeah, that's kind of my thoughts, feelings, and question.

I mean, I have a lot of thoughts here. So, and I don't know exactly where to go. I mean, I feel like for a long time we've had, like, code complete, right? And we've had, like, whatever, spell check and stuff like that, right? So, there's a piece of this that is, like, the next level of spell check and code complete, where it's just kind of like, you know, you already know how to do it, and it's just completing what you already know. So, there's a way to see it as optimization there, right?

How it affects the job market? I'm a little skeptical right now that the big cuts of these companies are actually from implementation of AI at their companies saving money. I feel like a lot of it is, like, saving face or showing that, like, they are the AI company of the future, that, like, there's all this pressure on everyone, on every board, on every CEO to do something with AI. So, I feel like some of that is just, like, saving face on that. I haven't seen, I mean, companies move slow, much slower than the outside world, right? So, I haven't seen that a real transformative change within individual companies yet.

Yeah. My take is the change is here, and, you know, you have to adopt it. And change is always hard, right? I was talking to a friend at Amazon, and he was telling a development cycle that, just yesterday, we had a long chat, and I was asking him a lot of questions about what changed. He said the feature to development phase for him in the past was six months, reduced to one month, and team of 30 people, mostly because of AI and the recent, you know, launches by Anthropic has been really, really good, where even the prototyped UI came out pretty good.

So, change is there. Organizations are adopting to change. And on the software engineering side, you know, agentic framework is going to be the future, especially for, you know, things that are not mission critical. Like, if I have a machine learning algorithm serving on the life to millions of customers, certainly not going to depend on AI to build a model for me, and then, you know, show that model to our users. Having said so, for non-critical mission, AI is becoming a lot more of a productivity booster, and there are some companies that are adopting it pretty fast. It's real. It's here. And moving and training about how to build an agent is going to be a skill set that all of us need to build on. That's my perspective on it.

I'm thinking about the whole productivity. Yes, it's making us more productive. But, again, I'm very skeptical. I have a lot of opinions about this. I think we are going to have, well, there's a lot of AI slop happening, and it's changing. I feel like even though we don't agree sometimes, we have to follow. Otherwise, we don't get a position if we are not using those tools, right? Because everything is about being fast. I've been also playing with some of data analysis and how to use AI to help, but still, I'm not confident. I still go back to R, and then I do some of my analysis in R, and I do some comparison.

For some things that was a bottleneck for me, yes, AI is great, especially around prototyping a lot, like production apps. That's working fine, but the whole analysis, I still think there is a lot of domain knowledge that we need to have that I don't think AI is there to replace us at all.

Technical interviews in the age of AI

Abigail asked, how do technical interviews look these days for data scientists or data engineers? Are we allowing AI tools? Is there more or less coding? This is sort of a follow-on to this AI question. How has AI shifted things?

Yeah, it's getting weird. It's getting weird. I mean, a couple of things. You put a job open, you open up a job. It used to be like somebody had to back in the primitive times. You had to know that the company existed, find that they were hiring, go prepare a resume, cover letter. Now it's like on LinkedIn, it's like apply. So we get like hundreds and hundreds of resumes for any job or thousands of resumes. So there's one question there, just like how do you sift through and how do you find the right candidate in that, which is just a weird problem that I don't think is well-solved by anyone right now.

In interviews, I interviewed someone and I swear they were listening to my question with chat GPT and then just read me the answer. And that was not very impactful for me. But I feel like that's coming there. And then it becomes like an interesting philosophical question about should you be using AI in your technical interviews because it is now a tool that is available. And so if you're the leading edge of it, you're using all the tools all the time. So is that a good thing now that you're using all the AIs in your technical interviews? Or are you at laggard if you're not? So I think there's some, just we haven't caught up in some of our practices yet to where we are.

Yeah, it's because I worked with a few startups. I was hiring for them this year in particular, and we allowed them to use whatever tool they wanted. It's almost like an open book, but it doesn't mean that the coding was less. They could use any tool that they wanted, but they had to go through the thinking with us live. And then I would ask some questions where they would have time, they would not have time to go to the AI. They would have to ask me. So like there are a lot of signals that we are trying to get. Can you use these AI tools, but can you also think strategically about the questions that we are asking you? And can you guide me through your thinking process as well? So we are trying to measure two things.

One thing also, like I know that big companies, they are still doing the old school, like Google, old school interviews where you go to Google Docs and you have to do the coding exercise, which, you know, it's crazy to think like, it's still like in the old days of like coding in a blank page without any help with, you know, the linter and all the other things. And I think my current, you might be able to talk a little bit more about Google, but, you know, still like the Google Docs, they ask you a question, you have to write the coding over there without any help. I don't think they have changed that. At Microsoft, we didn't change that at all.

Yeah, no, I, you know, I'm interviewing, right, Gabriella, and what I found with the large tech companies, it's the same pattern, right? The technical screen typically is mostly SQL, at least in the tech companies, followed by a product sense kind of interview. The next rounds, you do have technical rounds where you use the preferred language, be it Python or R these days, and where you're typically asked for some programming questions. Some companies like Apple and even Google for data science role also expect understanding of algorithms and may have some medium lead code kind of algorithmic questions asked in the context of data science.

Having said so, even larger companies are changing. A friend of mine at Instacart just told me that they're now building questions around how do you manage workloads using AI within the teams, and the questions are more about how do you leverage AI for productivity, and for an IC role, it's more about how do you use AI to become more productive, and for a managerial role, it is more about how do you think about the team as a mix of AI tools and FTEs, right? So, I think companies are moving fast. They're trying to integrate AI questions. I haven't encountered a lot of interviews where I was allowed to use AI tools. Most of the time, it is frowned upon or is not looked upon in terms of allowing AI being used in an interview setting, and most companies are not ready yet, though I see that as a future.

Pivoting into a new domain

Ben had asked Dan, how would you recommend pivoting into statistical programming? He mentioned, I have a statistics degree, 10 years of R use with no clinical experience.

Yeah, I switched mics. This is a very specific question to me, because it's a field I hadn't even considered. I love R, and I'm like, wow, if the industry's moving that way, I really want to get in on it. But I think that there's so many of us, we have different backgrounds. I've been in food manufacturing for a long time, and I'm an industrial statistician. I have tons of skills, and I think we all have very different, but similar stories. It's like, how do you bridge the gap? It's like, okay, I have three things with zero experience in that domain. So just looking for your ideas, I appreciate it.

No, I appreciate the question, Benjamin. I don't know. I'm not going to give you a great answer, but one is networking, right? One is networking, people hire people that they know. That's just one thing that tends to be true. I think there are networking groups for R and pharma, networking groups for R and pharma, things like that, work your way into those, maybe join a working group or something like that, just so you're known. Then you have some experience in R and pharma. Then I'd say, I think it was smaller companies, or we have a lot of contract research organizations, they're called CROs. They hire a lot of people, they hire a lot of more junior. You wouldn't exactly be junior, because you have so much experience, but you'd kind of like first round into the industry. So I think those are kind of the best ways to get the start and then to come over to a bigger company.

But I do want to put a plug that I just love when people come across industries. We have so much of an echo chamber of just like our own stuff. And when we get people from the outside who come in, they're like, why are you doing it like this? That is just the best. It just really shakes up our industry. So I'm really super excited about stuff like that, but I do recognize that it is difficult to get going.

Building trust across departments

He says, the positions I've taken are directly in the department that data is aimed to help, so not the data department itself. As a result, I end up being met with mistrust from the main technical or data department because I'm not directly in their department. How can I get over this trust hump to get the access that I need?

I think, you know, influencing without authority is always hard, right? And what I'm hearing is what you're producing is not being trusted upon because it's a very different department. I can only provide examples of what, you know, what happened with us. When we started and set up a team, we used to, you know, get a lot of ad hoc data requests from our stakeholders. And one of the challenges I faced was around helping our stakeholders to solve their questions themselves so that my team can actually do real work, right, rather than just answering ad hoc questions. And we created a lot of interesting dashboards, but what we found was the dashboards were not getting used. Almost always people will still reach out to us directly because they didn't trust the dashboard or they didn't know how to use the dashboard, one of them.

The way for us forward was what I showed was to my VP at that point in time was the people who got promoted were the people who are using dashboard, right? And once I showed that to my VP and their directs, that became a powerful catalyst because all the VP's directs started pushing their people to use the dashboard. And, you know, that reduced the burden for my team in terms of answering ad hoc questions. So, it's an indirect way trying to understand, you know, why people don't have trust and can you do a top-down pressure, work with the leadership. In this scenario, that helped us because the leadership then got bought into, you know, why their team should be using dashboards.

Yeah, top-down is one way I can think of because trust always, you know, has something more underlying. Trust can be because, especially in this environment, there's headcount pressures. Trust can be missed because of that because the other team might be thinking about, hey, if we use this data, what does it mean in terms of headcount for our team, right? So, you need to probably understand that. And depending on that, use either a top-down approach or build relationships in those teams.

You need trust, right? And I think this is, I don't know if you've ever had like a centralized data group or a centralized transformation group or something like that in any of the companies that you're in. And the first thing is like, why are these people telling me what I already know? Or like, why do these people think they're more experts than me? Like, these are like real human emotions. The way to get through it is to build an actual relationship and then actually deliver value to them, right? So, you're not pushing your agenda, right? You're pushing their agenda and you're like actually supporting them. And then they're going to see that you're a trustworthy partner.

Selling yourself when your title doesn't match

The job market, quite frankly, is abysmal. For folks whose work may overlap with data science but not have been called data science when they were employed and didn't have the title of data scientist, what are tips to, quote, unquote, sell yourself to the hiring manager? There's the, obviously, tweaking your resume and so forth, portfolios and so forth. But I've noticed there's an issue of, okay, but you're not X field. And it's like, yes, but so many of these things are transferable. I promise. I know what I'm doing. Thank you.

Thank you, Noor. All right, Gabriella, I would love to chat with you about this one because you mentioned, hey, all of our job titles are meaningless. What do you think about how to sell yourself? Yeah. So, first of all, whatever title you have inside the company, it's very specific to your company. So, you can have any title that you want in your resume. First of all, that's it. You might have, inside your company, you might be like a quantitative analyst. What is a quantitative analyst in the job market? I don't know, right? So, you can call yourself data science. If you are doing the work, you can say, I am a data scientist. And you can sell yourself as a data scientist.

You can add to your resume, add to your LinkedIn, you are a data scientist, period. You don't have to be justifying, oh, hey, I am a data scientist, but my title was something else inside my company. No. That's it.

Let me add something interesting. At Microsoft, a lot of people on my team went on the technical program manager ladder. Because they were developer advocates. And there was no ladder for developer advocates, per se. So, internally, if you go into the job role hierarchy, everybody was a TPM. But we were not doing TPM work. So, remember that titles are internally. You can be whatever you want outside.

Yeah, no, I agree to this. Right? First, let's be kind to ourselves. The market is very hard. It is. And in this market, people are getting very finicky about the requirements. Right? If a role needs a master's degree in statistics, they are going to stick to it, no matter what. So, first up, at least, you know, that is the present scenario. And we need to acknowledge that.

The second one for this specific question, how do you break into it? I plus one with, you know, you can certainly say that you are a data scientist or function in a role of a data scientist. Right? And try to pitch yourself. But to break in, like Dan was earlier mentioning, right, you may have to try startups or CRO or organizations that are more open to hire people with not an exact specific match, because that's the market right now. If you were talking about three to four years back, I think things would have been different. But right now, getting into a data science role, when you don't have the relevant data science title, in companies that are hiring for the specific requirement is hard. So, I just want to acknowledge that. It is hard.

Rapid fire questions

He says, since layoffs seem to be happening all the time now, and it takes so much longer to find a new job, are any of you looking at independent consulting or self-employment? I know that Gabriella is. So, Dan, have you heard of anybody looking at these things? And do you think the job market is going to improve?

I mean, I would say it's a different world of running your own company or independent contracting. So, I'd feel like you have to be passionate about it. I wouldn't go after it if it was like you're forced into it. This is all I could do. I think you're going to need to really have the passion and the drive to go after things like that. That's just my personal opinion on that. But I know the job market is awful right now. I know the hiring process is broken right now. I feel like networking and just kind of getting out there. I think Nate in the chat was talking about contribute to a package, something like that. You could do some work. I guess it'd be unpaid, but you got to get out there and you're honing your skills and staying fresh and meeting people. So, there's just a lot of positives there.

It seems like a lot of resumes are immediately rejected by an AI. Any tips for getting past that point? Makaran, what do you think? Yeah. I mean, you know, one, I recently, there's a company called callings.ai, the founder of that company, I talked to him and he said a very interesting thing. He was like, the first thing he does with resumes these days is convert them into RTF, rich text format, simple plain text. And he's like, the reason for that is many of these AI tools don't understand any graphics, right? Or at least mess up some of the graphics, botch up some of the graphics. So, you know, simple plain text without any images, you know, is the way to go for your resume.

Many people often miss that. Try to use tools, callings.ai has a very good tool because I met the founder, I'm making that call, where you can put your resume and see how the resume automatically, the AI translates that resume into, you know, job requirements. It's a good first step because the first step for your resume is to be, you know, adopted to AI tools and you may want to test that out. And if it's not doing a good job, go to sites like Adobe, maybe Adobe careers has also an automated AI tool. Try them, see how your resume gets translated, because if it doesn't get translated well or picked up well, then you're not going to get noticed. So, that's the first thing. Second, I think there's websites like jobscan.ai or IO, I think, yeah. They give you what are the keywords missing for the job you're applying to. So, try to have, you know, keyword matching so that AI doesn't restrict your resume from going forward. So, those are two tips I have, at least from a perspective.

Ethical implications of AI

Gabriela, I'm going to throw another question your direction and let's see if we can give Gregory some advice here. Gregory says, what do you think of the ethical implications of using AI? Meta and NVIDIA were both caught violating author's copyright and YouTube terms of service, respectively.

Well, as always, I have a lot to say. But let me share a good example. Yesterday, I was using Claude and I was testing something and then I was trying Claude to see if they could make up something for me. And then they said, oh, no, you have to be careful. I should be not making up things for you in this X, Y, and Z. And I was like, oh, so you don't do everything that I ask you for? Great, that's awesome. So, it was a good signal. But I would say there is a lot of like, there are so many lawsuits against open AI and more open AI on the right about especially like drawing and writings and this data being used to train all these enormous models.

All these companies, they had ethical AI team inside them. And maybe a year or two years ago, those teams were the first one to get laid off. So, there is no team anymore pretty much in those companies where they are thinking and they are vetting and they are using the whole responsible AI umbrella to think about what are the implications that this product, if I put in the market, what is going to happen? Like what are the guardrails that I have to have in place, et cetera, et cetera, et cetera. They still have, like at Microsoft, at least, we still had the responsible AI team, but a lot of them got laid off as well. So, I don't know. I don't have much to say other than if there is no regulation, it's so hard to change the culture around it.

Will hiring recover?

Since you all think, to some degree or another, that C-suite might eventually realize that you can't just replace people with agentic AIs and systems, maybe they haven't quite calmed on yet, do you think that there's going to be a shift in the way that think that there's going to be, if not a hiring boom, maybe a low humming buzz of hiring in the foreseeable future?

We'd like to think, right? I think the foreseeable future, I think there's not much foreseeable right now. You know, I think it's just like, it's so chaotic. I mean, the entire world is just so chaotic right now. It's very hard to look into the future and try to say what's going to happen. You can kind of look at the past. I know we have a lot of statisticians on the call here, but if you look at past and try to predict the future, hard to say, hard to say. It could be a huge boom for the economy, and you get all this productivity, and then you need a million AI engineers. So we hire a bunch of people, or it could automate everyone's job, and we need no people. So to me, I think it's hard to plan. I think prepare, the best we could do is just prepare, learn the tools, understand where we could actually add value to organizations, and go from there.

I am optimist on that front, to be honest. I am on the second part where I think this is going to lead to an economy of creation. Many people are going to try many ideas, and we'll see a boom of ideas, products that people were not able to release into the market. I personally think, as this settles down, in my opinion, AI has needed a lot of investment, especially in infrastructure. That's where a lot of jobs are getting lost. Meta suddenly realizing that they haven't invested enough, as much as, let's say, Google or Microsoft has, and investing more has led to some of the job loss. I do think the increase in productivity over a period of time certainly leads to increased jobs. So as this settles down, as the infrastructure costs go down, people start talking about ROI on AI investments. We would lead to more productivity and more jobs in this domain. That's my personal take.

I think we are going to get back to hiring. I don't know if it's going to be soon, but I would say probably in two, three years, I think we are going to get back to the hiring part. It's going to be different, definitely. The other thing that I've seen, because someone asked about LLC and starting your own consulting, what I'm seeing now is these people that are being laid off, they are creating other companies, and they are creating other startups, and everything is getting back again. It's almost like a cycle. That's why I've seen this happening before, like 10 years ago. It's not the same scale, definitely not, but I'm seeing the same cycle. People are getting laid off before they were leaving, and they start a new company, and they start hiring. Then they get people that were laid off, and then it starts all over again. So many startups are happening right now, a lot of investment, a lot of money. People are being laid off, but some people are being added to this new startup world as well. So it's a little bit like it takes a while, but I think in two years, starting two years, more or less three, I'll see everything going back again.

It's almost like a cycle. That's why I've seen this happening before, like 10 years ago. It's not the same scale, definitely not, but I'm seeing the same cycle. People are getting laid off before they were leaving, and they start a new company, and they start hiring. Then they get people that were laid off, and then it starts all over again.

We're already in a cycle that started, I feel like, in 2021 when there was a bunch of overhiring, and then something happened at the end of 2022, and things started going the other direction. So we're already in a cycle that we're riding, and maybe we're not at the end of that yet.

Well, we have two minutes left, and I don't want to say goodbye yet, because Rachel has a really fun question for each of you to end with. We can start with Gabrielle and go backwards. Yeah, my question was, what did you all want to be when you were a little kid? A veterinarian. Yeah. Makarand, what about you? Do you have any little kid aspirations to share? Maybe a pilot. Oh, that's a good one. Dan? I wanted to be a teacher, but then I realized I didn't want to be a teacher, and my goal was just to have some job.

Okay, everybody, this was so much fun. Thank you so much to Gabriela, to Dan, to Makran for joining us today. I hope you had fun. Lots of fun. I sure did. Yes. Goodbye to everybody. Also, if you would like to save the chat today, which is full of resources, there is three dots in the top right, so you can do that. All of the questions that did not get asked today, I am going to do my very best to transfer them to the Discord server, to the Data Science Hangout channel, and even if we do not have everybody in there that is in our conversation right now, please hop in and give me your thoughts on them.

Again, if you have jobs that you are looking for or hiring for, please share them in the job board in the Discord channel so that we can help connect people with opportunities that they need. Okay, everybody, I hope you have a fantastic rest of your day, your long weekend coming up, if you are in the United States. I will also say next week on the Data Science Lab on Tuesday, we have Worktree First Coding with Barrett from Posit, who is going to be showing us Conductor, a sort of Worktree First, Worktree Orchestrator. It is going to be very technical, very interesting. We are all going to learn a lot. Then on the Hangout next week on Thursday, we are joined by Chris Gates. He is the Director of Bioinformatics Core at the University of Michigan. That is going to be a lot of fun. Chris Gates is here. He is waving to everybody on camera. He is going to be with us next week. Thanks, everybody. Goodbye, Dan. Goodbye, Gabriella. Goodbye, Makrand.