Data Veracity and Diversity

Read Interview

Chris D’Agostino:
It’s great to have you back. It’s been a few months since we last chatted, but it’s great to have you here.

Yao Morin:
Yeah, it’s awesome. Thank you for inviting me back.

Chris D’Agostino:
When we last talked, one of the major things that you were talking about was data quality and the importance of that to the work that you’re doing at JLL. And if I remember correctly, it wasn’t so much the volume of data that you were concerned about because you can scale up to process that. It was more around the idea of the veracity, the accuracy, or the truth of the data, and being able to leverage that data for the work that your company’s doing. So can you shed some light for our listeners on just how are you thinking about that? And I know it was what you described as what keeps you up at night. So give us some insight into that.

Yao Morin:
Yeah. Absolutely. Before I go into that, I think that it will be good to just give a little bit context as to why we have this problem in JLL and I don’t actually think… Why I think it’s really good for me to have this platform and opportunity to talk to everyone about this, is I don’t think it’s a problem just for JLL, right? So let me first start with why the problem of having the wide data problem. That’s the term that I would use is that JLL is a professional service company that services clients that are from all kinds of industries, from all different countries. And also in commercial real estate, which is where JLL is specializing in, is that there isn’t a very standard way of with presenting data.

Yao Morin:
So like you may be walking on the street and seeing all this building, you’ll be like, well, what are you talking about? If you are doing commercial real estate, every building has a address. And every building has floors, right? Have room numbers. Why is it so different to actually represent the data that you have in JLL in a standard way, or in a way that we can leverage data very much. There are two aspects to it. First of all, if you just think a little bit more on that is like every single building actually could, the bigger buildings, you know there can be multiple entries, right? So for example, Salesforce Tower has an en entrance on Fremont and have another entrance on Market Street. And you know, which one is the right address for Salesforce tower? It’s actually both. Right? So that’s one problem.

Yao Morin:
Second problem is like, if you think about the field that we commercial real estate in is not just like office building, but real retail center, restaurants and, and multifamily housings. And it’s actually within a very big context of the world. And so we get a lot of data from all different sources. So it’s very difficult. One is you don’t have a key ID, like key things that you can tie all the data together. And then also you care about all the different kinds of data because we care about… So for example, for commercial building, I care about the people walking on the street, how much traffic we get, I care about subway. Is there a subway station around? Is it, I don’t mean the sandwich place, I mean the public transportation place, although maybe it’s really important as well for some people that there’s a Subway restaurant around it.

Yao Morin:
So like, that’s why it’s such an interesting and challenging problem. And I don’t think this is only a JLL problem. I think a lot of industries, it’s not just like focusing on your own thing, right. I used to work at a FinTech company. We also have that kind of problems. Like we care about all kinds of data. And the problem is how do you merge all of those data from different data sources, from different aspects of the technology can all come together so that we can actually make sense out of those data. Sorry, it’s really long answer, but I’m very passionate about this area.

Chris D’Agostino:
Yeah, no, it’s obviously it’s great. Can you shed some light onto like, just describe, I think for the audience, the product that you’re trying to create as a result of collecting all this data, right.

Yao Morin:
Yeah.

Chris D’Agostino:
So you joked about like how much pedestrian traffic is there, is there a subway or a train station nearby because if you’re representing some real estate and you’re trying to sell it to a company that’s one to draw talent into the building, the presence or lack thereof of a subway station changes the commute dynamic, right?

Yao Morin:
Yeah.

Chris D’Agostino:
Or whether or not there’s a Subway food store changes whether or not people are able to get a quick lunch if they need to grab lunch during the day. Help us understand what you’re creating as a result of all this data.

Yao Morin:
Yeah. Absolutely. And Chris, you are really hitting the nail at the head. It’s like, we want all those information all in one place so that our customers, our clients can actually understand what they’re getting into when they’re buying a building, renting a building, or trying to attract talent to working those buildings or trying to attract their customers to the retail center. Providing that information, and then so that they are armed with all this information that is relevant to their decision making process so that they can make the best decision. Right?

Yao Morin:
So like if I make it a little bit more concrete I’m trying to, my team and I are trying to achieve a lot of things. JLL technologies, as part of JLL is really to focusing on how do we bring all those data and then create those products that can help our customers to make the best decision. To like one example is we are building this data lake, leveraging data bricks to actually source all those different data together, merge them so that, and then building a data visualization layer so that we can show on a really intuitive way to our customer, like a 3D… Imagine Google maps on steroid, where you see all the buildings like with the height and where all the other resource, or like restaurants, subway station, wellness center, dry cleaning because our customers, our tenants that we help, our landlords that we help our building owner that we help really want to not just to do a business, but really providing a really good experience is for the people who are in those buildings.

Yao Morin:
So if I think about what JLL and JLL technologies is all about, it’s not just to do the business of commercial real estate. It’s really to also provide a good experiences for anyone who are in those buildings. And I believe that data is the most important thing that we can help those people to achieve, the companies to achieve those goals. And obviously data breaks really help us to do that as well, to really help us to pipe all the datas in. Because you can’t do any of that without all the data in one place and in a form that is consumable.

Chris D’Agostino:
Well, that’s really nice to hear, and wasn’t anticipating such a [crosstalk 00:07:43] endorsement.

Yao Morin:
A call out.

Chris D’Agostino:
But thank you. We’ll be sure to send you a Data Bricks t-shirt. So help me understand, you’ve got all these different data sources and clearly the… Like figuring out what’s going to be most relevant to help the customer create their decision on whether or not to buy or lease a particular building. And I would imagine the attributes that are important to each customer, they vary by customer, right?

Yao Morin:
Mm-hmm (affirmative).

Chris D’Agostino:
And so you’re really having to almost, same sort of concept in the retail and the banking space is also think about like a customer 360 and what data would be most relevant to them. Obviously, there’s plenty of things that are probably standard in terms of square footage, and price per square foot, and the age of the building and accessibility and all these other things. But there probably are some other aspects that really make a difference if the company is trying to draw a particular type of talent, the style of the building and the proximity to a park.

Yao Morin:
Oh. Absolutely.

Chris D’Agostino:
Restaurants and all of that makes a big difference as well. Right?

Yao Morin:
Yeah. Yeah. Especially in this environment that attracting talent, the right talent is so important to all the different companies, employers, right. So they really want to provide the right experiences for their employees so that they have a reason to come to work. Right. So it’s not really just about you come at your desk and then do the work. But a lot of the talents nowadays really think about what is around my building? Is it like cool to be around?

Yao Morin:
I was just talking to one of our brokers in Boston, his name’s Jim and Jim was telling me that people cares about having cold brew in the office, having like a dentist on site, even having childcare facilities nearby so that it really provide that all provide that kind of services for the talents that we have here. So that’s why it’s just so important that when they make those decisions, we have the data for them to look at it. Right. If like they’re asking us a question, “Hey, is there, is there a childcare facility nearby?” If we couldn’t answer that, we’re really just not providing the right information for our customer.

Chris D’Agostino:
All right. So a couple of things that I wanted to chat about that I hadn’t touched on was what things keep you up at night when you think about the data and AI space? I mean, we’ve talked about all the different data that you need to draw together from these different sources. Presumably you’re tr you’re trying to create a customized experience for your clients. You’ve got different source systems and maybe owners of that source data that are external to JLL. What kinds of things are you still struggling with as you think about the data and AI space?

Yao Morin:
Yeah, absolutely. On the data space is really to make sure our data is of high quality and accurate, right? You don’t want to, because these are millions to hundreds of millions of dollars of decision here. And we don’t want to provide data that is outdated, right. That is not accurate. So what keeps me up at night is obviously a lot about how do we make sure that we have the most comprehensive data, but also most timely and accurate data.

Yao Morin:
So this is something that I continue to make sure that my team measure themselves against is the comprehensiveness, timeliness and accuracy so that we can provide the right information. Because these decisions are expensive decisions.

Yao Morin:
Yeah. So on the AI space, I think that JLL has been recognized as one of the most ethical companies in the space and in the industry. So AI, I know that you guys may already talked about it is like, there are a lot of like ethical debate around using AI, using data. How do we make sure that when we make decisions using machine learned algorithm, and then we can actually make sure that we are ethical as well. So that’s another thing that is important to me that keeps me up at night is like, we don’t create algorithms. We don’t create intended consequences when we are not using data and AI properly.

Chris D’Agostino:
Yeah. How… You’ve had a really impressive career you’ve been at Intuit, you’ve been at StubHub, now JLL and I was struck by the comment that you made about when your clients are making these decisions, these are massive investments, millions of dollars are kind of at stake here in terms of them creating a multiyear lease in a large building, for example, costs a lot of money. I think about the transaction or sort of transactional nature of StubHub where you’re buying a ticket to a particular event. So on a per person basis, it’s maybe smaller amounts of money. And then when you think about Intuit, I don’t recall what your role was there, but with managing people’s finances and their tax returns that’s obviously very personal. And so I’m just thinking about sort of the scale of going from, okay I bought tickets to a concert that cost me a few hundred dollars, which you want to get right at StubHub, for sure. And I just recently had a bad experience with purchasing a ticket in Italy for an event.

Yao Morin:
Oh, no.

Chris D’Agostino:
A soccer match. And then they shifted the event and moved it up a day earlier and I couldn’t actually attend it.

Yao Morin:
Yeah.

Chris D’Agostino:
And it wasn’t StubHub, it’s a different company, but they were actually really great. They not only refunded me the money for the ticket, they actually gave me a credit for the ticket amount for a future purchase. So I basically got my money back and then some.

Yao Morin:
Oh, double? Oh, wow. That’s awesome. That’s awesome. I wish it was StubHub.

Chris D’Agostino:
I don’t even want to say the name of the company because I probably will get it wrong because it’s like this little known company maybe. So anyways, that was a good experience. But I’m thinking about just the scale at which these decisions, over your career, do I have it kind of right? Like, are you feeling maybe a bit more pressure to get it right? And especially with the use of AI and ethical AI to make sure that your team’s clicking and sort of firing on all cylinders here?

Yao Morin:
Yeah. I think that when I’m in those companies and as the data, because I consider myself and my team the custodian of our customers data. We don’t really own those data, regardless whether it is a small decision, whether you want to buy a ticket to a concert, which could be a huge decision for someone because this could be a once in a lifetime kind experiences, to like making a decision whether we are buying a billion dollar building, to whether you are claiming your donation to Wikipedia at the end of a tax season. That’s what I’m I do at Turbo Tax, at Intuit is be the data custodian for all the millions of users, Turbo Tax users. And I think that the pressure is still the same because I think that the data is so precious and then they trust us to give us our data.

Yao Morin:
And I don’t think I will think differently in any position that I’m in, as long as like our customers entrust their data to us. And I think that we, as the data practitioner or people who work in related fields, it’s just so important. We really need to get it right. And data privacy and security is definitely something that I have been really making sure that my team is understand the policy. Sometimes we also be part of the policy making within the company and then to just making sure that we be good custodian of the data of our customers, of the data that we collect from other sources and present to our customers in the best way that is possible. So yeah, no, actually in fact, in Intuit, because the data is so sensitive, all the tax data, all the financial data we, at Intuit, we really are very serious about our data security and privacy as well.

Chris D’Agostino:
Yeah. So I mean that is like the perfect executive answer to a question that maybe was sort of on the spot. Right. I think it leads into maybe the next area, which is like the culture, the data culture of the teams that you’re building. Right. It’s, you made it, couple of things came to mind when you gave your answer. Right. The first is, yeah maybe somebody spent a hundred dollars on a ticket, but it’s a once in a lifetime event that they might be sharing with somebody that maybe their parents or grandparents-

Yao Morin:
They will be proposing.

Chris D’Agostino:
Yeah.

Yao Morin:
Maybe they will be proposing in a game. Right. Who knows? [crosstalk 00:17:18].

Chris D’Agostino:
So value, can’t just be measured in terms of money. Right. Money spent.

Yao Morin:
Yeah, exactly, exactly.

Chris D’Agostino:
It’s the impact of the experience and the decision on that person. Right. Or the people affected.

Yao Morin:
Yeah.

Chris D’Agostino:
So that was a great answer. And so it leads into the kind of this idea around culture and making sure the teams understand not only the importance of protecting the data and working with the data well from a legal and regulatory standpoint, because you mentioned there’s lots of sensitive data, tax data for people. So it could get Intuit into a lot of legal trouble, for example. But just the idea that this data represents the lives of people and the moments in their lives. And we have to make sure that we’re working with it in the best way possible.

Yao Morin:
Yup. Absolutely.

Chris D’Agostino:
So can you talk a little bit about how you sort of coach your team or teams in the past, around data culture and what you’re doing to make sure that you are protecting that data?

Yao Morin:
Yeah. I think that the best way to do it, that I learned, I didn’t come up with it, but the best way I learned how to motivate a team to really think about himself as the data custodian, abiding to the right policy and taking the right measure to protect the data and then leverage the data in the right way is to anchor on our customer. Sometimes you don’t think about that, but whenever I talk to engineers and whenever I talk to data scientists and business analysts, they really get motivated to do the right thing by the customers. It means so much to them. And then we always try to anchor ourself is like, one is what kind of… Are we using this data to make our customers life better? Right. Is this measure that we are taking? Yes. It’s probably a lot of work that on our site, is this providing the right protection to our customers data? Right?

Yao Morin:
So for example, at JLL, what we really anchor in is like by gathering those data about dentists, about childcare, about dry cleaners, about restaurants around the buildings, is that going to help our customers to provide the best working environment for, for their employees. Or as StubHub, we leverage data to really help our buyers and sellers to understand the value of their tickets. So we spend a lot of AI, our data science effort in doing a pricing model to provide to our buyers and sellers, as a neutral party, to make them understand what is the value of the tickets at a market way so they can make the best decision.

Yao Morin:
So by anchoring all the things we do and all the things we don’t do about whether that is going to make our customers life better or making our products better so that our customers can benefit from it is the best way to motivate the team. Because I can say all the slogan, “Hey, we need to make sure we abide to GDPR. We need to make sure are compliant to CCPA. We are making sure we have good data classification,” all those are very empty if we don’t anchor in how that is related to providing customers benefits, using by what we do.

Chris D’Agostino:
So yeah, when we were talking earlier, you talked about that your team’s creating effectively like Google Maps on steroids. So this idea of here’s the property to be considered, and here’s a bunch of overlays for information that would be relevant to the decision making. Can you talk a little bit about how you and your team go through that product definition, right? That data product in what you do in terms of the user experience, sort of design thinking, you tie this to the customer. Help us understand the process there.

Yao Morin:
Yeah. Yeah. Great question. And I love the fact that we are talking about that, not just on the data infrastructure part, but actually what we do with the data. And then that’s very much what we are trying to do is, we are not just thinking about bringing all the data together, but we are also thinking what we are doing with the data. So that’s where the Google Map on steroid come about., Right. I have to say that this is not my idea. And it’s a idea of a lot of the awesome folks in JLL that we work together to bring that to life. But I do want to address a little bit on all the how do we build a good data product, right.

Yao Morin:
And I have to give a lot of credit to Intuit. How Intuit really helped me shape the way that I see product development, that includes data product. Which is at Intuit they use a method design for delight. There are a lot of like different aspect to it. But the one thing that I take back is, again, I feel like I sound like a broken record, but it’s anchoring at customers. Right? Having the rapid iteration of putting something out there, getting the feedback right away and then continue to iterate on it. And really one is to conduct enough customer interview to get feedback. And also one other thing that is like the secret sauce of Intuit is we want to observe our customers in the environment where they use our product.

Yao Morin:
So we will give our product to our brokers, to use in the field, to present to the customers. And then we will be observing how they are using it, interacting with it, what kind of reaction that our customers have so that we can continue to revise and iterate on the product. This is not just like on data product, but all the products. We really need to just ask ourself, what is the customer problems that we are solving and how are they reacting to it? And really like continue to anchor on it. Not so much about how much money I can make out of it, because I believe that if you build a product that your customers like, if they actually get value out of it, the money will come. So that’s kind of where the philosophy of like just product development in general and then data product is no exception out of it.

Chris D’Agostino:
Yeah. So it’s, I mean, tying all this together, right? The user experience and observing how human beings react to the products that you’re creating all the way back to the data sources that feed the data. And then it brings us back, I think, to sort of the final area of deep dive here, which is around the architecture and the… You’ve already talked a little bit about it today, where the importance of that to sort of coalesce this data and be able to run those workloads.

Yao Morin:
Yeah. Yeah. So great question Chris. When I first joined JLL, we had a lot of dispar. We already had a lot of data, so it’s not like, “yeah. Coming to JLL, save the day.” It’s not like that at all. But what we see though, is that all the data are scattered at many different little data warehouses around, in different areas. And a lot of you and I’m sure our audience knows that if data is all scattered you can probably leverage a little bit, but the value is like all of them together and be interlink with each other so that we can actually uncover the hidden information, the hidden insight. The surround of like finding the right information to having that unique end goal of what you get from data is from having data in one place and interlink and merged.

Yao Morin:
So that’s what we are adopting a very simple data architecture is the data, like how’s architecture work. We do have a lake and then we have the analytics layers build on top of it so that we don’t have to have multiple areas where we copy data for data scientists to use or for analysts to use. It’s really one really simple architecture that like provide that kind of all encompassing usage for different persona in the company, business analyst, and data scientists, developers, as well as like some of the more other business users.

Chris D’Agostino:
So we’re going to wind up now, close up with sort of the last thing which we ask all of our guests is-

Yao Morin:
Oh, no.

Chris D’Agostino:
Yeah. What’s the square root of… Now, so what advice would you give to people aspiring for careers in data and AI? And I think we’ve already touched on this, like certainly don’t get a master’s and a PhD in electrical engineering. So beyond that, what would you tell people to focus their time and energy on?

Yao Morin:
Yeah, that’s a great question. I think this is, it’s the question that is really difficult to answer because data and AI is so big, right? You can be a product manager that working on data products, you can be a developers that building a data product, or you can be a data engineer that building the pipeline. What I will say is data and AI definitely have a lot of potential. And that it’s a great field to be in. And it’s very exciting. And there’s still a long way to go. Right. And I know that we have been talking about the, at least, I feel like we have been talking about data and AI for the last 10 years. And it has been getting more and more emphasis from different companies, right? Like if you look at 10 years ago, not many companies actually have chief data officer role. And now like most of the companies have it because they really finally realize data cannot be an afterthought, it’s really the asset a company to build and then can continue to give.

Yao Morin:
What I will recommend to the people who want to be in the data and AI field, one is it’s a really good field. I think you make the right decision to be in this field. Second is like, I think that really get down to the basic, right. There are some basic skills that we really need to build so that we can have that good foundation we can draw on. Right.

Yao Morin:
I came from using Python, Sequel and writing spark queries and all those things are really important basics that we should get right, get good at. And then we can branch out. So I do feel like there are some, because there’s so much passion, there’s so much rush and competition. Sometimes I see some of the folks that who want to get into this field really couldn’t even do the basic things right now. So that will be a unfortunate.

Chris D’Agostino:
Yeah. So start with sort of the foundational elements to improve your data literacy.

Yao Morin:
Exactly.

Chris D’Agostino:
Understand how software processes can work with data in the environments that software executes in and distributed computing and things like that, understand the business, understand the customer needs, and then go get your PhD in something. And then you too will be the CDO of a major corporation.

Yao Morin:
Yeah, definitely, definitely. Also seek out other data practitioners advice. I think that as long as I understand, I know a lot of the CDOs in this field and they’re all like super passionate to help everyone to actually get to where they want to get to. So if, don’t hesitate to reach out, right. I feel like data and AI is a field that people are very passionate about what they do. And then also from what I experienced people are very nice and then willing to share.

Chris D’Agostino:
Yeah, absolutely. Well, Yao, thank you, chief data officer JLL, it’s always fun talking to you and-

Yao Morin:
Thank you Chris for-

Chris D’Agostino:
… if you ever want to compare notes on doubling, shortening these weight equation, let me know, I’d be happy to do it.

Yao Morin:
Chris, thank you so much for inviting me back. It’s always a pleasure to talk to you. Thank you very much for this opportunity.