Apr 21, 2021
Dan and Lee talk to Data and AI specialist Katie Ford. Katie has been working with some of the best data scientists in Australia over the last few years and shares her learnings from her work with CSIRO's Data 61 unit, Intel and more.
Shownotes:
Data61 - Home - Data61 (csiro.au)
________________________________________
TRANSCRIPT For this episode of The AI in Education Podcast
Series: 4
Episode: 4
This transcript was auto-generated. If you spot any important errors, do feel free to email the podcast hosts for corrections.
Welcome to the AI podcast. Hi Lee, how are you doing this
morning?
I'm good, Dan. Good. Good to be back again. How are you today?
I'm good, thanks. It's a bit of a grumpy day in Sydney here,
but
in this in this series of the podcast, we are interviewing some
really interesting people around the area of technology and
specifically in data and AI. So today, Hey, to brighten our day,
we've got the wonderful Katie Ford, data and AI specialist at
Microsoft. Hi, Katie.
Hello, Dan. Hi, Lee. How are you both?
Good.
All the better seeing you, Katie.
Exactly. So, you've been working with some of the best data
scientists in Australia over the last few years and also working
with CSIRO's data 61 unit, which you can tell us more about in a in
a second. And before that, Intel and before that, I think the
Australian government. You've got a a well trotted path. here in
the data and AI community and and in public sector. So, welcome to
the podcast and um yeah, how are things?
I'm good. I have two pugs at my feet that are looking at me quite
needy, so uh working from home has been uh challenging and fun and
uh yeah, we we'll see how the podcast goes in terms of dog
interruptions, won't we?
I know Lee I know Lee's got a dog in the background as well.
I I I do. I've got a 14month-old puppy that's on his sleep great
sleep session. So, he's good.
I've got my daughter who's away from school today, so she might
turn up as well because she's not well. So, so it's all going to go
today.
It's it's the new world of remote working, isn't it? Everything is
just normal and natural behind us. So, that's great. Uh great and
great you could join us, Katie. Great for you to put some time into
this.
Um so, look, should we just dive into it? Let's get some questions
going, Katie, because I think that's that's the best place to go.
Um so, Dan talked about I mean, you've got just such an incredible
history of the places you worked as using government and industry
and science and research and you joined Microsoft in the middle of
a global pandemic and started at a new company and a new role.
There's so much to unpack there. So, we'll get to that. But look,
tell this is about technology. So, what is it? What got you into
technology? What got you interested in being a techno nerd, geek,
whatever we want to call ourselves these days?
Yeah, I've got to I've got to be honest. Early in my career, it
wasn't apparent to me that I would go into the tech sector. I
remember my mom giving me a um a book about um the Aquarius star
sign, which is what what I am. I'm born in that period when I was
young. My mother is is, you know, she's the sort of person that
would ring you each morning and tell you that Jupiter and Mars are
at at at you know, at odds and your day is going to be full of
conflict. It's always a very start to the day. Um but she said, she
gave me this book. I would have been eight or nine. And I remember
flicking through it and um it said, you know, career-wise, you'll
be attracted to working in areas of deep technology in Silicon
Valley and Toronto. And I sort of chucked it to the side and
thought, "Ah, you know, what's that saying about about horoscopes?
It's all rubbish except what they say about Virgos." So, I sort of
put it to one side and I went down like I sort of thought of myself
as more of a creative type. You know, I was doing debating and
amateur acting and, you know, was very good at English and maths. I
loved math. But, um, I ended up doing law and history at university
working in public policy for a long time, really really important
areas like social housing and homelessness, um, which I absolutely
loved. And I remember a friend of mine when I left government
called me, Belinda Dennett, um, who who you both know from
Microsoft, and she said, "Katie, there's this awesome job going at
Intel leading their corporate affairs. You should have a think
about it." And I said, "Oh, you know,
I could I could barely program my microwave. Like, I don't know.
May like I'll have a discussion with them, but She said, "You know
what, Katie, the great thing about working in this space is it's
about the future, and we're really trying to get people from
different backgrounds to come into the the industry because we're
we're building, you know, to build for future generations. You need
to understand those generations." And I saw in um the New York
Times that day a profile piece on Genevie Bell, who at the time was
head of Intel Labs,
and she she she was speaking about all the factors around
technology. So, tech at the center, but all the human factors
around it that make it interesting and and and and really bring to
life how how technology works in the real world and I thought you
know what this is actually going to be amazing and has I haven't
left since. So that was that was seven years ago and I've been
loving the journey.
Wow. So almost almost could call you an accidental tourist into the
technology sector. I mean you you sort of started with your law
degree and you know you you sort of through astrology found
technology. That's a new one for me. I've never heard that journey
undertaken before. That's that's awesome. So, look, I mean,
obviously, yes, you you you have been in technology, you've been at
Intel, you've been at Cyro, and now you're at Microsoft and but
you've been in government as well. Do you want to look you've got
it'll take us the entire podcast just to talk about your your great
life journey, but give us sort of the potted version of what some
of the that those experiences that you picked up along that journey
to go through technology, Syro and here and the government uh
before that.
Yeah, just just on the government side, I remember remember I
mentioned I um I worked for the minister for homelessness and
social housing and we had some really ambitious targets around
reducing the rates of homelessness in the country and I remember
the challenge like it's a very complex area but I remember getting
most upset about the data so there were huge challenges with how
you measure homelessness how you track homelessness over time so
traditionally you measure it via the census um but the problem with
that definition is it includes, you know, gray nomads who are
traveling around the country. Um, you know, all those traditional
challenges with how data is captured. Um, you know, are we even
capturing rough sleepers? You know, are they going out and finding
people who are rough sleeping in the streets to include them in the
census? Um, you know, and and it was it was it was also um we were
trying to push preventing homelessness, right? So, not just
stabilizing the houses housing situation for those who are
homeless. But um how you act early when there are early indications
that people might um be at risk of homelessness. And there was no
way in our systems at the time to flag that somebody was at risk.
Um you know there can be a series of crisises in a person's life
that can mean they can fall into homelessness and often it can be
health services um your welfare services. There there are a whole
range of schools can can sometimes find out early that the housing
situation may not be stable, but there was nowhere to share data,
not even within the federal government, let alone between um you
know, state governments, local governments and and the federal
government. And I remember I was just heartbroken that that the
success of what we were doing depended on how open and closed our
data systems were, on um really rigid restrictions around what data
is shared and when. Um it it was it to me it was um yeah, it was it
was a heartbreaking situ situation and I'm happy to say that I
think in 10 years we've come a long way like I'm seeing some really
wonderful things at the moment in different governments you both
would be aware of um particularly around um the sharing of data
around children to protect vulnerable children um so we're coming a
long way there's still more to do definitely but um yeah that was
really challenging
yeah that sounds that sounds fantastic and and then with with the
data 61 hat on was that was that a similar similar projects with
CSIO as well was that the same
so data 1 data 61 is a part of CSRO so our national science agency
it's about a thousand sort of data scientists and engineers and um
I what I love I mean researchers are so undervalued I think in our
country but what I loved about working there um were just the the
enormous minds of the people that I worked with. I'd go down to the
machine learning area and I catch up with them. I'd bring my cup of
tea and they'd tell me about what they were doing and you know some
days they were working on um you know applying genomics to make um
climate change resistant crops right which is obviously going to be
really important for us as a country going forward. Um another
researcher um that I spent a lot of time with there amid des um how
you apply reinforce reinforcement learning which is a type of
machine learning um to understand whether somebody had bipolar or
they had depression and there's a huge divergence in the sort of
treatment that applies depending on which way you
and so you know he he have these this great research they put um
fMRI um scanners on um on the patients and they get them to play
games and your decision-m patterns are very different um depending
on um you know what sort of disease you might have and um he was
able to detect with far greater accuracy than um you know a
traditional medical system um whether a patient fell in one
category or another. And I remember at the time thinking, you know,
I did a lot of work at Intel in the education space and we talk a
lot about things like personalized medicine. Um but I I I think a
lot about how we personalize learning. You know, we have we have
such a vast array of students and learning styles and um different
abilities um out there. And I you know, I remember at the time
thinking, I wonder if similar technology will be available, you
know, 10 15 10 years down the track so that we can actually meet
the needs of each student and deliver education in a you know in a
in a different far more personalized way going forward.
That's so true, isn't it? I think we're getting better at that, but
there's been a lot of siloed um uses of data and education to date.
I know you've done a lot of work in Edu yourself, but there's been
a lot of siloed uses of data across, you know, different
productivity platforms that you know an incoherent setup in terms
of data warehousing and where that data is stored. multiple
software tools in edge use or you know from your learning platform
to your school information system you know it's it's it becomes a
minefield but I feel as if I know I've worked on a couple of
projects here I feel as if school systems are starting to feel as
if data is the new water I won't use the data is a new oil an
analogy I move to that data is kind of water something we need and
something we use and I'm seeing a big trend myself and I know the
projects you've been working on with us in edu have been fantastic.
Do you want to share some of those projects as well like project
constellation?
I can talk a little bit about the sorts of challenges and
opportunities we're seeing at the moment in the education space. So
obviously we've just been through almost a year of co you know a
lot of schools around the world have had no option but to go to um
remote learning or hybrid learning and I think we've seen over many
years a lot of feedback from school systems saying you know we've
all these disperate data sets. We've got more and more applications
being used every day. It's really hard to get a holistic single
view of a student and how they're going. And so there there are
there are a couple of really exciting projects I'm involved in at
the moment where we're integrating data from, you know, dozens of
different applications, whether they're SAS applications, also, you
know, student information systems, Office 365 data obviously, but
bringing it all together to understand at a really granular level,
you know, little Lee is struggling with obviously with these sorts
of questions. They're trigonometry questions. How do we personalize
uh you know, our our our response to that and and serve up content
that is going to help little Lee going forward?
I knew it would be Lee. I knew it. It's always Lee and Pythagoras.
Geez.
Yeah.
And you know, and Dan would be able to drill into, you know, on
specific tests with specific question. questions where where are
the where are the concepts not not um forming you learning is a
very complex process not even our you know best professors around
the world fully understand it but what we need is I think more
evidence-driven inputs into how we deliver education that's what
schools and and departments of education have been crying out for
so
yes
sorry to interrupt you terrible um I have a question because I
think what you bring up is a really interesting point And it plays
back to something you said earlier about obviously data has been a
key theme through your life and you talked about the stories of
when you were in government and the the issues of homelessness and
the challenge was the data like the problem was clearly there but
the data didn't always reflect the nature of the problem because
does the data really reflect the true situation? Are we capturing
the right sets of data? And I'm thinking about what you're saying
now in education and using infant data to influence children's and
uh educational outcomes. I just want to say the word pedagogy as
well. Pedagogical outcomes. Um Ed,
so I do you see a big risk here that we're actually, you know,
we're building these intelligence systems or we're thinking about
these intelligence systems for education built on data sets that
may actually not be a true reflection of the child's full potential
or the child's full picture or the broader picture. Do you see
where I'm going to with that?
Absolutely. Absolutely. And I think um I I think you're you're
right in any in any data um an analytics project, you're you're
definitely limited by the range of data you're or you're um sort of
able to collect and ingest. Um I think though what we're seeing
with with the move towards remote and hybrid learning is um more
and more um activities are captured online. You know there are
there we can now measure collaboration through things like teams.
Um there are different ways to measure um you know creativity um
for instance. But I think it's I think you're right Lee. It's it's
a it's a journey and we'll we'll be able to get some insights into
a into a student's learning um uh learning cycle. Uh and I think
but the problem traditionally is we just have you know nap plan
tests every couple of years and we get this tiny little slice of a
student's learning career and that's it. And by the time you have
that data it's too late to intervene, right? You just wait for the
next nap plan test to come through. So we now we're getting day by
day
far more granular data. that's coming in. That's an enormous
opportunity.
I I had a chat just you said that I had a chat with one of my
ex-colagues I used to teach with in the UK last night and they
obviously currently in lockdown with co and they've cancelled the
equivalent and applan tests for the kids like like uh lots of other
countries have done and it's really interesting they basically
surveyed the staff during the process and said look if we knew the
CO was coming this is they surveyed them last year they said if we
knew CO was coming what would you do to assess the kids? And the
staff overwhelmingly said that they just do more frequent
assessments so that then they had a better picture of the kids. So
that's what they decided to do this year. So they've been doing
more frequent assessments rather than large standardized tests at
the end and they've been using teams to do that for example to get
the that kind of telemetry out. But like Lee said, you know, it's
more complicated than that, isn't it? When you got the well-being
of the kids involved and lots of other factors around that. What
what are the priorities you think then for for education at the
minute? What what are their analytics priorities at the minute
across K12 and universities?
Yeah, and I was just going to say on that on that point there, Dan,
you know, some students may not perform well in a test environment,
there are lots of students out there that due to nerves or for
other reasons, they may not perform well in tests. So, the idea of
bringing in all that data from the learning applications that
they're using, things like mathletics, you know, there's there's
scores of them out there is that when they're not under pressure,
you can see how they're how they're tracking um over time. From a
priority perspective, I know a lot of the departments of education
are very concerned about students at risk of disengagement. The I
think co as as we know and as you've both talked about on this
podcast, you know, can um work well for you know that remote
learning, the hybrid learning can work well for some students but
but not as much for others. So, they're really worried about those
students who've been who who are fallen behind um through the
process. Um but having the evidence base to show that particularly
when that plan has been cancelled um can be really really um really
challenging. But I think that they're trying to do a few things.
They're trying to understand one students at risk and and and what
what is the right intervention, what's working from an evidence
base. They're trying to also identify um high performers as well
because that they also need their own support and that that can
raise different different sorts of challenges and opportunities
going for forward. Um, and there are lots of lots of I think every
every education system including the universities are under a lot
of pressure at the moment around the best use of um finite
resources. Um, and and so how you might um how you might sort of
predict um where you invest you know and that and that requires a
lot of analysis of um you know what has worked over time um and
what hasn't. There's been some really interesting research that has
come out. Some showing that certain sorts of um professional
learning have worked very well and and others don't. So I think
they're very keen to understand in a more fine grain way what's
worked on the ground in terms of student outcomes.
So Katie, that's um I I love what you say there about the make best
use of finite resources because this is the perennial problem in
any sector of any industry, you know, and education feels it
probably just as as worse as anywhere else is that this is where AI
can make a difference because it can help those that are the
professionals in a particular domain to move faster, react quicker
and have better insights and kind of that traditional or that human
being or augmented by AI kind of outcome that we we all seek to
achieve. Um so I I mean I guess I'm interested in your thoughts on
how you know that that's the dream that's the idea that AI is this
tool that accelerates human potential. Um what's your perspective
on that? I mean do you do you obviously you must buy into that
because you you work for the companies that do these things and but
I'm interested do you see you know, your time at CSIRO in
particular, working with science and research that were right on
the cutting edge of that that area. Did you see any good examples
of that or did you see that sort of that mindset creeping into the
research community as well?
Oh, definitely. Um, just just on the K to2 um
and the application of of ML in that area. So, I should have I
should have um uh sort of explained, you know, a big focus of what
we're doing is that big data integration piece at the moment, but
the second part of that is how you how you implement use specific
machine learning models in a way that is um that is responsible um
that is you know privacy preserving or privacy enhancing that takes
into account the fairness of the models and of course in education
you know if you're going to use machine learning models there's
great potential there um but you need to do it in a way that is
explainable and what I'm seeing is much faster time to insight um
when they can make use of range of cloud technologies things like
AutoML which you've talked about before which automatically um you
know runs through 80 or so different models um uh ranks them
according to performance and then you just click a button and it
can explain you know which features were the most important in that
process. It's so useful um for for for any anyone in the education
space. Um
yeah I I there was just talking about that I mean auto ML tools
like that are really great, but what it often flags in most
organizations in my personal experience as you know I've been
working with CSRO for quite some time now has been that you can
give somebody all the tools you need but there's also a sort of a
culture and a capability and we talk a lot about the agile
methodology framework and using kind of DevOps and MLOps as
equalized mechanisms of the of how to do this. Do you see that as
being a bit of an a challenge still for many organizations? They've
got the data but not the culture yet to do it. Yeah, I think I
think there's there's a there's a um there's there's a huge way to
go in terms of um helping um schools and researchers on the
skilling front. You know, there there you can't be expecting um you
know, English teachers, history teachers, researchers into climate
change to necessarily be across um you know, the cloud
infrastructure and how each service works. And I think what we do
really well is um provide access to a whole range of different
skilling options for um for people wanting to go down this path.
You know, I've just gone through a whole range of Microsoft
certifications which I found fantastic and I'm not I'm not I'm not
really from a technical background, but they were very accessible,
very very well explained um you know and and they're very hands-on.
They have labs inside each a lot of different components to give
you that confidence going forward. I mean that's a process that
will take some time. Obviously you're never going to see the change
overnight. But I'm really impressed with um you know um Zani Van
White for example up in Newcastle Maitland who is rolling out um
training to to hundreds of different um teachers and personnel um
up at the Catholic Dascese up there, you know, and I think I see
that in a lot of different jurisdictions. They're really trying to
push skilling because that is that is one of the hugest barriers to
adoption. I think also the the thinking, you know, like helping
helping um help helping departments and helping different educators
think through the outputs of this is really important. I remember
one of my favorite data scientists at CSIRO would come in and he'd
ring his hands, you know, he'd say, you know, but yes, we found
this, but they shouldn't then change policies. You know, they need
to implement AB testing. They need to do randomized control tests.
Like this is not scientific. And um which which I which I obviously
understand and often what these models point to is something that's
worthy of further exploration. right in machine learning as as both
of you know there can be confounding variables um that are just not
captured in that data set. So we may have an output that shows you
know what the school is the most important indicator of um student
success in mathematics but but that doesn't measure that you were
not measuring things like um you know what professional learning
was rolled out the individual teachers often you have to go to that
next level of inquiry I remember him explaining to me cuz I'm a big
chocolate eater I eat my weight in chocolate each year. And he
said, "You know, Katie, there is a high correlation between Nobel
Peace Prize winners and chocolate consumption." I said, "Oh,
really? That's the best news I've heard all day."
The statistic.
And and he said, "But what you know what's really the common factor
there is wealth. So countries that are wealthy and therefore have
stronger education systems are able to produce, you know, the
highest proportions of um of Nobel Peace Prize winners, right? So
we We need to we need to think through these things in a common
sense way with with the domain experts obviously that's that's the
key here. It's not just about um your data scientist focusing
in correlation versus causation. Yeah. And you look at these things
and data will tell you one thing and there's so many great examples
of that. You know the the wolf being identified in the picture be
even though it's actually because it's detecting the background not
the object in the front. But it's a yeah the data will lead you
down a path potentially that is not necessarily the accurate
assessment of the picture. It doesn't provide you the full picture
as you say in that particular example. It's about the wealth, not
about the fact that the chocolate is the uh although I think we'd
likely chocolate was a reason why you could get a Nobel Peace Prize
because I think like you I'd probably do pretty well.
I saw that in research as well. I think um I think the opportunity
for AI to underpin the next wave of scientific um research and
discovery is really really powerful. I know at CSIRO they saw AI as
a crosscutting capability that could be used in any of their
research areas and usually it's about uncovering different
different insights that would be the start of the next of their
hypothesis. Right? So it's it's you know it's usually the start of
a journey and um you know we we saw some really wonderful outcomes
in the health space in particular. Um SAR's got a really strong um
health research area but they're really really focused on using
different AI um different AI approaches to uncover um correlations
and insights that a human wouldn't be able to pick up.
Yeah, it's fascinating.
Yeah, that's amazing.
It is, isn't it? Um so, so where where is the future for you,
Katie? Where where do you think data and AI are going to be in, you
know, three or four years?
I think what we're going to see is a um a really um strong uptick
in in in the integration of of data across um system systems
potentially between systems. So we're doing a lot of work around
how you share in a privacy preserving way um the the models as well
as the data so that other experts whether they're um data
scientists in universities whether they're experts over in Finland
you know I think there's great global collaboration that I'm seeing
at the moment around a few projects where we sort of open source
and share what we've learned um and those models can then be used
by by others who may not have the expertise or the time or the
resources to develop them on their own. Um, and I I I'm loving this
sort of global community practice that I'm I'm getting involved in
at the moment. I was on an 8 a.m. call this morning. There's a 6
a.m. one later on this morning.
Fantastic. You know,
from the
Yeah. Well, fantastic. Not for you to get up in the morning, but
but I suppose one of the I suppose we've always looked at data and
standardization. It's been a perennial issue in schools. People
looking at like a data standard format and things. It's never
really ever landed. So, I'm glad that you think these people are
getting together now and especially with the exposure of data
trying to at least expedite that. So, if you've been working in uh
in in in Microsoft or any of these interesting things, where where
would you where would you be? What would you be doing? You know,
where do you think the best kind of focus would be to kind of move
forward in terms of data? What what areas would you like to look
at? if I wasn't working at Microsoft.
Yeah. Or or or CSI road and you you know the money was on no object
and you could kind of like finish like formal work and kind of go
and really focus on an area of society to kind of sort out with
data and AI. What what would that area be?
If if money was no object and I had access to enormous amounts of
capital, I would be I I really would be an investor. I think in in
different startups who are wanting to make an impact on things like
sustainable development goals. I just saw the breadth and the depth
of research in this country and you know I for me I've always been
attracted to organizations that are missiondriven and really want
to make an impact that sort of is is a top priority for me but I
think we we have such amazing smarts in this country and such huge
global problems. Um yeah I mean I would also like to catch up on
sleep. I have a young toddler and and so initially I'd catch up on
sleep and then I'd apply my mind um to work. But no, I'm loving I'm
loving it here. I'm loving the opportunity to make an impact. Um
you know, I think education is is one of those areas that we all
feel some sort of connection with. Um you know, I I I I would do
anything to see to help um a lot of the partners that I'm working
with to get to where they want to. Everybody's motivated by the
right and everybody wants to see better educational outcomes. It's
a fantastic um area to work with and I'm so privileged to work with
some of the most brilliant brains in the country. So, it's a it's a
privilege and an honor um to be working.
It's yes, it's it's great to great to know and great to have you on
board with the education team as well because yeah, education is
absolutely one of the priorities here. So, what would your advice
be to your children as they grow up?
I think what I've observed in my life anyway um is I think the
passion for learning matters more than anything. Um, you know, I
did a law degree, I did a history degree, I did an MBA, but I think
what they each taught me is that knowledge is a very dynamic thing.
Um, you know, there's of course a lot of data we have about how um,
you know, the, you know, my three-year-old's career will change
many, many times throughout his life. It's very hard to predict
what skill is going to be most important, but I can bet you that
that passion for learning, that curiosity, um that openness to
different experiences um is is going to be number one. But the
other thing I would say is you know I think you know he's he's he's
very lucky right he's he's born in a country like Australia and he
you know has has so many opportunities like like many of our
children do here but I think there's also a duty to do things that
um make a positive impact on the world to leave a positive a
positive uh footprint on on where you're going. So that's the other
thing I would encourage encourage him to do is to is to care and
try and focus his time on um areas that make an impact for
vulnerable people or for um the climate, whatever it might be, but
try and connect it to one of his passions.
Wow, what a what a way to end the podcast. Thank you so much for
sharing your insights today, Katie. We really really appreciate it.
And we'll put some links in the show notes of some of the things
you mentioned today, but thank you so much for being a guest.
Thank you, Dan. Thanks, Lily. Thanks for having me.
Thanks very much. Bye.