Apr 1, 2020

Mark Stanley from Literatu meets the podcast in our new online-only studio to talk all things Machine Learning and Artificial Intelligence.  We look at how data generally, and NAPLAN data specifically, is used to predict outcomes and how cutting edge Machine Learning can be used to improve writing skills for students, at any level - from primary school years up to undergraduate level.

For more information on Literatu and Scribo: https://about.literatu.com/

________________________________________

TRANSCRIPT For this episode of The AI in Education Podcast
Series: 2
Episode: 13

This transcript was auto-generated. If you spot any important errors, do feel free to email the podcast hosts for corrections.

Hi Mark, welcome to the AI and education podcast. Thanks for joining us today.
Hey Dan, it's a pleasure. It's good to talk again.
Yeah, no absolutely. So just for the listeners of the podcast uh today who might not know you, can you tell us a bit about yourself and then what leds your current role? Sure. Yes. I'm the uh founder of Literatu, which has been an education platform for about 5 years in Australia and uh internationally as well. So, we've specialized in a couple of different parts of the education platform space, but mainly in teaching and learning. You know, using data to guide teaching and learning is where we've spent a lot of time working with things like Napan Explorer or the NAPLAN data, the PAT data, all well, all that diagnostic data that schools collect. And recently we've opened up our scribbo platform in the last couple of years which is a really interesting artificial intelligence and machine learning platform that helps teachers to help students to write better. Uh we see that writing is a huge skill that's not growing fast enough critical for a digital future. So it's a bit of the background as to where the technical sides come from.
Big big ticket items there then.
Yeah. Yeah. Data data data. I think it's you know I think it's just so important to be guided by what's happening. Yeah. Um when you just interestingly the points you brought up there about Napan for example that is obviously a huge amount of data and I think when I first started talking to you many years ago when we were having those conversations about well there's this plan data what do we do about it that was really interesting that opened my eyes a lot based on the work that you were doing at the time because there was a lot of data there schools had access to but I had no context. to it.
Yeah, I think I think that's a real tragedy with with data analytics is that is that as soon as you get a spreadsheet full of data, everybody wants to graph it. You know, everybody wants to put a a line chart or a bubble chart or something around it and it starts to uh it starts to sort of take people into directions where they have to interpret the data themselves. I think you know good data should speak to the audience and so we tried to come at it a different way. We came at it from not a comparative schools way with with data, but more of an allowing teachers to internalize the data in the context of their own classroom, which is a real challenge for a teacher who's on the hop. You know, you got five five classes potentially in a high school of 30 students. You've got to have where they're at in the back of your head the whole time. We tried to uh to just bring that forwards in a very uh focused way around their classroom as opposed to the whole cohort and school.
So, the data you were using there, so before you even got to the AI and the machine learning part, you had to do I suppose quite a lot of data cleansing work or or data what would it be kind of tagging before you even got there.
Yeah. So the whole trick to any sort of machine intelligence is to have really reliable data and I think that's one thing that the nap planan people and the pat people do well is that is they put everything in one place a standard way every time but it's the it's the underlying metadata that makes the biggest difference. Like when you say someone didn't uh answer a question well, you've really got to delve into what that question was about to be able to understand where the skill gap is. So, there's a lot of meta relationship lifting that we did around the data to to put more meat on the bones rather than say you got six out of 10. Uh we can tell you why you got six out of 10.
Yeah. And that's really important, right? Yeah.
Yeah. Well, you know, you're kind of left hovering if you sort of if you work on the law of averages, you know, six out of 10's possibly above an average of five uh that might be enough but you know the next year what we see is the gaps that students develop in year three uh reemerge in year five and year seven and in year 9. It's a it's a consistency that is a bit scary because if you don't understand where the shortfalls are in primary they will reappear in secondary almost you know 80%.
And looking at your the insights that you found doing all the work you done and the schools you were working with around that project and and program what what were the most interesting insights that came out. What did you find?
What we found is that patterns kept reappearing. Uh we also we also found that over eight years uh schools didn't really change that much in the learning growth. So where they started 8 years ago in that plan in say year 7, they were still at the same um distance point. Yeah. 8 years later. So pedagogically whatever the school was doing uh wasn't wasn't lifting the cohort, it was maintaining the cohort. And in particular writing, you know, the The biggest Australian sort of problem everywhere is writing. Writing's been going backwards for eight eight years.
So kind of makes you think, you know, how how much technology has changed in 8 years, but how many outcomes haven't changed in 8 years? It kind of you wonder wonder what it's going to take to sort of drag it across the line and start moving moving north again.
Absolutely. Absolutely. So So before we look at the I'm really really interested in the literacy side of this because this this really is profound when we start applying machine learning to that. But but before we get to that part. So when you do a project and the the things you've been doing around data and insights, what was your step then you know as a founder and just for our listeners you know the schools and business decision makers inside schools you know they embarking for the first time on AI and machine learning for example themselves or with partners like yourself to help them. What's the next step to start to utilize AI and machine learning? How did you how did you go through that process? process of okay well I've got the data now I'm going to embark on AI and ML and and see what we can do what was your process for that
yeah well it's a long process it's not a short one at all so so really I think what a lot of people do is they say uh they say the acronyms AI ML it should actually be the other way around ML AI because it all starts with data really well ststructured data and that's something that the schools don't have um they've got lots of data you know that I I love quoting that famous uh that famous poem data data everywhere and not a drop to drink right they've got they've got SIS they got LMS they got all this data they got no consistency between the data sets so they tend to latch on to things like NAP plan and PAT quite strongly because it's such a a structured data set so the first trick we had to kind of come to was was work with schools to standardize a disperate data into a shape that then could be used to process and and relate data points through a machine learning process. So once you have structured data, you can start machine learning it, which means running all sorts of algorithms across the data, looking for correlations and hotspots and looking for growth, you know, between between years and across years. I mean, so many interesting things, but if you've got holes and gaps in your data, it's a real problem and you don't really achieve the result you want. So you have to go through the data cleanse, the data load, the data you know, sort of framework. Once you have that as a standardized shape, you can start doing some machine learning for lots of schools. And then when you get to a correlative state in data where it's quite strong and you can reasonably see that the machine can use that data to make a decision as to what happens next, you can start automating decisions from that point on using the AI components.
So the intelligence is almost the last thing that happens, but it's quite often the first thing people talk about.
Yeah. That's really really important. So, you know, you've then applied that to the the data of an app plan in in in some of your work previously like you just mentioned, but then now you've augmented that into actually looking at writing styles. Can you tell us a little bit more about that?
Yeah, so we we always wanted to to help uh students write, you know, better. I mean, I used to watch my guys come home from school with a with a blank sheet of paper and instruction to write a story. And it was kind of like really challenging then. It still is now for lots of kids is How do you start to unpack the metacognitive skills that they've got onto a single focus point which is a piece of paper? And the only way you can improve writing is is through practice. And practice unfortunately means teachers have to give more feedback. And more feedback obviously takes a lot more time. So when you talk to humanities teachers they're you know they're up against a wall of words or what I call a bag of words. You know they sort of go home with 30,000 you know 30 essays thousand words an essay 30,000 words uh we expect those teachers to consume each word and understand exactly how they can come back into class and help each student and it's just an impossible task really. It's an impossible ask.
Yes.
Absolutely.
So we decided to sort of let the machine do what the machine's good at and that is discover all of the opportunities that are around to deal with natural language processing and start to really put a a very standardized machine learning model around language and making that available to teachers of any shape and genre. So, we can talk to a year three student about writing and we can talk to a university student about writing because the words that go onto the page, they don't discriminate. They're not discriminated by a computer. We tend to do the same thing all the time, which is what computers are good at. The cognitive piece that teachers bring to obviously writing and and education is they can think outside those boundaries, which is what you really need to do to improve kids. But in terms of lifting and just getting the work, you know, reading every single word, looking for grammar uh well grammar and punctuation are two hot spots but certainly the use of vocabulary keywords were they on topic did they have sentence structures that were appropriate for their age all of this sort of stuff is mechanical stuff and we've broken the back of that with scribbo and and can really do a good job on analyzing the mechanics for teachers that
that's really interesting isn't it so you've essentially what you've done is is broken down language into almost like a data schema Absolutely. Because if you look at if you look at the simplicity of writing uh and and it's it's obviously complex process, but the simplicity of it is that you know people have a vocabulary that that gets strung together into sentences of a certain type and the sentences get put into a paragraph format that's about something and so it is a mechanical process. You know, genres and styles come into it. There's a few other influences but but essentially you know the objective of every sentence is to make is to really to make the next one uh to be read. So you you approach writing from a sentence by sentence basis and uh on that basis almost all writing is is very very similar no matter what genre.
Yeah, that that's it's fairly true because I we myself and Ray always on this podcast have a bit of a kind of a heated debate about whether AI or machine learning or any type of technology can replace teachers and we always kind of change our viewpoints depending on the context because it's quite quite interesting and and one of the points for me is a lot of the stuff as mechanical as an ex-teer you know you you do have to have standardization so you know I always used to have this debate with uh one of my consultant friends who was an art adviser in the UK and the the question was always well how do you mark art you know I think something's good but somebody else thinks it's something different but underneath there there has to be a data schema because you have to give students a mark to give it p fair parity and that's the same with English you can you can read a piece of pros and there is kind of variability in the mark scheme to be able to give that creativity mark. But a lot of the time it is about well how have this se sentence been struct structured? Have they used a wide range of vocabulary? Have they reinforced the topic that they were supposed to be talking about or have they gone off on a tangent? So a lot of those things you know are mechanical and you can get teachers reading through things in extreme cases where they're literally counting or looking for the verbs and kind of making a note of that to say, "Hey Dan, you you used 20 nouns in in this piece of text. That's excellent or whatever it might be.
Yeah, look, it's it's it's a really good good point you make there. And I think what teachers are always faced with is working backwards. You know, if you get 30,000 words to to mark over a weekend, you come back to school on Monday and you're pretty much shell shocked.
But time time moves on. You know, the next KLA or the next curriculum elements in your face, you've got to keep going. If you're a humanities teacher like history or geography and something's not well written, you know, the habit is to sort of blame the English department and say, "Well, they're the guys that should be teaching writing, not me." But really, literacy is is a cross domain and a cross teacher and a cross school discipline that has to be supported. So, that's where a lot of lifting gets done by Scribbo as well is is helping humanities teachers who aren't naturally trained in the depths of the English language to help them isolate good writing versus writing that needs more support. And and really, um, as you say, after a while, after two years, we've we've really come up with some formulas that work to identify where good writing is being achieved and where it's not. And and so that's that's the I guess the beauty of it is that we can read 30,000 words in 2 minutes and say, "Here's who you should be talking to about what."
Now, we're not taking anything away from a teacher. We're just saying, you know, what happened last week? We got to move forwards on this week. Uh let's let's build a lesson plan that's relevant for this week. You
You can't fix last week. All you can do is fix this week.
Yeah. So where so um where do you see then based on what you've uh you know experienced in schools where do you see AI in education now and where do you see it in the future? Where where do you think we are now and what what's what's there to look forward to?
Yeah, I I think it's a really really um unfortunate that they called it artificial intelligence right back in the 50s.
It really should have been called augmented intelligence and I think that's where we we see the fit is that you you won't get a replace replacement of a teacher. You will never replace um a cognitive person with a machine like machine learning, AI, whatever degree of sophistication you get to, it won't have that cognitive ability for quite a number of years. So, so it's all about supporting teachers with better information. I mean, really, if I could tell you in a couple of seconds that your essay, Dan, was right off topic and your language was was rushed and and you know, not not wellformed for a year 11 student. Uh without you having to read it. I think that's a benefit. You know, you've got you've got a talking point with me straight. I've got a talking point with you straight away. I can say, "Hey, Dan, I want to talk to you about this essay. I haven't read it yet, but I don't need to." You know, because that's the the partnership between, I think, machine learning and AI and cognitive processes is really where people will start extracting the benefit when they start working together with what's available.
And I was even intervening before that teacher element like when we were looking the screen tool that you've created. When I was looking through it, the thing that impressed me as a user, I could, you know, it was given me that AI while I was looking at it as well. So, yes, it'll support the teacher, but it also kind of can can support me in certain respects as well.
Yeah, it's got it's got two sides to it. I mean, well, you know, we're both we're both got kids in education. Mine have both left it, but you know, when I used to watch them write, most the bulk of their writing was done the night before it was due, right? So, it's a really really hard time for everybody. It's a frantic time. It's the time when feedback is needed the most, but feedback is received the least. So, it's a stressful time for everybody.
That's a really good point. Yeah.
In education. So, so Scriebo sort of steps up and says, "Look, I know you're trying to write something. Let me help." And, and it basically helps unpack clumsy writing. It improves the quality of the writing. That's what it was designed to do from a student's perspective. And then it from a teacher's perspective was designed to take it a bit further and say, "Hey, listen. Yeah, this guy is completely off track. Doesn't know what a sentences, he's using them all the time. So, there's two two sort of layers to it, but I think we call it connection rather than correction. So, we see grammar and spelling that you get in Word and Word's getting really clever now. You know, it's starting to see AI coming through in Word, which is really impressive. Uh, you know, so suggesting alternatives to students and it's really quite clever. Um, but you know, you start to corral the energy and you make a connection with students so you know exactly where you can help them as a teacher. That's the mission.
Yeah, absolutely. So, the future of AI in education from your point of view as a as a a developer and as a parent and as a you know, all your different hats where where where do you see is it bright?
Yeah, absolutely. I I think you know teachers have been you know I saw a really interesting uh survey yesterday. It was on LinkedIn and it said um I I wished I'd thought of it because it had a multiple choice question. It said uh to all the principles of schools. What's been the biggest impact on education in your school in the last two years? And it had um the board uh your your heads of department or covid-19 and obviously the correct answer was co 19 right so here we here we are I mean I think there's such an opportunity for schools to say you know we can crack the egg a different way or or slightly differently just by getting teachers more into a mentoring role which I think is what's challenging A lot of teachers today, right today, as kids are at home and everywhere, they're all working in a remote environment. And so that the role of the teacher is not disciplined anymore. It's coordination, it's it's communication, it's staying in touch and connected to each student in their own space. And I think that's that's the challenge that AI can help them with because you don't have time to read 30,000 words. You have time to push a button and say, "This is what I need to know." And I think that's where AI is going to just do the lifting. It's just going to do some some critical lifting. I hate seeing teachers teaching, you know, slogging through subjects. I'd rather them be mentoring the kids about what they're doing with their subject. So, I think that was a really excellent overview, Mark. I really appreciate your time today to jump on our podcast and share some of the learnings that you've had over the last couple of years or more than that now actually um with your work in AI. Is there any finally is it anywhere that you point teachers or business decision makers to go to learn more about AI? In the show notes, we put connections into to some of the tools that you've been making recently and creating and supporting schools with literacy and nap plan, but are there any places that you'd suggest that they go to kind of start their AI machine learning journeys?
Well, you know, it's interesting. It's the the machine learning piece is all about data. And I haven't I've seen I've seen a number of schools do data quite well, but there the definition of good is a nice graph. Okay, so the first thing you have to do about if you're talking about machine learning is forget the graphics because it's pretty much a server side grunt machine. It's a lot of mathematics. It's a lot of
structuring and grinding of data and relationships. That's something I don't think particularly schools are set up to do. I think there's a there's a lot of what schools do do that can be can be standardized and and that's really why we chose writing because everyone does it uh everyone does it differently but the components of writing are very much the same. Uh mathematics is different again because you got different influences in mathematics. So so I think you've got to if you're going to do something with AI you've got to pick something that has a broad application. Uh narrow applications won't get you a broad result but a broad application uh will. So
yes
you know so so if you're going to approach it look look for data sets that run across the school not down the school and then and then structure things in a way that uh you keep things simple. What what takes you the most time now and what what could well how many hours could be saved if something came along and did all that lifting for you. Uh and and those are the pockets of hotspots in schools. I think that can really benefit from some AI discovery.
Yeah, that's fantastic. Well, thanks very much for your insights there. That's really, really helpful. So, we'll catch you around the trap soon, Mark. Thank you so much.
Hey, always enjoy talking to you, Dan. Anytime at all.