Bloom and assessment

I had an ‘ahha’ moment at 3.30am early this week. I have been thinking about students assessment for a long time ( a couple of years actually) and have been an active Bloom devotee and used Bloom to plan for higher order thinking.

As I am an ICT teacher, I teach years 7-12. We use the subject to teach students (7-10) to use ICT across the curriculum using a variety of ICT skills. For example, in year 7 we teach students how to make digital stories using Storyboards, Flash and Photostory. Storyboards are the thinking and planning tool to organise their stories.  While Flash is not strictly a drawing tool, it is a good introduction to a web tool that comes in handy later in web development. Photostory is a great tool to make still images appear to move and can highlight parts of the image. Photostory also allows audio transcription and background sound. So there are many components to learn about before the final task reveals a compiled WMV movie.

I’m a great believer in Bloom, particularly the original taxonomy. When I set a task I am always aiming for higher order thinking - the analysing, synthesising (creating),  and evaluating. I explain to my students the Bloom levels before I start a unit of work. I explain the different levels and the type of thinking used at each level. I explain the difference between lower order and higher order thinking and show students how they can use this to achieve higher marks. I remind them of this taxonomy often as it is a good way to help them differentiate between various work examples. This taxonomy is translated into a marking scheme and it goes something like this;

Bloom level	Marking scheme /10	/5
Knowledge / understanding	0 - 2	1
Comprehension	2 - 4	2
Application	4 - 6	3
Analysing	6 - 8	4
Synthesising	8 - 10	5
Evaluating	8 - 10	5

When a task is set, such as developing a storyboard, students are given examples to show what to do and what is expected. The students that can successfully apply this are marked at the application level. Students who have deconstructed or analysed the structure and format and have demonstrated that, are marked at the analysing levels. A complete reinterpretation would indicate a student synthesising the task.

I use Bloom in this way so I can have a level of consistency and reliability to my grading scheme. I find it particularly helpful when I have a number of developmental exercises leading up to an assessment task. I want my marks to indicate the level my students are thinking at.

My ‘ahha’ moment came when I realised that this could be extended to a rubric that not only describes learning but would also help teachers develop their units of work and assessment tasks.

Remember Understand	Knowledge and concepts are not recalled, remembered or understood	0	Partial recall of knowledge is achievable and basic concepts are understood	1	Recall of knowledge is good and concepts are understood	2
Apply	May Partially remember and understand concepts but is unable to use them to solve problems	0	Some problems are solved by applying acquired knowledge, facts, techniques and rules.	1	Solves problems by applying acquired knowledge, facts, techniques and rules in a different way	2
Analyse	Able to demonstrate a degree of problem solving but cannot differentiate or deconstruct problems.	0	Breaks information into parts, identifies some structural make up, makes some inferences, finds evidence to support view.	1	Examins and breaks information into parts, identifies motives or causes, makes inferences, finds evidence to support generalisations.	2
Evaluate	May demonstrate some level of problem solving but cannot say why. cannot articulate an opinion	0	Limited presentation and defence of opinions in making judgements about information. Some validation of ideas or minimal quality of work based on a set of criteria.	1	Presents and defends opinions by making judgements about information. validates ideas or quality of work based on a set of criteria.	2
Create	May be able to defend a point of view but cannot propose new solutions	0	Insufficient compilation of new  information. Some compilation of elements in a new pattern or partially proposes alternate solutions.	1	Compiles information together in a new and different way. Combines elements in a new pattern or proposes alternate solutions	2

When these are aligned with a model like SAMR it is fairly simple to put a digital context to it. I am hoping to present this to our Curriculum Executive as an aid to course planning.

Ding!

I had a major light bulb moment yesterday.

For years I had been aware of the seemingly unique problem I had with introducing technology to our staff. There was not necessarily a reluctance but more of a ‘why should I bother’ attitude. They seemed tired of continual change of the advancement of pedagogical twists and turns and how this was reflected in teaching and learning. We tried improving our literacy results by introducing a rigorous planning tool which was, on the whole, not used successfully. It was as though teachers would use it, document their curriculum and then proceeded to do their own thing. The transition to a 1:1 netbook is equally being met with a similar passive resistance. The school mandates the purchase but teachers are not using them in the classroom. It’s easier to use the textbook. One senior teacher remarked to me that there was no use using them in year 11 and 12 because of the written exam.

Probably all this is not new. I imagine schools all over the world are wrestling with similar issues. My problem is that I didn’t see the wood for the trees. I’m like that frog in the tub that’s gradually heating up and can’t escape until it’s too late. It took a new person who has just come into the school to give me some fresh eyes. The issue is that our school is made up of 57 different language groups and the predominant cohort is Asian, that’s not the issue. The major awakening I had is that our student body can be generally described as compliant. Sure, we have our difficulties but generally we don’t get boo out of the kids. I have witnessed this first hand. In some of my courses where students need a step by step (scaffolded) approach to learning, such as a piece of complex software where I put them through a series of videos I have made specifically for the project. These are interspersed with structured activities. When the students are engaged with these, honestly, you can hear a pin drop. It’s spooky. But is this good learning? No, I don’t think so. It’s the ultimate technological sage-on-the-stage.

So, what have been the results of teaching compliant students school wide? Well, for one thing teachers are not challenged to change the way they teach. For them the easy way is to work  through the textbook and assess on that learning. Good old traditional teaching. Never fails, and  students don’t seem to mind.

We are not disparaging. We have a plan. We aim to get our student body motivated to speak up through workshops, surveys and face to face discussions about how we can improve their learning experiences. We will keep putting research and pedagogical argument in front of the staff and push for change not once or twice but constantly. I’m not going to win this, we are.

Digging Into Data

I want to share some data digging I have done recently.

I was interested to see the inquiry our English domain had done on their year 12 exams. Basically, they looked at all the verbs used to introduce short answer questions such as ‘explain’, ‘describe’, ‘analyse’ and so forth. They then did a lesson on all those words to help their students better understand the requirements of the final exam.

I thought it was a good idea so I went through all the IT Application exams from 2007 - 2014 and made a list of all the instructional verbs. You can see the raw data by clicking the link below.

https://docs.google.com/spreadsheets/d/18kvZc2hcXl6CeYTgGsnzUg-CA43FJLPZDfCCzdtp-B0/edit?usp=sharing

Here’s a list of the words and the number of times they occur. It’s easy to see the most popular words but keep in mind the chart represents words used inclusively over the last seven years. The picture changes when we focus on the list for the last two years.

                   

‘Explain’ and ‘recommend’ still lead the way but the list is noticeably shorter.

I then decided to match the verbs against Bloom's Taxonomy and chart it. This is what it looks lor the years 2007 and 2008. Note the missing analysing section.

And now compare it with 2013 - 2014

So what do I learn from this? It looks like there is a swing away from lower order thinking (remembering, understanding and applying) to more of an emphasis on analysing, evaluating and, to some extent, creating. This will help me direct my teaching. I can now help my student better understand the exam requirements and also focus in on their skills of expressing their higher order thinking.

Communicating with staff about change

I have been attempting to lead staff into a change in thinking about the use of technology in teaching and learning. Below is the content of an email I sent in response to a request for some web links to use with students in the classroom.

---------------------------------------------------------------------------------------------------------------------

I think it’s a good idea to have access to as much technology as possible in order for you to make useful and appropriate decisions about your student’s learning. This includes the software, apps and web resources you use with your students. However, how these resources are used is up for discussion. As the picture on the left shows, it’s not merely using the technology that’s important, it’s how it is used to promote thinking. That’s the issue. I should also point out I don’t necessarily agree with the ‘right answer’ ‘wrong answer’ classification but I certainly agree with the conclusion - ‘Technology is a tool, not an learning outcome.’ To help you understand this better I include the SAMR (Substitution, Augmentation, Modification and Redefinition) model below. It shows the various levels where technology is used and the thinking it promotes. There is nothing wrong in making pezis or posting to Edmodo at a substitution level but in order to aim at modifying and redefining learning events we need to focus on engaging students at a greater level of inquiry.

For some of us, we are going to do some major thinking around the how, why and what we teach. I know I have. Dan and I have been submerged in a Bastow course ~ Leading Schools in the Digital Age and now we are bringing many of the concepts we have been working through home to Wellington.
Have you had an opportunity to view the examples of student’s portfolios yet? I include them again for your information.  They are not complete. Students need you to prompt them to reflect on their learning and upload some examples of their achievements. Here are the examples of some students ePortfolios for 8K I have previously posted. I have their permission to distribute their links Megan ROSS https://sites.google.com/a/wellingtonsc.vic.edu.au/ros0021eportfolio/ Dale MANANDIC https://sites.google.com/a/wellingtonsc.vic.edu.au/man0034-e-portfolio/ Victoria BANH https://sites.google.com/a/wellingtonsc.vic.edu.au/ban0024-eportfolio/ Here are some useful links which you may find helpful.

Cartoon strips and animations

Go!Animate Takes storytelling using online cartoon strip makers a little bit further by allowing students to spark life into characters and create short animations. A super tool to foster creativity and engage students.

Xtra Normal For creating text to speech animated movies. A useful tool for social development.

MakeBeliefsComix Encourage writing, reading, and storytelling. With this tool, students can create online comics.

Hot Potato An easy tool that’s free to educators. Build your own crosswords, matching exercises, short answer, multiple-choice, and jumbled sentences. Makes homework prep a breeze.

ToonDoo A web application that allows students to create their own comic strips. Highly motivational, it allows students to express themselves in a more creative way.

I'm have a great time in this LSDA course. Getting into Twitter has been an eye-opening experience. I'm now connecting to some really great thinkers and it's affecting me. Take today for example,  one of my friends tweeted this http://www.edutopia.org/blog/hands-off-teaching-cultivates-metacognition-hunter-maats-katie-obrien and it has had a great impact. My first response has been to send the article to every staff member in my school in the hope of prompting some discussion. I'm sure it will because I won't be able to keep quiet about it.

Metacognition has always something I have been drawn to and now I see it as a key to unlocking productive learning. I must confess after reading the article I realise that I am a learning hog. I need to encourage student emulate some of my practices.  As the article says ...
"Teaching is hard work -- you have to be constantly engaged and aware of your process and how to improve it. That's exactly what makes an expert learner. So share the wealth! If you really want your students to be better learners, then let them walk a mile in your shoes."

The Learning Curve

Here's something I have always wondered about - a description of learning. I found this HERE. Basically it goes something like this, as told by Tim Ferris;

“Whenever someone first begins learning a new subject or skill, there will be a period of accelerated learning that brings a very satisfied feeling of learning in a very short amount of time. This part of the learning is related to the concept discussed in my previous post about the 80/20 rule, in which 80% of the material can be learned in 20% of the time, which explains why so much is learned so quickly in the beginning, making the learner feel very confident.

Shortly after learning the basics of a new language, skill, or subject, comes a point where a person begins to realize how difficult a new skill actually is, and has run out of the “beginner” material that is simple concepts and memorization. Additionally, at this point, the person realizes that they are no longer learning as quickly as they were before, dropping their confidence and morale a little bit. Regarding languages, this is the point where the person begins creating their own sentences and thoughts in the new language instead of using simple canned responses.

At some point later, the person’s learning confidence hits rock bottom, and the brain begins neurally adapting whatever it is they are learning, pulling it deeper than simple surface level memorization, working to allow the brain to do less thinking to accomplish the same tasks. It may be muscle memory or habit formation.

The graph then plateaus out to a place where the person is still using effort to learn, but it feels like they are not learning as quickly as they did in the beginning.

Then eventually, the person reaches the inflection point, which is casually referred to as the “click”, and the learning becomes easy and accelerates the person to fluency, or proficiency.”

I come across this manifestation quite often not only in the classroom but in training staff with various approaches with software such as our Learning Management System. They may understand it’s a good idea to use it but coming to terms with it’s complex structure puts them off and often they give up (Just email me the work!) I wonder where the breakeven point is where the desire to know something overcomes the issues of actually learning it. Obviously this is not a problem for Mitra’s learners, or maybe we never hear of the failures because they just drift away.

Fullen in his paper says that technology should be easy to use and intuitive, irresistibly engaging and elegantly efficient and incorporate latest design principles for user experience. In the best innovations, digital tools are participatory, engaging, co–creative, and collaborative. It should not, on the other hand, lack engagement for the learner. The user experience and design elements should not feel heavily dated. I essentially agree with him but it’s not always that simple and no amount of careful scaffolding appears to help.

I remember a story of a young boy who had found a moth cocoon. He had placed it on a table and was intently observing it. The moth had started to emerge from its cocoon. The process was labouring and the moth struggled. The hole it had made for itself seemed too small and it couldn't get out. The boy felt sympathy for the moth in its struggle to emerge. He suddenly had a thought. He grabbed his pocket knife a slit the opening of the cocoon where by the moth easily gained an exit. But something was wrong. The moth could not use its wings. The reason for the struggle was to push blood through the its wings to make them functional.

The connection I’m trying to make in all of this is that learning can be hard but perseverance is an admirable quality, we should not be put off by difficulty. We need to build resilience in our learners. Failure is an option.  

Sugata Mitra: Build a School in the Cloud

This is an interesting concept because it demonstrates unfettered, uncontrolled learning in the context of poverty and ignorance. Given the opportunity, particularly children with curiosity, will use the technology to learn. Mitra wants to build schools with this approach to learning and only wants Granny’s to staff the school.

In my thinking I now have a spectrum to guide my approach. If I think ‘middle of the road’ I will tend towards an innovative school approach. If I move right of this, towards Mitra’s school of the cloud, I would probably meet up with some innovative home schoolers. No doubt I would like to visit Mitra’s school just to see how it works because it is totally beyond anything I could conceive working as a viable educational institution. What about assessment (formative, of course)?