Bloom and assessment

I had an ‘ahha’ moment at 3.30am early this week. I have been thinking about students assessment for a long time ( a couple of years actually) and have been an active Bloom devotee and used Bloom to plan for higher order thinking.

As I am an ICT teacher, I teach years 7-12. We use the subject to teach students (7-10) to use ICT across the curriculum using a variety of ICT skills. For example, in year 7 we teach students how to make digital stories using Storyboards, Flash and Photostory. Storyboards are the thinking and planning tool to organise their stories.  While Flash is not strictly a drawing tool, it is a good introduction to a web tool that comes in handy later in web development. Photostory is a great tool to make still images appear to move and can highlight parts of the image. Photostory also allows audio transcription and background sound. So there are many components to learn about before the final task reveals a compiled WMV movie.

I’m a great believer in Bloom, particularly the original taxonomy. When I set a task I am always aiming for higher order thinking - the analysing, synthesising (creating),  and evaluating. I explain to my students the Bloom levels before I start a unit of work. I explain the different levels and the type of thinking used at each level. I explain the difference between lower order and higher order thinking and show students how they can use this to achieve higher marks. I remind them of this taxonomy often as it is a good way to help them differentiate between various work examples. This taxonomy is translated into a marking scheme and it goes something like this;

Bloom level	Marking scheme /10	/5
Knowledge / understanding	0 - 2	1
Comprehension	2 - 4	2
Application	4 - 6	3
Analysing	6 - 8	4
Synthesising	8 - 10	5
Evaluating	8 - 10	5

When a task is set, such as developing a storyboard, students are given examples to show what to do and what is expected. The students that can successfully apply this are marked at the application level. Students who have deconstructed or analysed the structure and format and have demonstrated that, are marked at the analysing levels. A complete reinterpretation would indicate a student synthesising the task.

I use Bloom in this way so I can have a level of consistency and reliability to my grading scheme. I find it particularly helpful when I have a number of developmental exercises leading up to an assessment task. I want my marks to indicate the level my students are thinking at.

My ‘ahha’ moment came when I realised that this could be extended to a rubric that not only describes learning but would also help teachers develop their units of work and assessment tasks.

Remember Understand	Knowledge and concepts are not recalled, remembered or understood	0	Partial recall of knowledge is achievable and basic concepts are understood	1	Recall of knowledge is good and concepts are understood	2
Apply	May Partially remember and understand concepts but is unable to use them to solve problems	0	Some problems are solved by applying acquired knowledge, facts, techniques and rules.	1	Solves problems by applying acquired knowledge, facts, techniques and rules in a different way	2
Analyse	Able to demonstrate a degree of problem solving but cannot differentiate or deconstruct problems.	0	Breaks information into parts, identifies some structural make up, makes some inferences, finds evidence to support view.	1	Examins and breaks information into parts, identifies motives or causes, makes inferences, finds evidence to support generalisations.	2
Evaluate	May demonstrate some level of problem solving but cannot say why. cannot articulate an opinion	0	Limited presentation and defence of opinions in making judgements about information. Some validation of ideas or minimal quality of work based on a set of criteria.	1	Presents and defends opinions by making judgements about information. validates ideas or quality of work based on a set of criteria.	2
Create	May be able to defend a point of view but cannot propose new solutions	0	Insufficient compilation of new  information. Some compilation of elements in a new pattern or partially proposes alternate solutions.	1	Compiles information together in a new and different way. Combines elements in a new pattern or proposes alternate solutions	2

When these are aligned with a model like SAMR it is fairly simple to put a digital context to it. I am hoping to present this to our Curriculum Executive as an aid to course planning.

Ding!

I had a major light bulb moment yesterday.

For years I had been aware of the seemingly unique problem I had with introducing technology to our staff. There was not necessarily a reluctance but more of a ‘why should I bother’ attitude. They seemed tired of continual change of the advancement of pedagogical twists and turns and how this was reflected in teaching and learning. We tried improving our literacy results by introducing a rigorous planning tool which was, on the whole, not used successfully. It was as though teachers would use it, document their curriculum and then proceeded to do their own thing. The transition to a 1:1 netbook is equally being met with a similar passive resistance. The school mandates the purchase but teachers are not using them in the classroom. It’s easier to use the textbook. One senior teacher remarked to me that there was no use using them in year 11 and 12 because of the written exam.

Probably all this is not new. I imagine schools all over the world are wrestling with similar issues. My problem is that I didn’t see the wood for the trees. I’m like that frog in the tub that’s gradually heating up and can’t escape until it’s too late. It took a new person who has just come into the school to give me some fresh eyes. The issue is that our school is made up of 57 different language groups and the predominant cohort is Asian, that’s not the issue. The major awakening I had is that our student body can be generally described as compliant. Sure, we have our difficulties but generally we don’t get boo out of the kids. I have witnessed this first hand. In some of my courses where students need a step by step (scaffolded) approach to learning, such as a piece of complex software where I put them through a series of videos I have made specifically for the project. These are interspersed with structured activities. When the students are engaged with these, honestly, you can hear a pin drop. It’s spooky. But is this good learning? No, I don’t think so. It’s the ultimate technological sage-on-the-stage.

So, what have been the results of teaching compliant students school wide? Well, for one thing teachers are not challenged to change the way they teach. For them the easy way is to work  through the textbook and assess on that learning. Good old traditional teaching. Never fails, and  students don’t seem to mind.

We are not disparaging. We have a plan. We aim to get our student body motivated to speak up through workshops, surveys and face to face discussions about how we can improve their learning experiences. We will keep putting research and pedagogical argument in front of the staff and push for change not once or twice but constantly. I’m not going to win this, we are.

Digging Into Data

I want to share some data digging I have done recently.

I was interested to see the inquiry our English domain had done on their year 12 exams. Basically, they looked at all the verbs used to introduce short answer questions such as ‘explain’, ‘describe’, ‘analyse’ and so forth. They then did a lesson on all those words to help their students better understand the requirements of the final exam.

I thought it was a good idea so I went through all the IT Application exams from 2007 - 2014 and made a list of all the instructional verbs. You can see the raw data by clicking the link below.

https://docs.google.com/spreadsheets/d/18kvZc2hcXl6CeYTgGsnzUg-CA43FJLPZDfCCzdtp-B0/edit?usp=sharing

Here’s a list of the words and the number of times they occur. It’s easy to see the most popular words but keep in mind the chart represents words used inclusively over the last seven years. The picture changes when we focus on the list for the last two years.

                   

‘Explain’ and ‘recommend’ still lead the way but the list is noticeably shorter.

I then decided to match the verbs against Bloom's Taxonomy and chart it. This is what it looks lor the years 2007 and 2008. Note the missing analysing section.

And now compare it with 2013 - 2014

So what do I learn from this? It looks like there is a swing away from lower order thinking (remembering, understanding and applying) to more of an emphasis on analysing, evaluating and, to some extent, creating. This will help me direct my teaching. I can now help my student better understand the exam requirements and also focus in on their skills of expressing their higher order thinking.

Communicating with staff about change

I have been attempting to lead staff into a change in thinking about the use of technology in teaching and learning. Below is the content of an email I sent in response to a request for some web links to use with students in the classroom.

---------------------------------------------------------------------------------------------------------------------

I think it’s a good idea to have access to as much technology as possible in order for you to make useful and appropriate decisions about your student’s learning. This includes the software, apps and web resources you use with your students. However, how these resources are used is up for discussion. As the picture on the left shows, it’s not merely using the technology that’s important, it’s how it is used to promote thinking. That’s the issue. I should also point out I don’t necessarily agree with the ‘right answer’ ‘wrong answer’ classification but I certainly agree with the conclusion - ‘Technology is a tool, not an learning outcome.’ To help you understand this better I include the SAMR (Substitution, Augmentation, Modification and Redefinition) model below. It shows the various levels where technology is used and the thinking it promotes. There is nothing wrong in making pezis or posting to Edmodo at a substitution level but in order to aim at modifying and redefining learning events we need to focus on engaging students at a greater level of inquiry.

For some of us, we are going to do some major thinking around the how, why and what we teach. I know I have. Dan and I have been submerged in a Bastow course ~ Leading Schools in the Digital Age and now we are bringing many of the concepts we have been working through home to Wellington.
Have you had an opportunity to view the examples of student’s portfolios yet? I include them again for your information.  They are not complete. Students need you to prompt them to reflect on their learning and upload some examples of their achievements. Here are the examples of some students ePortfolios for 8K I have previously posted. I have their permission to distribute their links Megan ROSS https://sites.google.com/a/wellingtonsc.vic.edu.au/ros0021eportfolio/ Dale MANANDIC https://sites.google.com/a/wellingtonsc.vic.edu.au/man0034-e-portfolio/ Victoria BANH https://sites.google.com/a/wellingtonsc.vic.edu.au/ban0024-eportfolio/ Here are some useful links which you may find helpful.

Cartoon strips and animations

Go!Animate Takes storytelling using online cartoon strip makers a little bit further by allowing students to spark life into characters and create short animations. A super tool to foster creativity and engage students.

Xtra Normal For creating text to speech animated movies. A useful tool for social development.

MakeBeliefsComix Encourage writing, reading, and storytelling. With this tool, students can create online comics.

Hot Potato An easy tool that’s free to educators. Build your own crosswords, matching exercises, short answer, multiple-choice, and jumbled sentences. Makes homework prep a breeze.

ToonDoo A web application that allows students to create their own comic strips. Highly motivational, it allows students to express themselves in a more creative way.

I'm have a great time in this LSDA course. Getting into Twitter has been an eye-opening experience. I'm now connecting to some really great thinkers and it's affecting me. Take today for example,  one of my friends tweeted this http://www.edutopia.org/blog/hands-off-teaching-cultivates-metacognition-hunter-maats-katie-obrien and it has had a great impact. My first response has been to send the article to every staff member in my school in the hope of prompting some discussion. I'm sure it will because I won't be able to keep quiet about it.

Metacognition has always something I have been drawn to and now I see it as a key to unlocking productive learning. I must confess after reading the article I realise that I am a learning hog. I need to encourage student emulate some of my practices.  As the article says ...
"Teaching is hard work -- you have to be constantly engaged and aware of your process and how to improve it. That's exactly what makes an expert learner. So share the wealth! If you really want your students to be better learners, then let them walk a mile in your shoes."

The Learning Curve

Here's something I have always wondered about - a description of learning. I found this HERE. Basically it goes something like this, as told by Tim Ferris;

“Whenever someone first begins learning a new subject or skill, there will be a period of accelerated learning that brings a very satisfied feeling of learning in a very short amount of time. This part of the learning is related to the concept discussed in my previous post about the 80/20 rule, in which 80% of the material can be learned in 20% of the time, which explains why so much is learned so quickly in the beginning, making the learner feel very confident.

Shortly after learning the basics of a new language, skill, or subject, comes a point where a person begins to realize how difficult a new skill actually is, and has run out of the “beginner” material that is simple concepts and memorization. Additionally, at this point, the person realizes that they are no longer learning as quickly as they were before, dropping their confidence and morale a little bit. Regarding languages, this is the point where the person begins creating their own sentences and thoughts in the new language instead of using simple canned responses.

At some point later, the person’s learning confidence hits rock bottom, and the brain begins neurally adapting whatever it is they are learning, pulling it deeper than simple surface level memorization, working to allow the brain to do less thinking to accomplish the same tasks. It may be muscle memory or habit formation.

The graph then plateaus out to a place where the person is still using effort to learn, but it feels like they are not learning as quickly as they did in the beginning.

Then eventually, the person reaches the inflection point, which is casually referred to as the “click”, and the learning becomes easy and accelerates the person to fluency, or proficiency.”

I come across this manifestation quite often not only in the classroom but in training staff with various approaches with software such as our Learning Management System. They may understand it’s a good idea to use it but coming to terms with it’s complex structure puts them off and often they give up (Just email me the work!) I wonder where the breakeven point is where the desire to know something overcomes the issues of actually learning it. Obviously this is not a problem for Mitra’s learners, or maybe we never hear of the failures because they just drift away.

Fullen in his paper says that technology should be easy to use and intuitive, irresistibly engaging and elegantly efficient and incorporate latest design principles for user experience. In the best innovations, digital tools are participatory, engaging, co–creative, and collaborative. It should not, on the other hand, lack engagement for the learner. The user experience and design elements should not feel heavily dated. I essentially agree with him but it’s not always that simple and no amount of careful scaffolding appears to help.

I remember a story of a young boy who had found a moth cocoon. He had placed it on a table and was intently observing it. The moth had started to emerge from its cocoon. The process was labouring and the moth struggled. The hole it had made for itself seemed too small and it couldn't get out. The boy felt sympathy for the moth in its struggle to emerge. He suddenly had a thought. He grabbed his pocket knife a slit the opening of the cocoon where by the moth easily gained an exit. But something was wrong. The moth could not use its wings. The reason for the struggle was to push blood through the its wings to make them functional.

The connection I’m trying to make in all of this is that learning can be hard but perseverance is an admirable quality, we should not be put off by difficulty. We need to build resilience in our learners. Failure is an option.  

Sugata Mitra: Build a School in the Cloud

This is an interesting concept because it demonstrates unfettered, uncontrolled learning in the context of poverty and ignorance. Given the opportunity, particularly children with curiosity, will use the technology to learn. Mitra wants to build schools with this approach to learning and only wants Granny’s to staff the school.

In my thinking I now have a spectrum to guide my approach. If I think ‘middle of the road’ I will tend towards an innovative school approach. If I move right of this, towards Mitra’s school of the cloud, I would probably meet up with some innovative home schoolers. No doubt I would like to visit Mitra’s school just to see how it works because it is totally beyond anything I could conceive working as a viable educational institution. What about assessment (formative, of course)?

“In Reclaiming Our Teaching Profession: The Power of Educators Learning in Community, Hord and Tobia (2012) outline what leaders do to generate the favourable conditions for powerful professional learning to occur:

• Create an atmosphere and context for change

• Develop and communicate a shared vision for change

• Plan and provide resources

• Invest in professional development

• Check progress

• Give continuous assistance. “  

21st Century Leadership: Looking Forward An Interview with Michael Fullan and Ken Leithwood

It is interesting to note that the Victorian Department of Education and Training’s research web site (http://www.education.vic.gov.au/about/research) lacks significant educational research papers on any forward thinking regarding a system wide approach to changes in teaching and learning. If Fullan and Leithwood have much to say about a system wide approach to 21st Century leadership and the necessity for change then our leaders are not listening, or so it would appear. On the basis of the amount of research targeted at social justice which is good in itself, there appears to be a deafening silence on how our students should navigate a digital future where change is incredibly prolific and productive communication is as mandatory as knowing how to spell.

“ … It’s a matter of striving for the best, a kind of “moral Olympics” in which you're doing better and better for the good of the students. So in that sense we are competing with Finland and that’s a good thing. Because we want to outperform them, not for the sake of surpassing them, but because we want to do better and better and because the world will be better as a result. “

21st Century Leadership: Looking Forward An Interview with Michael Fullan and Ken Leithwood

Or am I being overly critical? The Finnish educational model produces results and Ontario is keen to chase their success and claim it for their own. Have we dropped the ball somewhere? I can see the need for system wide change and that this needs to be supported and driven from the top, at a Government level. Who is responsible? How can a small-time, worker bee like me really affect change?

I like use the analogy of serving up food to students as a way of describing teaching and learning. Imagine serving up scrambled eggs on toast every day to students. Ok, so some variation is needed so I add a bit of bacon or tomato or cheese (they like cheese!) to vary the menu. I think kids would eventually get tired of the same old food. Some may even refuse to eat it. Sure we can starve them but that won't change the basic pedagogy.

Then the realisation hits… why don’t we let the kids design their own menus and eat their own food, or even share it around. That would make food much more interesting and engaging. I’m sure the kids could teach me a thing or two about food but I would still have to direct them. We can't eat steak all the time - there are constraints, but it would be much better than eggs on toast every day.

Theoretically, at the end of learning experiences with new pedagogies, students should breeze through standardised tests that measure mastery of curricular content. Of more importance would be measuring the full range of students’ deep learning competencies: 1) students’ mastery of the learning process, including their ability to master new content; 2) students’ key future skills, including their abilities to create new knowledge using the collaboration and communication skills necessary for high-level value creation; 3) students’ proactive dispositions and levels of perseverance in the face of challenges; and 4) the effect of students’ work products on intended audiences or problems. Technology can theoretically be harnessed to support all of these types of measures, but we have so far seen few clear examples of this happening. (Pg 40. A Rich Stream)

It was announced this week that in 1985 Apple introduced its first 1:1 program in American schools. That’s 30 years ago. We have had technology in our schools here in Australia for more than 20 years. I remember buying a room full of 286 PC’s for $50 each and using them in a thin client network in 1997. My question is why are we still saying “Theoretically….” Why is it taking so long for the general teaching population to understand the benefits of a digital approach to these new pedagogies? In the reading, Fullan and Langworthy (2014) puts forward the contention that new pedagogies will promote a greater mastery of student learning where students will create new knowledge, show greater perseverance and promote their work to a wider range of audiences. These are valuable outcomes which need to be established beyond theories. Am I to assume that schools that have had a 1:1 program for the last 20 years have had little to show in comparison to the old tried and proven methods of teaching?

21^st Century Pedagogy – Change Educational DNA
https://www.youtube.com/watch?v=K-Ij_ZnmAHU

Yes, Let's change the DNA of teachers. I'm all for that, but where do we start? Surely we need to start at our teaching institutions as young teachers now will have grown up in a digital environment. They should have a good idea on how to embrace deep learning rather than propagate traditional ideas. Or are traditional ideas impressed upon them once they have secured a position in a school? It is my experience in mentoring preservice teachers is that they are driven by the same constraints of assessment as everyone else. They are more concerned about adopting "acceptable practice" than attempting the type of innovation we are promoting. Changing the DNA of teachers will be a challenging task indeed!

21st-Century PLNs for School Leaders
http://www.edutopia.org/blog/21st-century-PLNs-school-leaders-george-couros

The conclusion of George Couros's article about 21st-Century PLNs for School Leaders is worth highlighting; Gaining the skills of using Twitter and blogging certainly opens ... "up your own learning to the world, you will be surprised not only how your knowledge elevates, but how your passion for teaching and learning will benefit as well." These skills makes us media savvy and able to better communicate our ideas and views. With a proliferation of digital environments it is important for teachers to become the authors and producers of media thereby participating on a more active level. At least that's a start.

What are …..

My current feelings and reactions to my experience in this course:

At the moment I feel sort of isolated. I appear to be the only one posting anything. Is this a learning community? Who am I supposed to be connecting with? Perhaps I’m missing something? The readings and video’s are engaging me and I am thinking around these. Some of these issues are not new to me such as the notion that education is built on a production line mentality. I would like to respond to this but I don’t want to cloud the atmosphere.

What I am learning and what have been the most powerful learning experiences for me:

I have always had a deep respect for Bloom but I lingered on the old taxonomy. Recently I took a concerted look at the new taxonomy and was blown away. Why didn’t I do this earlier?

The knowledge dimension	The cognitive process dimension
	Remember	Understand	Apply	Analyse	Evaluate	Create
Factual
Conceptual
Procedural
Meta-cognitive

Another thing I have learned is how the normal distribution curve applies applies to the human construct. This helps me understand why I need to differentiate my teaching and learning.

What I feel my students are learning:

I spoke to my students about metacognition today. We talked about why it’s important to articulate and think about their own thinking and learning. I did this because hopefully it will lead to self awareness and contribute to a sense of maturity and empathy. Being empathetic helps collaboration and leadership skills and makes for better learners. My desire is to keep this conversation going with students for the next couple of years.  

A significant event for me and how I felt and reacted:

Last year we tried to establish a digital learning relationship with a Chinese school. It not only didn’t work but I felt we didn’t get off first base. It turned out to be a lot harder than I imagined even though the research we read up on predicted the exact same outcome. We are still getting overtures from similar schools but I am now very wary. It took a lot of time and effort for very little result.

Issues or questions: (include professional learning needs, problems, concerns, frustrations)

Frustrations … Hmm that’s a good one. I’ll save this when I get to know you all a bit better.