- 5 de dezembro de 2017
- Posted by: Shaun Dowling
- Category: CACD, CESPE
While recently trying to analyse the results of the 2017 CESPE IRBr exam, the summary section of the 2017 concurso not only confirmed my initial concerns when I saw the question but, on receiving the ‘espelhos’ results from my students, further problems with this years English exam have arisen.
I was previously concerned that the summary question did not have a word limit as this could open the exam to many problems in marking and text interpretations. This would be, of course, unfair on the candidates. It is always good practice to put a word limit so the examiner can see and justify that the student has included all the relevant summarised information and not included unnecessary extra information unimportant to the main idea of the text.
Of the candidates I have so far sampled, all had between 180-190 words while the examiner’s suggested answer uses 231. As every year and always on our courses, there was a 200 word limit it was wise for the candidate to come below this previously used word limit. I can see no indication that summarised were penalised for word usage, as there is no indication as such on the ‘espelhos’ answer sheets. If anyone has been penalised please let me know.
It should also be noted that this year’s text was much shorter that previous years, which is another criticism, the examiner used a text with only 942 words when previous years the texts were around 1200—1250 words. The size of the summary can be proportionated to the size of the text.
In past years the examiner’s marking was a little clearer than this year and he/she would write 6 content areas that were given a score. This was presumable the 6 messages that should be considered and included in the summary. This gave 12 points and there were 3 extra points for the overall writing of the text making a total of 15 points. Here is a sample:
Image: Sample of 2014 summary marking
This year there was no such correction or comment by the examiner this caused grave concerns as we shall see below.
Here are two samples of two students from this year’s third Phase. I chosen them as they have a very similar score. In order not to cause further criticism I will not deal with the first summary criteria ‘capacidade de síntese e concisão’ here.
Also, not wishing to show the handwriting of each candidates, I have typed the two texts and you will see I have underlined the obvious problems with mistakes and language appropriacy. Note, the examiner showed no marking on the returned texts and underlining this is done to help the reader.
Here is the score of this first summary:
The candidate scored only 2 for ‘correção gramatical e propriedade de linguagem’ and an overall score of 6.
Here is candidate’s summary:
The world faces quick changes, which stem from two tendencies in the case of soft power. Firstly, there is an increase in the role played by networks, due to the redistribution of power mostly from Western to Easter countries. Webs are more important also because states nowadays share responsibilities with other players. Furthermore, as the majority of the world(‘s) population is urban, cities have enhanced participation, posing a threat to the nation-state exclusivity. Since networks are easily formed and they allow for the cooperation of various actors in multiple subjects, they propel international transformations.
Secondly, the use of online tools by almost 50% of the world(‘s) population for business and leisure fosters global changes. An accelerated spread of information has led to both claims for expanded access to knowledge and the empowerment of individuals. This process renders governments less able to manipulate data depending on the public. However, states are combating this openness with fake accounts and misleading propaganda. Despite being a source of soft power and resources especially in the West, as a consequence of budget cuts. This trend conflicts with the growing important of soft power.
Not many problems here and as I use online tools of readability to check work, the use of language is of a higher standard than that of the examiner. :).
Let’s look at the difference in the mistakes and language appropriacy that I have underlined above with the second sample below.
Here is the 2nd candidates similar score. Again the sample scored 2 for ‘correção gramatical e propriedade de linguagem’:
The candidate wrote a very similar text but in 3 paragraphs:
The notion of soft power have been gaining more importance in the contemporary world due to the transformations in the geo-political environment. Hence, there are two paramount global trends which arise new challenges to ancient nation-states world order.
Firstly, the rise of network is forming a completely different global outcome. The increasing of complexity in worldwide networks has three main reasons (such) as the emergence of a multi-polar order – caused by prior diffusion – the failure of traditional power hierarchy, and the rapid and widespread process of urbanisation. These transformations are challenging the common notion of sovereignty.
Secondly, the ongoing digital revolution is empowering individuals as never before. Almost half of the world’s population is already connected online. Consequently, the flow of information is rapidly increasing thus it is allowing more people to becoming aware of the news and more active online, which is evident by the great number of online campaings from the last few years that encompasses people from a myriad of nations. Therefore, the notion of soft power is becoming even more relevant for states to achieving their foreign policy goals. Conversely, government expenditure on propaganda is decreasing, making it more difficult to adapting to the new era
Despite scoring exactly the same points (2) there is a clear difference in mistakes and appropriacy, although the level of the vocabulary in both is not too different. Without even analysing the coherence and readability of the texts the first one clearly deserved a high score than the second due to the number of mistakes.
Giving the examples of other students whose work I have reviewed, this first candidate should receive at least 8 points and not just 2 for this evaluation.
The conclusion we can see is that the difference between these two summaries is clear, and the lack of transparency of the examiner causes an undermining of the scores given for this area of the exam.
In previous years both the word count and the examiner’s own correction on the exams helped avoid possible controversies. Unfortunately, this year (2017) there is grave concern that the system of evaluation and scoring is flawed.