Paper 2: Questions

Introduction

In this second paper I will continue to review the changes in the English texts used in the first phase (TPS) of the CESPE concurso/examination. Also I will go on to analyse the question types used over the past few years and demonstrate their main forms.

First we will review data taken from the last 6 years to help demonstrate these changes and describe the kinds of questions that have been predominantly used.1. Overall data

In the first table below we see details of the number of reading texts used in English section of the exam and their size (number of words) i from 2011 to 2016.

Year Text 1 Text 2 Text 3 Text 4 Text 5 Total (average words used)
2016 459 259 307 193 206 1,424

(285)

2015 444 580 663 557 2,244

(561)

2014 651 528 504 294 1,997

(499)

2013 314 212 204 488 422 1,218

(244)

2012 388 600 208 345 1,541

(385)

2011 213 323 245 781

(260)

Table 1: TPS Text sizes

From table 1 we see that over the past 5 years the TPS English section has included either 4 or 5 reading texts. While the size of texts had been gradually increasing, this year’s exam (2016) fell back to 5 shorter length reading texts similar to 2013. From this information we can deduce that 13 questions in 5 texts means the texts are general shorter while the 4 texts for 13 questions demand bigger/longer texts. While this is generally true, in paper 3 I will comment later on how the 2015 exam used longer length texts as it was specifically influenced by the texts’ genre.

Table 2 demonstrates the changes in questions types in the last 6 years. There have been only two different questions forms used on the tests i) True or False, and ii) Multiple choice questions

Year True or False Questions Multiple Choice Number of Question
2016 13 0 13
2015 13 0 13
2014 13 0 13
2013 4 8* 12
2012 6 6* 12
2011 6 6 12

Table 2 Question types

  • Includes the question a version of the same excerpt. Choose a version of the same excerpt which has retained most of the original meaning (see paper 3)

In table 2 the first significant change happened in 2014 when the number of English questions in the exam increased from 12 to 13. As this section of the overall exam is almost 10%, this had significant consequences those taking the exam.

The second change is in the number of True or False and Multiple choice questions The weight of these questions used to split between these 2 different question styles up to 2013. However, for the past 3 years the examiner has stopped using Multiple choice questions and used only True or False forms.

2. Questions forms

We see today that all of the English questions contain four individual statements that should be evaluated as either being ‘True’ or ‘False’. Here we see a typical sample which will be analysed further in this paper.

Sample questions

In reference to the text, decide whether the following statements are right (C) or wrong (E).

1 The hierarchical structure of the diplomatic services in the 21st century is remarkably different from that prevalent in the previous centuries.

2 In the first paragraph, the author presents the main ideas he collected from “Most of the recent scholarly works” on which his argument is built along the text.

3 The text presents an opposition between club diplomacy and network diplomacy, which are different and irreconcilable ways of settling international conflicts.

4 Discussions about inclusiveness and diversity in diplomatic circles have led to the expansion of the power of some countries.

The most important area of the above question is the instructional sentence, candidates have a tendency to overlook this information so let’s review it in more details

3. Stem Questions

Before the 4 statements the question starts with an instructional sentence which is called the stem. These are worded differently but their meaning is basically the same. All ‘True or False’ stems in the exam include three main forms of wording

i) a reference to the whole text or area of the text,\

ii) the request to choose 2 options and..           

iii) True or False.

The principle language used by the examiner over the last few years are as follows:

i) reference to the text

In 2016 the descriptions are clearer than seen in the previous year:

           Considering [the text]/According to [the text]/ In the text

In 2015 the exam has more literary genre texts and this caused some stems to be more specific to a person or an individual subject/theme in the text. These items included language such as:

            In relation to [the text]/according to [the author/text]/in relations to /in reference to..

In 2014 the examiner uses just 2 main referencing items:

           Based on/Considering the information

ii) the choice of 2 options

Before 2015 the phrases used in the stem sentences were:

Judge if… or Decide if…,

In the last two years the language used was: only

            Decide whether….

Since whether is more preferable in formal written forms that require only two options, then the examiner changed the wording to improve the language in these past 2 years.

iii) True or False

While all of the question are True or False, because this exam is set by a Brazilian exam board, instead of the words ‘true’ or ‘false’, the exam uses ‘right’ or ‘wrong’  with ‘C’ meaning ‘Certo’ in Portuguese, to indicate right and ‘E’ for ‘Errado’ for wrong’. ‘C’ and ‘E’ are marked on the separate answer paper the candidate must hand in at the end of the exam.

A typical sentence stem then looks like this

  1. i) In reference to the text, ii) decide whether the following statements are iii) right (C) or wrong (E).

4. The main questions types

As multiple choice questions have not been used in the exam since 2013, they will not be analysed in this section. As previously seen in table 2 the dominant question seen in the last 3 exams test True or False and these fall into two areas;

                    a) text comprehension and

                    b) meaning/synonyms

4.1. Text Comprehension

As shown in point 2, all of the questions the candidate contain four different statements which must be evaluated as correct or not based on the reading text. In the typical sample question below CESPE uses statements that requires the candidate to look at the reading in three slightly different ways. These 3 different ways call on the candidate to use either micro or macro- reading skills. Note:  I will explain these skills more in paper 3 when the reader will see a more in depth and technical evaluation of the statements.

Sample questions

In reference to the text, decide whether the following statements are right (C) or wrong (E).

1 The hierarchical structure of the diplomatic services in the 21st century is remarkably different from that prevalent in the previous centuries.

2 In the first paragraph, the author presents the main ideas he collected from “Most of the recent scholarly works” on which his argument is built along the text.

3 The text presents an opposition between club diplomacy and network diplomacy, which are different and irreconcilable ways of settling international conflicts.

4 Discussions about inclusiveness and diversity in diplomatic circles have led to the expansion of the power of some countries.

Explanation:

i) In No 1 and No 4, each statement requires the candidate to find specific information within the text. In testing this is called the assessment of the candidate’s micro-reading skills

ii) In statement 2, the candidate must have a general understanding of an area or section of the text, more commonly, a paragraph. In testing this is called the assessment of the candidate’s macro-reading skill

iii) Statement 3 see below requires the candidate to have a clear understanding/idea of the whole text. In testing this is called the assessment of the candidate’s macro-reading skill

Therefore, we see the testing of specific, overall and sectional understanding of the texts demands the candidate use both main reading strategies to successfully answer the statements.

4.2. Vocabulary

Secondary to comprehension statements (point 4) the exam uses ‘lexical-based statements’, which focus on the meaning of vocabulary from the readings (see paper 3 for more details on this). These we will call ‘vocabulary statements’ and are either mixed together with some comprehension statement or, as we will see in the following examples, they form their own separate section/question.

The typical example used in many past exams focuses on different vocabulary meaning aspects

Example 1 (TPS 2015)

About the vocabulary the author uses in his text, decide whether the statements below are right (C) or wrong (E).

1 “umpteen” could be correctly replaced by torn.

2 “cluttered” (is synonymous with scratched.

3 “prospective paintings” can be understood as paintings about which Bacon was still thinking or planning.

4 “took delivery” means received something that has already been paid for.

Explanation:

On further analysis the vocabulary statements above are formed by into

  1. a) single word synonyms
  2. b) similar meanings of lexical items (phrases from the text) 

a) In statements 1 and 2 above, the candidate has to decide whether the alternative word is a straight substitution for the original word in the text – a synonym which is acceptable in the context of the text.

b) Statements 3 and 4 demonstrate lexical phrases where the examiner is also requesting a similarity in meaning.

Now we see a slight change in the a very similar question in 2016.

Example 2 (TPS 2015)

Each of the fragments from the text presented below is followed by a suggestion of rewriting. Decide whether the suggestion given maintains the meaning, coherence and grammar correction of the text (C) or not (E).

1 “At odds with” As bizarre as

2 “make the necessary trade-offs to allow deadlocks to be broken” : strike a compromise as a way out of an impasse

3 “to encompass a myriad set of issue areas” to comprise a vast range of fields of interest

4 “To showcase this phenomenon, however, is no to suggest ossification” (Highlighting this fact does not amount to acknowledging stagnation

In the second sample, these statement all include lexical items and this shows a growing change, which we will see in a later study, of the examiner moving away from single word synonym style statements. A more form-focused assessment must also be highlighted as the stem requires the candidate to consider the ‘coherence and grammar correction of the text’. Therefore, the candidate needs to be aware of the correct structure, such a tense and other grammatical difficulties, as well as the possibility of poor text organisation.

While meaning is the main area being evaluated, the grammar and coherence makes the degree of information required by the candidate to be slightly more difficult.  While the grammatically correct forms of single words always had to be considered in past exams, the fact that we are using almost complete phrases increases the difficulty considerably

In 2016 the exam contained a slightly different form of question which included the same vocabulary criteria as previously above. In example 3 and 4 below show the questions used and at first they look as if they are Multiple choice questions but they are in fact constructed as True or False in their form. In example 3, single words are tested and the second question has lexical phrases

Example 3 (TPS 2016)

In text I, without altering the meaning of the sentence, the noun “realms” (R.35) could be replaced by (mark right — C — or wrong — E):

1 spheres.

2 domains.

3 grounds.

4 divisions.

Example 4 (TPS 2016)

In text IV the expression “deferred gratification” (R.5) could be replaced, without changing of meaning, by (decide whether the items below are right — C — or wrong — E):

1 expected gratification.

2 generous expectation.

3 paid-off expectation.

4 put-off gratification.

While these vocabulary statements still fall into the same question types discussed previously, there is a possibility of a number of meanings being correct so there are multiple correct/incorrect answers for the vocabulary/lexis in the stem. This was the main question difference in the 2016 exam from the traditional vocabulary questions in examples 1 and 2 in this section.

5. Comment

From the first two papers we now see that this formalised assessment has been mainly shaped and changed over the past 4 years. Gradually we the TPS CESPE exam has developed towards a limited form of testing question. The change to use thirteen True and False question stems simplifies the testing process for the examiner and as a result for the candidates. However, the use of 4 or 5 texts is still the main doubt for the next exam. This is perhaps, as we will see later, is influenced by the genre of the texts.

As we move into a more detailed technical description in the next paper (paper 3), the question statements with be analysed in even more depth including a critique on some of individual questions found in the exam. I will touch on the face validity of previous year exams in order to demonstrate what, from a diplomatic point of view, could be considered the best exams so far and move onto include some older style questions that were used in the past but can also be found in other CESPE exams. Lastly for those candidates reading this I will suggest what study strategies should be used to prepare for the TPS exam.

For candidates who want to review study materials for the TPS, first check to see if the texts used are similar to those sources mentioned in paper 1 and focus on themes related to high level diplomacy. Should the material include a large amount of Multi choice questions with single word tests then it is more likely that the author is still focusing on CESPE exams for other ‘concursos’ and has not been able to update their work to comply with this ever-developing and extremely high level MRE ‘concurso’. Studying texts and not grammar is the main way to improve and focus on this first stage.

RIO BRANCO TPS 2017



Deixe um comentário