Views and Positionality on Development by AI ChatBots

Context

SDC36306 - Perspectives and Themes in International Development Studies.

See the study handbook for learning outcomes and further course details.

  • Master
  • Around 70
  • Prior knowledge and skills in GenAI: all students at the beginning of the assignment had an account and therefore had at least some experience. Most of them use it for brainstorming. The student population in the course was already quite aware of the possible bias in the output of ChatGPT.

Rationale

The application of AI-generated information and knowledge is pervading academia with dazzling speed. Many scholars are deeply concerned about this development because AI Chatbots present logically structured “facts”, “arguments” and “references” to academic literature with an impressive degree of confidence, but which on closer scrutiny are often found to be untrue, utter nonsense and/or non-existing. As of yet, less attention has been devoted to the question of whether AI Chatbots also bring specific political or ideological views of the world. For the purpose of our course, we are interested in whether AI favors particular perspectives on themes in international development over others.

Educational design

This assignment aims to determine whether AI ChatBots favour particular perspectives on major dilemmas in international development that have to do with the broad theme of poverty and inequality. You will do this by pursuing the following steps:

  • Group discussion: think about a well-known example of a policy dilemma that occurs in the fight against poverty and/or particular undesired dimensions of inter-personal or inter-group inequality (e.g. economic, social, gender, race, etc.). For inspiration you may want to consult the many reports that have been published in the wake of the UN’s Sustainable Development Goals Programme (2015-2030).
  • Note: it is critical for the remainder of this assignment that you define the dilemma in very clear terms.
  • Go to ChatGPT to set up a conversation. You are requested to address a maximum of 6 questions regarding the development dilemma you have defined. It is advisable to let these questions be built partly, or fully, on the answers that ChatGPT has provided to your previous questions so that the conversation will be structured around AI-generated output.
  • Group reflection: what do you take away from this conversation? Try to structure the conclusions from your discussion.
  • Write a group report of c. 500 words in which you explain the dilemma and reflect on the positionality of ChatGPT. Where do you think it has based its factual information on? Are the answers provided to your questions complete, or are they missing crucial aspects? What elements does ChatGPT emphasize? Does it offer an optimistic, pessimistic, realistic or unrealistic perspective on the dilemma you are interested in? What are the action implications of ChatGPT suggestions? Can you discern a specific political or ideological undercurrent in the answers or would you regard these as neutral and objective?
  • Submit your report together with an appendix with the full exchange of your group with ChatGPT through Brightspace.

Good luck!

Criteria for pass/fail:

  • The group report reflects steps 2 and 3 above, broadly debating the questions raised under step 4.
  • The group report with appendix is submitted through Brightspace within the deadline.

Evaluation

The students worked really well on the assignment. They also took the liberty to test other GenAI applications and see how the output was different. The true insight in how GenAI works, and how ‘neutral’ it is became clear in the reflective dialogue. This dialogue is planned one week after handing in the assignment. For example, students judge the output as being objective and neutral. But is difficult to state if the training input is not objective and neutral.

Students are not trained in how to ‘prompt’.

Tips / ideas for use in other courses

Tips: the output of ChatGPT isn’t always very specific, or with enough context. When the dilemma is focused, it is, therefore, possible for students to judge whether the output is complete, or has diverse perspectives.

It is an interesting debate on how to support students in the use of GenAI throughout their learning journey at the University. When do we address it and in what way?

Contacts

Lotje de Vries (course coordinator), Ewout Frankema and [email protected]