An autonomous intelligent agent is a program that operates in a particular environment, perceives aspects of that environment, and then carries out actions that change that environment to carry out some task. Typically, these actions are a mixture of responses to its perception and proactive actions such as exploration.

Your task for this coursework is to design an agent-based system containing the following four aspects:

An Environment. This is the (virtual) place where the agents will operate. It could be one of:

• A simulation of a physical environment in which mobile robotic agents move. This could be the simulation used in the classes earlier in the semester (perhaps extended), a robot environment such as The Player Project (http://playerstage.sourceforge.net), or a project in Unity or a similar game environment if you are familiar with one from elsewhere.
• A chatbot environment such as the ones used in the classes.
• The blackboard system used in the class where we discussed language agents writing poetry
• The Bristol Stock Exchange system introduced in the classes later in the semester (https://github.com/davecliff/BristolStockExchange) or a similar simulation of some aspect of the economy or society
• A game environment such as PacMan (https://gym.openai.com/envs/MsPacman-v0/), the Open Racing Car Simulator (http://torcs.sourceforge.net), RoboCup (https://www.robocup.org/leagues/23) or similar (see e.g. http://www.gvgai.net)
• One of the more complex task environments from the OpenAI Gym (https://gym.openai.com)

There is no need to develop the environment yourself—the focus of the project will be on the agents in the environment (chatbots, robots, trading agents, game-playing agents, autonomous drivers, etc.) – but it is likely that you will set up the details of the environment to address your specific question. You are allowed to use the code from the classes, but please try to make it clear broadly which parts of the code are taken from the class examples, and which is your own work (we appreciate that this is sometimes complicated to do at a line-by-line level, but you should indicate this in broad terms).

Autonomous Agents. You should introduce one or more autonomous agents into the environment, which use some kind of AI to solve a task.

• Examples of AI could be an AI planning system such as Goal Oriented Action Planning (http://alumni.media.mit.edu/~jorkin/goap.html), a search algorithm such as A* search, a genetic or swarm search, a reinforcement learning algorithm, fuzzy logic, or a hard-coded reactive or state-machine AI.
• The task will be one relevant to the environment: e.g. a robot vacuum cleaner clearing up dirt, a chatbot taking an order from a customer, a trader trying to optimise its returns, a game player trying to get a high score in a game, etc.

Within reason, you can use any language to do this. If you are planning to use anything other than Python, Java, Matlab/Octave, JavaScript, and standard web technologies such as HTML/CSS, then please mention this in your topic approval.

A Question. You should be asking a specific question (or a set of related questions) about your system. For example:

• How do different approaches (a genetic algorithm, an A* search algorithm, a hard-coded heuristic) compare in terms of task performance?
• How does the performance of the system change as we vary the number of agents in it?
• If the system is trained on one version of the environment, does that learning transfer over to a new version of the environment
• How do different kinds of communication/coordination between agents effect the efficiency of those agents on the task
• How much improvement does storing some information (e.g. a map of the environment) make compared to carrying out the task in a purely reactive way?
• How do different kinds of sensing/perception systems affect the capacity of the agent to carry out its task?
• How sensitive is the agent to error/noise?

A Set of Experiments. You should answer your question by carrying out a set of experiments. Remember the structure that we talked about in one of the lectures:

• implement code that carries out a run of the agent’s behaviour and measures performance
• then, run that code multiple times to get a measure of average performance
• then, repeat that process for the different conditions in your question, and use descriptive statistics, charts/visualisation, and/or inferential statistics (e.g. significance tests) to test your question

Then, you are in a position to discuss the question using these experimental results as your evidence.

Examples

Here are a few examples of things that you could do. You don’t have to do one of these—indeed, we would prefer you to come up with your own idea—but, these would all be acceptable project ideas if you want to do them:

• To take the “robot vacuum cleaner” from the early classes, and experiment with different numbers of robots, and different coordination strategies (e.g. robots try to stay a fixed distance from each other, compared to sharing a map that they build up)
• Contrast random, fixed and planned orders of asking questions in a chatbot, and see (perhaps by doing a brief user test) which one is better.

Take a number of different trading strategies and r