Wrong reference answer in MT-Bench

In MT-Bench, the answer to question_id=111 in the first turn is wrong.

question:

{"question_id": 111, "category": "math", "turns": ["The vertices of a triangle are at points (0, 0), (-1, 1), and (3, 3). What is the area of the triangle?", "What's area of the circle circumscribing the triangle?"], "reference": ["Area is 3", "5pi"]}

GPT-4 reference answer:

{"question_id": 111, "answer_id": "NJPTPe2PZ7547UMekB9kBM", "model_id": "gpt-4", "choices": [{"index": 0, "turns": ["To find the area of the triangle, we can use the formula for the area of a triangle with coordinates A(x1, y1), B(x2, y2), and C(x3, y3):\n\nArea = (1/2) * |(x1(y2 - y3) + x2(y3 - y1) + x3(y1 - y2))|\n\nIn this case, the coordinates are A(0, 0), B(-1, 1), and C(3, 3). Plugging these values into the formula, we get:\n\nArea = (1/2) * |(0(1 - 3) + (-1)(3 - 0) + 3(0 - 1))|\n\nArea = (1/2) * |(-0 + 3 - 3)|\n\nArea = (1/2) * |0|\n\nArea = 0\n\nThe area of the triangle is 0. This means that the three points are collinear, and there is no triangle formed by these points.", "Since the three points are collinear and do not form a triangle, there is no circumscribed circle for this set of points. A circumscribed circle can only be formed around a triangle with non-collinear points."]}], "tstamp": 1686287200.2472498}

The vertices are $(0, 0), (-1, 1), (3, 3)$, but GPT-4 said The area of the triangle is 0.
The problem comes from this step: $$Area = (1/2) * |(0(1 - 3) + (-1)(3 - 0) + 3(0 - 1))| = (1/2) * |(-0 + 3 - 3)| = 0$$
A minus sign is missing.
The correct calculation should be: $$Area = (1/2) * |(0(1 - 3) + (-1)(3 - 0) + 3(0 - 1))| = (1/2) * |(-0 + -3 - 3)| = 3$$

Start by comparing question_id 111 in fastchat/llm_judge/data/mt_bench/question.jsonl with its entry in fastchat/llm_judge/data/mt_bench/reference_answer/gpt-4.jsonl. Verify the arithmetic in both turns and correct the reference data so the answer matches the stated triangle coordinates. Done means the question and GPT-4 reference answer agree with the correct calculation.