MathCAMPS Logo

Gemini-1.5 Pro

Performance on individual Common Core standards, grouped by grade level.
IFUP Acc. = Incremental Followup Accuracy, CFUP Acc. = Counterfactual Followup Accuracy, Total FUPs Seen = Number of followup questions a model sees, since a model only sees followups if it answers the main question correctly.

Grade K

Standard Overall Acc. IFUP Acc. CFUP Acc. Total FUPs
K.OA.A.4 0.92 - - -
K.NBT.A.1 0.94 - - -
K.OA.A.5 0.97 0.98 0.99 127
K.CC.C.7 0.99 - 1.00 96

Grade 1

Standard Overall Acc. IFUP Acc. CFUP Acc. Total FUPs
1.OA.A.2 0.98 1.00 0.99 132
1.OA.A.1 0.98 0.97 0.96 138
1.OA.D.8 0.98 - - -

Grade 2

Standard Overall Acc. IFUP Acc. CFUP Acc. Total FUPs
2.NBT.B.7 1.00 0.96 1.00 154
2.MD.B.5 0.95 0.93 0.95 144
2.NBT.B.6 0.94 1.00 0.99 138
2.OA.A.1 0.97 - - -
2.NBT.B.5 0.98 0.98 0.99 135
2.MD.C.8 0.98 1.00 0.99 152

Grade 3

Standard Overall Acc. IFUP Acc. CFUP Acc. Total FUPs
3.OA.D.8 0.97 0.97 0.97 128
3.MD.D.8-quadrilateral 0.98 - - -
3.OA.A.4 1.00 - - -
3.MD.D.8-polygon 0.92 - - -
3.MD.D.8-triangle 1.00 - - -
3.NBT.A.2 1.00 0.96 1.00 155
3.OA.A.3 0.93 0.93 0.77 104
3.OA.C.7 0.97 1.00 0.87 125

Grade 4

Standard Overall Acc. IFUP Acc. CFUP Acc. Total FUPs
4.NBT.B.4 0.99 0.97 0.99 134
4.OA.B.4 0.94 - - -
4.MD.A.2-decimal 0.89 0.96 0.93 122
4.MD.A.3 0.99 - 1.00 85
4.MD.A.2-fraction 0.54 0.81 0.98 63
4.NBT.B.6 0.88 - 0.89 74
4.NF.A.2 1.00 - 0.96 99
4.NBT.B.5 0.99 - 0.96 92
4.OA.A.3 0.78 0.95 0.95 82

Grade 5

Standard Overall Acc. IFUP Acc. CFUP Acc. Total FUPs
5.OA.A.1 0.95 0.96 0.73 101
5.NBT.B.6 1.00 - 0.11 75
5.NF.A.1 0.50 0.59 0.84 88
5.NBT.B.5 0.93 1.00 0.91 82
5.NF.B.4 0.80 0.90 0.94 119
5.NBT.B.7 0.82 0.72 1.00 116
5.NF.A.2 0.82 0.80 0.96 133

Grade 6

Standard Overall Acc. IFUP Acc. CFUP Acc. Total FUPs
6.EE.B.7 0.97 - - -
6.NS.B.2 0.98 - 0.15 47
6.EE.A.1 1.00 - 1.00 97
6.NS.B.3 0.76 0.78 0.98 110

Grade 7

Standard Overall Acc. IFUP Acc. CFUP Acc. Total FUPs
7.NS.A.1-fraction 0.62 0.78 0.89 107
7.NS.A.2 0.89 0.81 0.96 147
7.NS.A.1-decimal 0.98 1.00 0.94 173
7.NS.A.3-fraction 0.52 0.72 0.80 62
7.NS.A.3-decimal 0.90 0.96 0.95 130

Grade 8

Standard Overall Acc. IFUP Acc. CFUP Acc. Total FUPs
8.EE.A.2 0.96 - - -
8.EE.C.8 0.02 - - -
8.EE.C.7 0.63 - - -