MathCAMPS Logo

Gemma 7B

Performance on individual Common Core standards, grouped by grade level.
IFUP Acc. = Incremental Followup Accuracy, CFUP Acc. = Counterfactual Followup Accuracy, Total FUPs Seen = Number of followup questions a model sees, since a model only sees followups if it answers the main question correctly.

Grade K

Standard Overall Acc. IFUP Acc. CFUP Acc. Total FUPs
K.OA.A.4 0.86 - - -
K.NBT.A.1 0.95 - - -
K.OA.A.5 0.96 0.46 0.55 126
K.CC.C.7 0.54 - 0.61 49

Grade 1

Standard Overall Acc. IFUP Acc. CFUP Acc. Total FUPs
1.OA.A.2 0.91 0.50 0.49 123
1.OA.A.1 0.97 0.66 0.70 138
1.OA.D.8 0.88 - - -

Grade 2

Standard Overall Acc. IFUP Acc. CFUP Acc. Total FUPs
2.NBT.B.7 0.85 0.56 0.72 133
2.MD.B.5 0.96 0.61 0.58 145
2.NBT.B.6 0.85 0.59 0.52 126
2.OA.A.1 0.86 - - -
2.NBT.B.5 0.94 0.63 0.59 132
2.MD.C.8 0.95 0.50 0.65 148

Grade 3

Standard Overall Acc. IFUP Acc. CFUP Acc. Total FUPs
3.OA.D.8 0.89 0.51 0.77 119
3.MD.D.8-quadrilateral 0.58 - - -
3.OA.A.4 0.88 - - -
3.MD.D.8-polygon 0.63 - - -
3.MD.D.8-triangle 0.78 - - -
3.NBT.A.2 0.96 0.68 0.77 148
3.OA.A.3 0.95 0.68 0.52 104
3.OA.C.7 0.91 0.71 0.64 115

Grade 4

Standard Overall Acc. IFUP Acc. CFUP Acc. Total FUPs
4.NBT.B.4 0.94 0.58 0.65 124
4.OA.B.4 0.24 - - -
4.MD.A.2-decimal 0.66 0.57 0.63 86
4.MD.A.3 0.50 - 0.72 40
4.MD.A.2-fraction 0.27 0.90 0.70 30
4.NBT.B.6 0.40 - 0.25 32
4.NF.A.2 0.27 - 0.37 19
4.NBT.B.5 0.55 - 0.55 51
4.OA.A.3 0.43 0.69 0.61 52

Grade 5

Standard Overall Acc. IFUP Acc. CFUP Acc. Total FUPs
5.OA.A.1 0.54 0.42 0.49 57
5.NBT.B.6 0.60 - 0.06 52
5.NF.A.1 0.00 - - -
5.NBT.B.5 0.32 1.00 0.61 29
5.NF.B.4 0.36 0.19 0.50 43
5.NBT.B.7 0.51 0.26 0.75 75
5.NF.A.2 0.17 - 0.33 10

Grade 6

Standard Overall Acc. IFUP Acc. CFUP Acc. Total FUPs
6.EE.B.7 0.87 - - -
6.NS.B.2 0.33 - 0.12 17
6.EE.A.1 0.97 - 0.89 89
6.NS.B.3 0.43 0.43 0.75 60

Grade 7

Standard Overall Acc. IFUP Acc. CFUP Acc. Total FUPs
7.NS.A.1-fraction 0.05 - - 2
7.NS.A.2 0.08 0.14 - 13
7.NS.A.1-decimal 0.91 0.47 0.51 161
7.NS.A.3-fraction 0.04 - 0.33 3
7.NS.A.3-decimal 0.74 0.66 0.79 111

Grade 8

Standard Overall Acc. IFUP Acc. CFUP Acc. Total FUPs
8.EE.A.2 0.72 - - -
8.EE.C.8 0.00 - - -
8.EE.C.7 0.18 - - -