• between - filter two lists, intersection
  • functions - between/common in functions (24 lines)
  • inline - no functions (19 lines)
• counting - simple for loop with bug
  • nospace - no blank lines in loop body (3 lines)
  • twospaces - 2 blank lines in loop body (5 lines)
• funcall - simple function call with different values
  • nospace - calls on 1 line, no spaces (4 lines)
  • space - calls on 1 line, spaced out (4 lines)
  • vars - calls on 3 lines, different vars (7 lines)

• overload - overloaded + operator (number strings)
  • multmixed - numeric *, string + (11 lines)
  • plusmixed - numeric +, string + (11 lines)
  • strings - string + (11 lines)
• partition - partition list of numbers
  • balanced - odd number of items (5 lines)
  • unbalanced - even number of items (5 lines)
  • unbalanced_pivot - even number of items, pivot var (6 lines)

• initvar - summation and factorial
  • bothbad - bug in both (9 lines)
  • good - no bugs (9 lines)
  • onebad - bug in summation (9 lines)
• order - 3 simple functions called
  • inorder - call order = definition order (14 lines)
  • shuffled - call order \(\ne\) definition order (14 lines)
• rectangle - compute area of 2 rectangles
  • basic - x,y,w,h in separate vars, area() in function (18 lines)
  • class - x,y,w,h,area() in class (21 lines)
  • tuples - x,y,w,h in tuples, area() in function (14 lines)

• scope - function calls with no effect
  • diffname - local/global var have same name (12 lines)
  • samename - local/global var have different name (12 lines)
• whitespace - simple linear equations
  • linedup - code is aligned on operators (14 lines)
  • zigzag - code is not aligned (14 lines)

• 162 total participants
  • 29 Bloomington ($10)
  • 130 Mechanical Turk ($0.75)
  • 3 E-mail
• 1602 trials
  • 18 trials discarded

True Output

12
12

Correct (7)

"12",12

Common Error (4)

3
12

Incorrect (0)

barney

scope - samename

def add_1(added):
    added = added + 1

def twice(added):
    added = added * 2

added = 4
add_1(added)
twice(added)
add_1(added)
twice(added)
print added

scope - diffname

def add_1(num):
    num = num + 1

def twice(num):
    num = num * 2

added = 4
add_1(added)
twice(added)
add_1(added)
twice(added)
print added

between - functions

def between(numbers, low, high):
    winners = []
    for num in numbers:
        if (low < num) and (num < high):
            winners.append(num)
    return winners

def common(list1, list2):
    winners = []
    for item1 in list1:
        if item1 in list2:
            winners.append(item1)
    return winners

x = [2, 8, 7, 9, -5, 0, 2]
x_btwn = between(x, 2, 10)
print x_btwn 

y = [1, -3, 10, 0, 8, 9, 1]
y_btwn = between(y, -2, 9)
print y_btwn 

xy_common = common(x, y)
print xy_common 

between - inline

x = [2, 8, 7, 9, -5, 0, 2]
x_between = []
for x_i in x:
    if (2 < x_i) and (x_i < 10):
        x_between.append(x_i)
print x_between

y = [1, -3, 10, 0, 8, 9, 1]
y_between = []
for y_i in y:
    if (-2 < y_i) and (y_i < 9):
        y_between.append(y_i)
print y_between

xy_common = []
for x_i in x:
    if x_i in y:
        xy_common.append(x_i)
print xy_common

counting - nospace

for i in [1, 2, 3, 4]:
    print "The count is", i
    print "Done counting"

counting - twospaces

for i in [1, 2, 3, 4]:
    print "The count is", i 


    print "Done counting" 

Uncorrected

Corrected

1
2
3
4
5

for i in [1, 2, 3, 4]:
    print "The count is", i 


    print "Done counting" 

Correct Trials

Incorrect Trials