Autograder Help

[Gvarner84]

First of all, new here... so hopefully I am posting this in the correct thread.

I am a mathematician, not a programmer. I can usually figure out how to write algorithmically based programs, but am struggling on getting my autograder to work.

I downloaded the example autograder for the calculator example off of gradescope and attemted to edit it.

I have a setup.sh, requirements.txt, run_autograder, run_test.py and a folder with the tests.
The run_autograder code says:

#!/usr/bin/env bash
# Set up autograder files
cp /autograder/submission/problemset1.py /autograder/source/problemset1.py
cd /autograder/source
python3 run_tests.py

run_tests.py says:

import unittest
from gradescope_utils.autograder_utils.json_test_runner import JSONTestRunner

if __name__ == '__main__':
    suite = unittest.defaultTestLoader.discover('tests')
    with open('/autograder/results/results.json', 'w') as f:
        JSONTestRunner(visibility='visible', stream=f).run(suite)

Inside the test folder, I have an init.py file and two test. One is below:

import unittest
from gradescope_utils.autograder_utils.decorators import weight, number
from problemset1 import satisfied_checker


class TestSatisfiability(unittest.TestCase):
    def setUp(self):
        self.calc = satisfied_checker()

    @weight(2)
    @number("2.1")
    def test_eval_add(self):
        #"Evaluate p and (not q) or r"
        val = self.calc.eval("p and (not q) or r")
        self.assertEqual(val, "satisfied")

    @weight(2)
    @number("2.2")
    def test_eval_sub(self):
        #"Evaluate p and (not p) or q or r"
        val = self.calc.eval("p and (not p) or q or r")
        self.assertEqual(val, "not satisfied")

    @weight(2)
    @number("2.3")
    def test_eval_mul(self):
        #"""Evaluate (p and not q) and (q and not r)"""
        val = self.calc.eval("(p and not q) and (q and not r)")
        self.assertEqual(val, "not satisfied")

    @weight(2)
    @number("2.4")
    def test_eval_div(self):
        #"""Evaluate (p or not q) and (q and not r)"""
        val = self.calc.eval("(p or not q) and (q and not r)")
        self.assertEqual(val, "satisfied")

The corresponding program solution is:

def satisfied_checker(statement):
    def make_function(statement):
        def function(p,q,r):
            return eval(statement)
        return function

    A = [True, False]
    satisfied = False
    message = "not satisfied"

    as_function = make_function(statement)

    for p in A:
        for q in A:
            for r in A:
                if satisfied == False:
                    if as_function(p,q,r) == True:
                        satisfied = True
                        #p1 = p
                        #q1 = q
                        #r1 = r
    if satisfied == True:
        message = "satisfied"
    return message

Which is inside a file called problemset1.py.

While the autograder compiles, when it attempts to autograde the assignment it returns messages:
Test Failed: satisfied_checker() missing 1 required positional argument: 'statement'

Any help as to where I went wrong would be greatly appreciated.

Thank you

[mattrussell2]

The error you're seeing is from your testing file. In particular, the line

self.calc = satisfied_checker()

What you're trying to do here is to set self.calc to be the satisfied checker function, but this line of code is actually attempting to call the function. Note that in the example code, the Calculator() call of a similar form is instantiating a Calculator class object [which is effectively calling the constructor method of the Calculator class, which doesn't happen to take any arguments].

In your case, if the satisfied_checker() program is a function [which is fine], then you don't need to call it here. Instead, you can simply do

self.calc = satisfied_checker

Then, future calls to self.calc will call the function student's satisfied_checker function.

However, once that's fixed, you'll still run into problems because of the lines of the form

val = self.calc.eval("p and (not q) or r")

Note the example Calculator class has an eval method inside of it. In that case, the self.calc.eval() function call will call the function eval on the instance of the Calculator object which was instantiated in the setUp method of the testing class. However, in your case, as the satisfied_checker is simply a function, you could rewrite lines of code with self.calc.eval to be self.calc.

If it makes things simpler, I would probably just replace all instances of self.calc with satisfied_checker throughout the code as well, and in that case remove the setUp function entirely. If the program requires setUp to exist, you can simply have it run the command pass to effectively do nothing.

[Gvarner84]

Thank you.
I had actually gotten it figured out and was about to (finally) update this.
That was the problem and it is working just fine now!