Silinx · User's guideTry Silinx

Automatic grading

User's guide · Teaching with Silinx · English

Silinx grades digital-systems exercises the way Mooshak grades programming ones. A student submits a circuit, the exercise's tests simulate it, and each test gives a verdict and its points (#3191). There is no judge server: silinx grade does the grading, and adapters fit it into the tools a course already has: GitHub Classroom (classroom.md, #3193).

The exercise

An exercise is a JSON file, <name>.exercise.json. It holds:

A file's text can be written inside the exercise ({ "path", "text" }) or kept in a file beside it ({ "path", "file" }, read relative to the exercise's folder and never outside it).

{
  "silinxExercise": 1,
  "id": "counter4",
  "title": { "en": "4-bit counter", "pt": "Contador de 4 bits" },
  "statement": { "en": "A 4-bit counter with a synchronous reset and an enable.", "pt": "…" },
  "top": "counter4",
  "ports": [
    { "name": "clk", "dir": "in", "width": 1 }, { "name": "rst", "dir": "in", "width": 1 },
    { "name": "en", "dir": "in", "width": 1 }, { "name": "q", "dir": "out", "width": 4 }
  ],
  "allow": { "languages": ["vhdl", "verilog"], "drawings": ["fsm", "asm", "sch", "tt", "pn", "circ"], "primitives": true },
  "limits": { "simNs": 1000000, "steps": 5000000, "seconds": 10, "memoryMb": 256 },
  "reference": { "files": [{ "path": "ref/counter4.vhd", "file": "ref/counter4.vhd" }] },
  "tests": [
    { "id": "count", "visible": true, "points": 30, "kind": "vectors", "clock": "clk", "periodNs": 20,
      "vectors": "rst en | q\n1 0 | 0\n0 1 | 1\n0 1 | 2\n0 0 | 2\n" },
    { "id": "random", "points": 40, "kind": "reference", "clock": "clk", "cycles": 200, "seed": 7,
      "hold": { "rst": ["1", "1"] } },
    { "id": "bench", "points": 20, "kind": "testbench", "top": "tb_counter4", "done": "TB DONE",
      "files": [{ "path": "tb/tb_counter4.v", "file": "tb/tb_counter4.v" }] },
    { "id": "impl", "visible": true, "points": 10, "kind": "synthesis", "latches": false, "luts": 8, "ffs": 4, "slices": 4 }
  ]
}

A test is hidden unless it has "visible": true. Each test is worth points (1 when not given).

Kinds of test

The verdicts

CodeVerdictWhen
ACAcceptedthe test passes
WAWrong Answeran output differs: the test, the cycle, the signal, expected and obtained, and the cycles around it (a visible test's report only); a test bench's assertion; a difference from the reference that the equivalence check finds
CECompile Errorthe submission does not compile, a drawing has errors, a listed file is missing
IEInterface Errorthe top's ports are not the statement's, no module of that name, a language, a drawing or the primitives the exercise does not allow, a module named like the exercise's test bench
NSNot Synthesizablethe open synthesis rejects the design; a warning where none is allowed
LILatch Inferreda latch where none is allowed
TLETime Limit Exceededthe test's time, its simulation steps, the delta cycle limit (a combinational loop), the loop iteration limit
MLEMemory Limit Exceededthe heap of the submission's process
RERuntime Errorthe design's own error stops the simulation ($fatal, an index out of range)
RLResource / Timing Limitmore LUTs, flip-flops or slices than allowed
GEGrading Errorthe grader could not run the test (its setup's fault, not the submission's): the message says why

A Compile Error or an Interface Error applies to every test. The total is the sum of the Accepted tests' points.

Running it

silinx grade counter4.exercise.json submissions/ --csv grades.csv --json grades.json
silinx grade counter4.exercise.json my-project --visible      # a student's run: the visible tests only
silinx grade counter4.exercise.json --student-copy student/counter4.exercise.json

In the interface

Project ▸ Exercise… opens the project's exercise, or makes one with New Exercise: its top's ports, and a first visible test to fill in. A .exercise.json opened from the Files opens the same way. The document shows the statement in the interface's language, the ports, and the tests with their verdicts and points once run.

The MCP server's grade tool does each of these (run, ports, reference, expected, student_copy, template, results, new).

The example

exercises/counter4/ is a 4-bit counter with a synchronous reset and an enable. It has:

Its students' copy is silinx grade exercises/counter4/counter4.exercise.json --student-copy ….

Security

A submission is untrusted.