Back in February I wrote a RISC-V simulator in C. It only does RV32I, the base 32-bit integer instruction set, so there’s no multiply, no floats and no compressed instructions. It loads an ELF file, runs it one instruction at a time, and stops when the program asks to exit or does something illegal.

It ran my demo program and a few trap tests I wrote by hand, so I figured it worked, and then I pretty much ignored it for about seven months.

When I came back to it I started wondering how I actually knew it worked. The demo passing only really shows that the instructions the compiler picked for that demo work with the values it happened to use, which isn’t much. So this post is partly about the fun parts of building one of these and partly about what happened when I ran the official tests on it.

RV32I is tiny

One thing I like about RISC-V is how small the base ISA is. RV32I has 40 instructions, which covers loads, stores, branches, jumps, integer math and logic, plus a few others like ecall and fence. There are 32 registers, and x0 is hardwired to zero, so writing to it does nothing and reading it always gives you 0.

The core of the simulator is the usual fetch and execute loop. It fetches the 32-bit instruction at pc, pulls out the fields, switches on the opcode, does whatever the instruction says, and moves pc. Most of cpu.c is one big switch statement.

For x0, instead of checking whether rd is zero before every register write, the simulator lets the write happen and then sets x0 back to 0 at the end of every instruction.

C
        cpu->regs[0] = 0U;
        if (advance_pc) {
            cpu->pc = next_pc;
        }
        cpu->steps += 1U;

In hardware it’s even simpler since you just don’t build a register there.

The immediates are scrambled on purpose

The part of RISC-V that looked the weirdest to me at first is how immediates are encoded. Every instruction is 32 bits and there are six formats (R, I, S, B, U and J). In the branch format the bits of the immediate are spread around the instruction in what looks like a random order. This is how the simulator puts the branch offset back together.

C
int32_t rv32i_imm_b(uint32_t instruction) {
    uint32_t imm = (((instruction >> 31) & 0x1U) << 12) |
                   (((instruction >> 7) & 0x1U) << 11) |
                   (((instruction >> 25) & 0x3fU) << 5) |
                   (((instruction >> 8) & 0xfU) << 1);
    return sign_extend(imm, 13U);
}

Bit 12 of the offset comes from bit 31, bit 11 comes from bit 7, bits 10 to 5 come from 30 to 25, and bits 4 to 1 come from 11 to 8. There’s no bit 0 because branch targets are always even. It looks like someone was messing with you, but it was done on purpose for the hardware, and I think it’s pretty clever once you see why.

Whenever an instruction has rs1, rs2 or rd, they’re in the same place in every format, so the hardware can start reading registers before it even knows what kind of instruction it has. The sign bit of every immediate is also always bit 31, so sign extension can start right away at the same time as decoding. The rest of the shuffling is about keeping as many immediate bits as possible in the same position across formats, so the hardware needs fewer muxes to pick where each bit comes from. It makes the software a little uglier so the hardware can be simpler, and in C you only have to write the shifts once anyway.

The simulator does sign extension with this trick, which I really like.

C
static int32_t sign_extend(uint32_t value, unsigned bits) {
    uint32_t mask = 1U << (bits - 1U);
    return (int32_t)((value ^ mask) - mask);
}

If the sign bit was 0, the XOR sets it and the subtract takes it back off, so nothing changes. If the sign bit was 1, the XOR clears it and the subtract borrows all the way up through the top of the word, which gives you the negative number without any branches.

Getting programs in and out

The simulator loads regular ELF files, which is what the RISC-V GCC toolchain outputs. It goes through the program headers and copies every loadable segment into 128 KiB of flat simulated RAM. If a segment is bigger in memory than in the file, which is how .bss (all your zero-initialized globals) works, it fills the rest with zeros.

There’s no OS to exit to, so I borrowed the Linux convention. You put 93 in a7 (the exit syscall number on RISC-V Linux), put the exit code in a0, and run ecall, and the simulator stops. If anything else goes wrong, like an illegal instruction, a misaligned access or a read outside of RAM, it halts with a trap and tells you which one it was.

Does it actually work

This is where riscv-tests comes in. It’s the RISC-V project’s own test suite, and rv32ui is the part for RV32I user-level instructions. It’s 42 small test programs, and each one goes after one instruction (or a pair, like loads and stores) with a bunch of edge cases.

The problem is that the tests are written for an environment that assumes a real machine. The standard one sets up machine-mode trap handling with CSRs and reports results by writing to a memory location called tohost that the host watches. My simulator has no CSRs and no trap handlers.

The tests only really depend on a few macros though. A test calls RVTEST_PASS or RVTEST_FAIL at the end and keeps the current test case number in gp. So I wrote my own small version of that header where passing exits with code 0 and failing exits with the number of the test case that broke.

C
#define RVTEST_PASS li a0, 0; li a7, 93; ecall
#define RVTEST_FAIL mv a0, TESTNUM; li a7, 93; ecall

The standard environment actually does almost the same thing. Its pass and fail macros also put 93 in a7 and run ecall, they just go through a trap handler that writes to tohost.

Then I wrote a small script that builds each test with the cross compiler, runs it, and checks the exit code. On the first run 40 out of 42 passed, which surprised me because I was expecting a bunch of sign extension bugs. The two that failed were both pretty interesting though.

fence_i failed with an illegal instruction. That test is about self-modifying code. It writes new instructions into memory, runs fence.i, and then jumps to them. fence.i isn’t part of RV32I, it’s its own small extension called Zifencei, so the simulator was right not to know it. On real hardware fence.i tells the CPU that you just overwrote some code and it should throw out anything stale in its instruction cache. My simulator doesn’t have an instruction cache and reads every instruction fresh from memory, so fence.i has nothing to do, and the fix was to make it a no-op. That test passes now.

ma_data stopped on a misaligned access. It checks that loads and stores work at addresses that aren’t a multiple of their size, like reading a 4-byte word that starts at address 1. The RISC-V spec lets an implementation either handle those or trap on them, and my simulator traps on purpose (it even has its own test for that trap). So instead of changing that I skip ma_data, with a comment explaining why.

So now it’s 41 passed, 0 failed and 1 skipped.

Who the tests are really for

Reading through the test macros showed me who these tests were mainly written for. In the add test, a lot of the cases are “bypass” tests, which compute a value, put zero, one or two nops after it, and then use the value.

The nops matter for a pipelined CPU. In a pipeline an instruction can need the result of the one right before it before that result has been written back to the register file, so the hardware has to forward it from later in the pipeline. Bugs in that forwarding only show up at certain distances between the two instructions, and the nops are there to test those distances. So these tests are mostly aimed at people building real hardware.

My simulator finishes each instruction completely before it starts the next one, so those cases can’t fail for it. I think the fair way to put it is that the tests show every instruction does the right thing, including the edge cases, but a good part of what they check is stuff my design doesn’t have.

That might matter later. If I ever add a pipelined timing model (it’s on the roadmap in the README along with four other things), those bypass tests would actually start testing something.

Some numbers

I also timed it for fun. A sieve that counts the primes below 100,000 runs 1,661,814 instructions and takes 44 to 58 milliseconds on my laptop, including startup. That’s roughly 29 to 38 million instructions a second for a plain switch statement in C with no tricks. It also gets the right answer, 9,592 primes, which comes out as exit code 120 since exit codes are only a byte. (It’s compiled for plain RV32I so there’s no multiply instruction, but a sieve doesn’t need one anyway.)

The rv32ui tests are tiny compared to that. The shortest one, simple, is 4 instructions and the longest, ld_st, is 926.

Anyway

I guess the main thing I learned is that “it runs my demo” didn’t really prove much. The demo used whatever instructions GCC felt like using, and the test suite goes after every instruction on purpose with the edge cases people usually get wrong, like shifting by 31, sign extending a byte you just loaded, or writing to x0. Even then it’s worth reading what the tests check, since some of them are checking things your design might not even have.

The code is on GitHub as RISC-V-SIM, and the riscv-tests runner is in there now as make riscv-tests.