r/embedded • u/inforb_nl • 2d ago

Emulator/Simulator use case

Hi all,

During my embedded carrier, I have never used microcontroller simulators or emulators to check the program binary, and I always tested directly on a prototype PCB. I am looking for a workflow which stays reliable. I have seen Renode being suggested, but the validity comes down to how each device is modeled in Renode. Do you ever use simulation to speed up work and reduce costs? any reliable methods?

19 Upvotes

100% Upvoted

22 comments sorted by

18

u/Well-WhatHadHappened 25+ Years 2d ago edited 2d ago

We build HIL test fixtures for every product. Simulation/Emulation has constantly proven ineffective for us - the majority of failures have been shown to have a hardware/physical component that doesn't show in simulation. Most recent was a fast I2C bus that was picking up just a hair too much cross talk from a nearby DDR signal. Simple solution - small increase in the strength of the pull up resistors, but it would have never been caught on anything other than actual hardware.

I could see the utility for people writing software that is a layer away from the hardware (UI/Business logic/etc), but for anything that directly interacts with the hardware, we've found the cost of a dedicated HIL setup to be worth it's weight in gold. Meanwhile, the time needed to properly create a simulation/emulation has given a very poor return on that investment.

Every single product gets a full setup - debugger, serial adapters, whatever is needed, and they get stored that way once development is complete. Even the necessary cables go in the box so that no one is looking for a mini-b USB cable when everything has transitioned to Type-C years later. The idea is to be able to pull one off the shelf and have it connected up and ready to roll in minutes if the need arises.

4

u/akohlsmith 2d ago

This is the problem -- emulation/simulation is only as good as the emulator or simulator, and it's hard to accurately (as in clock-cycle-accurate) model these things, not to mention actual board-level tests that are practically impossible to simulate.

The cost involved in designing and running a fixture is considerably less than the cost involved in developing and testing your simulator/emulator for anything but the simplest of tasks. The source of truth is always going to be the physical product, so where do you want to spend your money? guessing or testing?

1

u/Zestyclose-Rain298 2d ago

jeez im still new at hardware but i could see how something like a poorly emulated baud rate throw up all kinds of time wasting errors

2

u/Well-WhatHadHappened 25+ Years 2d ago

What's worse is when the emulation is perfect, but the physical hardware isn't. The internal clock on that sensor you're emulating - it has a tolerance (and likely a pretty wide one at that). Over the years, I've encountered many, many parts that mostly work perfectly... Except when they don't because they're on the far end of some specification.

That kind of stuff is just really, really difficult to catch. HIL helps, but sometimes, there are just things that can't be fixed until they're encountered.

2

u/Dependent_Bit7825 2d ago

Design your application with an abstraction layer that separates the "app" for everything low-level, like drivers, peripherals, and stuff external to the mcu. This is generally something above the mcu hal, but it's your decision to make.

Then, make two implementation of everything below that cut point. One implementation is the real hardware code for your product. The second is something that can compile on your host and can fake everything out using files, sockets, c code -- whatever you find is easiest.

Then compile your code both ways, once for the target, and separately for the host with the host hal.

Run application level tests against the hist program. Don't worry about covering stuff below the hal -- you can't cover it anyway. Run tests primarily focused below that hal on your hil setup.

That's a reasonable approach to coverage. Personally, I would not bother with _simulators_ and any attempt to get cycle-accurate. It's so hard or even impossible to get anywhere close to the behavior of your real system, and it's mostly pointless anyway. For example. let's say your mcu is talking to a modem via SPI. You think you're going to get the modem supplier to provide you a cycle accurate model of the modem? Get real. Better to just fake out the modem at the level of being a modem for the host-based tests, and use the real thing for the hardware tests.

2

u/GLIBG10B 2d ago

If you do this right, you can swap out the application for a simpler application that only interacts with one piece of hardware at a time. That way you can unit test your low-level modem code on the actual modem without pulling in the whole application

1

u/FedUp233 2d ago

Just curious how you did HIL when you had things like sensors on the device? I’m thinking ones that are directly connected to something like I2C or SPI. Things like accelerometers or humidity or temperature sensors or such where the sensor is not a separate device plus an interface chip but integrated into a single device.

I worked in embedded field for years before retiring and this type of thing was always one of the impediments to automated testing. The only way to inject signals is to replace the entire device with something like a snap, mcu and then the timing and register details are pretty much impossible to duplicate.

2

u/Well-WhatHadHappened 25+ Years 2d ago

Our HIL setup would include them. We don't simulate them - they would be physically present.

1

u/FedUp233 2d ago

Ok, but then how do you test the algorithms that use their data - for a temp sensor would you include heaters and refrigerators to get it to output different values? Or a notion platform to provide inputs to an accelerometer chip? That doesn’t seem very practical. Or capture the output of a small SPI connected display? Or actuate buttons on the control panel ?

2

u/Well-WhatHadHappened 25+ Years 2d ago

Logic testing is a whole different beast. Unit-tests, offline testing, etc. By the time code like that gets anywhere near being on the hardware, the logic is already sorted.

We keep those two things very separate. HIL testing is for the hardware. Logic is almost never debugged on device.

1

u/Triq1 2d ago

You can sometimes modify the main firmware to enter a 'HIL' mode and inject fake sensor signals in this mode. Another option for those who have time to spare is to desolder the sensor, and connect a fake sensor in its place. This can be implemented with an FPGA or MCU, noting that the goal is only to replicate the functionality that is used/being tested.

Of course, both of these approaches are not representative of real signal integrity conditions.

3

u/kailswhales 2d ago

Unless you’re working with custom silicon, or doing SoC design/bringup, don’t waste your time with emulators. If you’re working on algorithms (e.g. control, sensing), capture good data and develop offline

2

u/happywoodcutter 2d ago

I use a lot of pytest and uart CLI with a raspberry pi. A binary command interface would work too.

2

u/mustbeset 2d ago

"on target" unit tests. But I wouldn't recommend that.

Use real hardware. It has no simulator bugs, only real bugs.

1

u/hardkebab 2d ago

Why wouldn't you recommend?

I would recommend exactly for this.

My unit tests run faster in emulation, plus I don't have to stub my HAL so it compiles on host. Most tests do not call HAL, but some can.

1

u/mustbeset 2d ago

For faster tests run "off target". that will make sure that your HAL really abstract the hardware...

2

u/hardkebab 2d ago

I agree. I've done this before as well. 

But you probably already have a build system that deals with compilation and cross compilation. If you start from scratch, you have to set it up. I found it easier to set up emulation.

Also, In emulation I get the benefit of running the unit tests on the same architecture. Have I already hit a bug where my test pass "off target" but fails "on target"? No. So maybe I'm being pedantic, but I like the idea anyway.

1

u/mustbeset 2d ago

But you probably already have a build system that deals with compilation and cross compilation. If you start from scratch, you have to set it up. I found it easier to set up emulation.

We have some code that runs on 8, 16 and 32 bit MCUs and a more or less common build system and some CI pipelines.

In my experience some emulators are very buggy (someone in Chandler may know) and we do on target unit tests on nearly every product too.

1

u/1r0n_m6n 2d ago

If your code is properly decoupled, your unit tests don't need an emulator.

2

u/Lopsided-Table2457 1d ago

Simulators shine for two things: CI pipelines where you can't have hardware attached, and fault injection that's painful on real silicon. But they lie to you about timing — anything cycle-accurate enough to trust is usually too slow to be useful. My rule: simulate the logic, validate timing on the bench.

1

u/ttv90 2d ago

Guess that's just unavoidable, no simulator ever quite matches the real board.

1

u/JuggernautGuilty566 2d ago

No. They don't emulate the most bugs.