Can My Robot Tell What Ground It Is On? I Measured It, and the Answer Came Eight Seconds Late
A week ago I wrote that my slip detector could not tell a robot stuck with its wheels spinning from a robot driving normally. A reasonable proposal came out of that failure: stop estimating slip continuously and identify the ground instead. Pick a speed to suit the surface you are on, slow on the slippery patch and nominal on grip. Two modes instead of a continuous probability.
This post measures the precondition for that proposal: can the robot work out which surface it is on from the sensors it has? I set up three separate tests and I will walk through all three. The result was not what I expected. The information was in the data. The problem was when it arrived.
First I Had to Fix the Label: Slip Is an Event, Terrain Is a State
My old label was called is_slipping and it answered the question "am I
losing traction right now". That is an event and it lasts a moment. The
question "which surface am I on" is a state and it holds for four
metres. One label cannot serve both questions, and I had been using the event label to
study the state.
So I changed the experiment. Every sample now records the robot's ground-truth position, and the label comes from whether that position falls inside the patch. The script reads the patch boundaries out of the world file rather than me typing them in, so if the world changes the label follows.
While setting that up I noticed something convenient. The worlds were already what I needed: grippy ground, a four-metre low-friction patch between 2.5 and 6.5 metres, then grip again. Every run already crosses both surfaces. I did not need to build a new world, I needed to read the one I had properly.
The Experiment Sheet
| World | Patch friction | Samples | Samples on patch | Fraction labelled slippery |
|---|---|---|---|---|
| slip_mu02 | 0.02 | 867 | 616 | 0.710 |
| slip_mu05 | 0.05 | 867 | 615 | 0.709 |
| slip_mu12 | 0.12 | 867 | 397 | 0.000 |
| slip_mu20 | 0.20 | 867 | 405 | 0.000 |
Slippery here means the robot is on the patch and the patch friction is 0.05 or below. That threshold is not arbitrary: an earlier sweep found 0.05 is where slowing down starts to pay for itself. The worlds at 0.12 and 0.20 contribute only negative examples, that is examples of "this ground is fine".
The sensors available are the wheel encoders and an IMU. Thirteen features are derived from them, most of them built on the residual between wheel acceleration and body acceleration. That detail matters at the end.
Test One: What Does the Model Do on Ground It Has Never Seen?
The method is simple. Leave one terrain out, train on the other three, test on the fourth, so the model never sees the friction value it is asked about. The yardstick is the majority baseline, which is the strategy of giving the same answer every time without looking at a single sensor.
| Held out | Model accuracy | Majority | Gain |
|---|---|---|---|
| 0.02 | 0.448 | 0.710 | −0.263 |
| 0.05 | 0.409 | 0.709 | −0.300 |
| 0.12 | 0.897 | 1.000 | −0.103 |
| 0.20 | 0.844 | 1.000 | −0.156 |
The first two rows are bad. On the slippery worlds the model finished 26 and 30 points below a strategy that answers "slippery" every time without consulting a sensor. In the bottom two rows the majority is already 1.000, so any gain there is structurally at most zero and those rows carry no information about the failure. The result is the first two rows.
Test Two: Is the Problem the Model or the Data?
At this point there are three possible diagnoses and each one costs a different month of work. The model family could be wrong, the training data could be too small, or the information could simply not be in the sensors. There is a cheap test that separates them, and I call it the ceiling test. Train the model on data that includes the test terrain and read the score. That number is the best this feature set could reach if transfer were free.
I ran it at four feature window lengths. The window is how many seconds of history each feature is computed over. The majority baseline in this table is 0.645.
| Feature window | Ceiling accuracy | Ceiling gain | Mean gain on unseen terrain |
|---|---|---|---|
| raw, 0.2 to 1.5 s | 0.727 | +0.082 | −0.205 |
| 2 s | 0.785 | +0.140 | −0.232 |
| 5 s | 0.855 | +0.210 | −0.181 |
| 10 s | 0.867 | +0.222 | −0.212 |
Two things read out of that table at once. The good news is that there is a state signal in the data and it grows as the window lengthens, from 8 points above baseline to 22. The bad news is that it never transfers to a new terrain, since the last column is negative on every row. So the part of the signal that is real is a slow part.
Test Three: When Does the Answer Arrive?
A slow signal is not a problem in an offline classifier. But this model is supposed to drive a policy, and the policy has to decide while the robot is still on the patch. The arithmetic is this: the patch is four metres, the commanded speed is 0.4 m/s, so a clean crossing takes about ten seconds.
The table below is the five-second window model on unseen terrain, after smoothing and hysteresis, which is the form a policy would actually consume.
| World | Patch covered | Delay before the decision | False alarm off patch |
|---|---|---|---|
| 0.02 | 51% | 8.3 s | 0% |
| 0.05 | 22% | 13.2 s | 0% |
| 0.12 | n/a | n/a | 7% |
| 0.20 | n/a | n/a | 28% |
In a ten second crossing the model decides at 8.3 seconds. On the 0.05 world it takes 13.2 seconds, longer than the crossing itself.
The 51 percent coverage is not as good as it looks, for two reasons. It averages two command profiles, 30 percent on the continuous run and 71 percent on the stop-and-go one. And the more annoying reason is that the run gets covered partly because the robot is stuck there, so it spends far longer on the patch than a clean crossing would take. The time the detector gets grows with how badly the robot is already failing, which is the opposite of what a controller wants.
The delay is not a side effect of the smoothing either, because with raw features and no window at all I measured 8.0 seconds. Buying more coverage is not free: on the 0.20 world the same settings fire 28 percent of the time off the patch, so on perfectly grippy ground the robot would crawl for no reason a quarter of the way through.
So Why Is the Fast Part Missing?
The reason sits in the feature definitions. The features are derivative-based, most of them the residual between wheel acceleration and body acceleration. Look at the two states:
| Feature | Stuck robot | Robot driving normally | Cohen d |
|---|---|---|---|
| wheel_speed | 0.4000 | 0.3999 | +0.07 |
| wheel_accel_abs | 0.0000 | 0.0118 | −0.20 |
| resid_signed_long | 0.0334 | −0.0013 | +0.56 |
| gyro_std | 0.0002 | 0.0002 | −0.04 |
A robot stuck at constant wheel speed has zero acceleration everywhere. A robot cruising at constant speed also has zero acceleration everywhere. The best of the thirteen features has an effect size of 0.56, which counts as weak, and separates the two states at 0.552 balanced accuracy. A coin gets 0.500.
This also explains why slip detection appeared to work in the first place. It was catching the transient at the edge of the patch, the moment traction changed. Transitions are observable, states are not, and the policy needs the state in the first second.
A Warning: All of This Was Measured on a Faulty Rig
Two days after this measurement I found that my robot had been driving nose down at 13.3 degrees for the whole study, dragging the front edge of its chassis, and that the dragging was producing the slip I thought I was studying. Everything above was measured on that rig, and the section it lives in is marked invalid in my results file. Rather than quietly reusing the numbers, here is what survives and what does not.
What survives: the structural argument does not depend on the apparatus. Two steady states produce the same derivative. That is arithmetic and levelling the robot does not change it.
What does not survive: the numbers. The 0.552, the 8.3 seconds, the ceiling curve. All measured on a tilted robot, and none of it can be quoted for a level one until it is run again.
One neighbouring result has been re-measured: on the corrected rig, on a two-degree ramp at 0.02 friction, the EKF removes 0.009 metres of 6.852 metres of odometry error, about a tenth of a percent. Before the fix the same comparison was 13 millimetres out of 4.11 metres, three tenths of a percent. Different apparatus, same answer: fusing wheel odometry with an IMU does not repair slip error, because neither channel sees slip.
What is queued: repeating the observability test on the ramp. I expect a different answer there, which is the reason to run it. On a slope, the gravity component in the accelerometer and the wheel-versus-body mismatch move together, and that coupling is exactly what flat ground did not offer.
The Three Tests I Now Run Before Blaming the Model
1. The ceiling test. Let the model see the test terrain and read the score. If the ceiling sits at the majority baseline, the information is not in the sensors. No architecture will put it there and you need to add a sensor.
2. The transfer test. If the ceiling is high but held-out performance is negative, the problem is transfer. Collecting more terrains or looking for better invariances is then worth a month.
3. The decision-time test. This is the one I did not have, and it is the one that ended this phase. Accuracy is computed over a whole run, while a controller consumes the answer at a single instant. The question to ask is when the answer arrives relative to the window in which it is still worth having. A classifier that is right at second nine of a ten second crossing is not slightly worse than one that is right at second one. It is useless in a different way, and more accuracy does not repair it.
Limits
Simulation only, one differential-drive robot, one nominal speed of 0.4 m/s, and friction values chosen to make the effect measurable rather than realistic. Two command profiles per world. The observability claim is about this sensor set, wheel encoders and an IMU, and says nothing about a robot carrying anything else. And the rig warning above applies to every number here.
The code is in a private repository while a provisional patent application and a journal review are open. If you want to read it, write to me and I will give you access.