Making gear shifts boring

One of the jobs I’ve been working on for Manchester Stinger Motorsports this year is an electronic gear shifter. The basic premise is simple: take driver inputs in the form of paddles or buttons, check that their input makes sense and then perform the actual shifting action.

That sounds fairly simple, which is usually a warning sign.

If the actuator fires when it should not, it could request the wrong gear, initiate a shift at the wrong time, mechanically fight the gearbox, or leave the driver with a mismatch between commanded and actual gear. None of these are minor software faults; they can damage hardware, force a retirement, or, in the worst case, unsettle the car mid-corner and cause a loss of control, potentially resulting in a crash.

That is what makes the project slightly scary. The firmware is not just blinking an LED or logging a sensor value. It is commanding a motor attached to a gearbox attached to a moving race car. There are several layers of consequence between a single line of malformed C++ and something expensive, or dangerous, happening in real life.

So the important part of the project has not really been making the motor move. Making motors move is easy. Making sure the motor only moves when the shift request is valid, the state is fresh, the encoder is alive, and the controller knows how to stop, is the actual problem.

The controller is built around an ESP32-S3 running ESP-IDF. It talks to the car over CAN bus, drives the shift actuator through a Castle Creations Copperhead 10 ESC and motor, and reads the actuator position using a Samesky AMT20 absolute encoder over SPI. It uses the ESP32’s internal CAN controller (TWAI) in combination with my custom CAN library to read CAN packets from the ECU.

This allows the controller to keep an internal gear count without blindly trusting itself. Before sending a torque command to the ESC, it reconciles that count with a fresh ECU gear packet from CAN, calculated from the ratio between engine speed and transmission output speed.

The shifter’s firmware starts two application tasks from app_main:

xTaskCreatePinnedToCore(shifter_task,
                        "shifter_task",
                        config::SHIFTER_TASK_STACK_WORDS,
                        nullptr,
                        config::SHIFTER_TASK_PRIORITY,
                        nullptr,
                        config::CONTROL_CORE);

xTaskCreatePinnedToCore(can_task,
                        "can_task",
                        config::CAN_TASK_STACK_WORDS,
                        nullptr,
                        config::CAN_TASK_PRIORITY,
                        nullptr,
                        config::CONTROL_CORE);

The can_task keeps the latest ECU gear snapshot fresh. The shifter_task owns the actual shift process.

Both tasks are pinned to the same ESP32-S3 core. The CAN-facing worker tasks are also pinned to that control core. The point is that, during the active shift window, priority decides what runs. The shifter task has the highest application priority, so lower-priority CAN processing should never interrupt the core actuator loop.

This does not mean CAN hardware interrupts stop existing entirely. The TWAI peripheral can still receive and queue frames. The important thing is that task-level CAN processing is not allowed to wander into the middle of the shift loop.

This implementation differs from my original plan. My original plan was to make the shift interrupt do everything. The driver presses the button, the ISR runs, and the shift happens, and nothing else can interfere.

That has a certain blunt appeal.

It is also not a great idea. A gear shift is a relatively long mechanical event, and interrupt context is not the place to sit around waiting for mechanical things to happen. Long ISRs can interfere with software timers, watchdog servicing, and ESP-IDF or FreeRTOS driver code that expects the system to still be alive.

We therefore use a more traditional model of programming where the GPIO interrupt does the smallest possible useful thing:

static void IRAM_ATTR shift_input_isr(ShiftRequest request)
{
    BaseType_t higher_priority_task_woken = pdFALSE;
    bool accepted = false;

    portENTER_CRITICAL_ISR(&shift_input_mux);
    if (pending_shift_request == SHIFT_REQUEST_NONE &&
        shift_task_handle != nullptr)
    {
        pending_shift_request = request;
        accepted = true;
    }
    portEXIT_CRITICAL_ISR(&shift_input_mux);

    if (!accepted) return;

    disable_shift_input_interrupts();
    vTaskNotifyGiveFromISR(shift_task_handle,
                           &higher_priority_task_woken);
}

The ISR latches the first request, disables further shift input interrupts and then wakes the shifter task.

It does not drive the motor or wait for the encoder and it does not yet make any decisions about gear validity.

This is a good model because it gives the firmware one owner for the actuator. During a shift, the shifter task is in charge. There is no second shift request arriving with conflicting opinions.

Before the ESC is commanded, the firmware must check that the request is allowed and that the state it is using is recent enough to trust.

The shift range check is deliberately boring:

static ShiftSafetyStatus shift_request_safety_status(
    ShiftRequest request, gear_t gear_count)
{
    switch (request)
    {
    case SHIFT_REQUEST_UP:
        return gear_count == GEAR_5
            ? ShiftSafetyStatus::OutOfRange
            : ShiftSafetyStatus::Allowed;
    case SHIFT_REQUEST_DOWN:
        return gear_count == GEAR_1
            ? ShiftSafetyStatus::OutOfRange
            : ShiftSafetyStatus::Allowed;
    case SHIFT_REQUEST_NEUTRAL:
        return (gear_count == GEAR_1 || gear_count == GEAR_2)
            ? ShiftSafetyStatus::Allowed
            : ShiftSafetyStatus::NeutralUnavailable;
    default:
        return ShiftSafetyStatus::InvalidRequest;
    }
}

This prevents things like requesting sixth gear on our five-speed gearbox, or trying to shift down below first. The neutral request is only valid from first or second because neutral sits between them.

The CAN task also reads the latest ECU gear snapshot and checks whether that data is sufficiently fresh:

static bool frame_is_fresh(const MSM_CAN::RxFrame &frame,
                           uint32_t max_age_ms)
{
    return static_cast<int32_t>(now_ms() - frame.timestamp_ms)
        <= static_cast<int32_t>(max_age_ms);
}

MSM_CAN::RxFrame speed_gear_frame{};
if (MSM_CAN::get(HALTECH_ID_SPEED_GEAR, speed_gear_frame) == ESP_OK &&
    frame_is_fresh(speed_gear_frame, SPEED_GEAR_TIMEOUT_MS))
{
    output.speed_gear_valid = true;
    const uint16_t combined_gear =
        MSM_CAN::unpack_u16(speed_gear_frame.data, 2);
    output.ecu_gear = static_cast<int8_t>(combined_gear & 0xFF);
}

The important part here is that the ESC is only commanded if the following are true:

1. The requested shift is valid from the current internal gear.
2. The ECU gear packet is fresh.
3. The ECU gear has been reconciled with the internal gear count.
4. The encoder can be read successfully.

If any of those checks fail, the ESC command is never sent; the shift is rejected.

If the ECU gear is fresh but disagrees with the internal gear count, the controller resynchronises the count from the ECU before evaluating the request:

Gear ecu_gear = gear_count;
if (ecu_gear_to_gear(can_output.ecu_gear, ecu_gear))
{
    if (gear_count != ecu_gear)
    {
        ESP_LOGI(TAG_SHIFTER,
                 "Synced gear from ECU: previous=%s ecu_gear=%s",
                 gear_to_string(gear_count),
                 gear_to_string(ecu_gear));
    }
    gear_count = ecu_gear;
}

Once all the relevant prechecks have passed, the task selects a torque command based on the requested shift:

static float torque_for_shift_request(
    ShiftRequest request, uint16_t current_position)
{
    switch (request)
    {
    case SHIFT_REQUEST_UP:
        return SHIFT_UP_TORQUE;
    case SHIFT_REQUEST_DOWN:
        return SHIFT_DOWN_TORQUE;
    case SHIFT_REQUEST_NEUTRAL:
        return torque_for_neutral_from_position(current_position);
    default:
        return 0.0f;
    }
}

The torque values are currently set to calibration values:

static const float SHIFT_UP_TORQUE = 0.4f;
static const float SHIFT_DOWN_TORQUE = -0.4f;
static const float SHIFT_NEUTRAL_FROM_1_TORQUE = 0.4f;
static const float SHIFT_NEUTRAL_FROM_2_TORQUE = -0.4f;

The neutral shift has separate torque values depending on whether it is being requested from first or second. That matters because neutral is a half-shift, and the mechanism approaches it from different directions.

This is also where I should complain about the ESC briefly. This is not a slight on Castle Creations. The Copperhead 10 is doing the job it was built to do, and there is nothing wrong with using a robust RC ESC if the rest of the system is designed around it properly.

That said, integrating an RC ESC into a safety-conscious embedded control system is more than just mildly annoying.

In an ideal world, the actuator interface would look more like this:

motor.set_torque_nm(requested_torque_nm);

In our case, the interface is closer to:

void ESC::set_torque(float torque)
{
    torque = std::clamp(torque, -1.0f, 1.0f);

    float pulse_us = NEUTRAL_US;
    if (torque > 0.0f)
        pulse_us += (FULL_FORWARD_US - NEUTRAL_US) * torque;
    else if (torque < 0.0f)
        pulse_us += (NEUTRAL_US - FULL_REVERSE_US) * torque;

    ledc_set_duty(LEDC_LOW_SPEED_MODE,
                  PWM_CHANNEL,
                  pulse_us_to_duty(pulse_us));
}

Our firmware can still make this work, but it means the ESC has to be treated as something with state and configuration of its own, not just as a “dumb” output stage. The calibration values in the firmware are only one half of the story. The ESC settings matter as well, and the controller needs to be tested with the exact ESC configuration that will be used on the car.

This is also why I am quite conservative about clearing the ESC command. If a shift fails a precheck, times out, or loses encoder feedback, the firmware sends zero torque.

The active shift window is intentionally simple:

esc.set_torque(shift_torque);
const int64_t shift_start_us = esp_timer_get_time();

while ((esp_timer_get_time() - shift_start_us) < SHIFT_TIMEOUT_US)
{
    const ShiftPositionStatus status =
        shifted_position_status(request, gear_count);

    if (status == ShiftPositionStatus::Reached)
        break;

    if (status == ShiftPositionStatus::ReadFault)
        break;
}

esc.set_torque(0.0f);

The task commands the ESC, polls the encoder, and stops as soon as the target is reached, the encoder fails, or the timeout expires.

The other half of the shift is feedback. The actuator position is measured using an AMT20 absolute encoder, and the driver follows the datasheet read-position sequence:

uint8_t rx = 0;
if (!transfer_byte(READ_POSITION, rx))
    return false;

bool acknowledged = false;
for (int i = 0; i < MAX_WAIT_BYTES; ++i)
{
    if (!transfer_byte(NOP, rx)) return false;
    if (rx == READ_POSITION)
    {
        acknowledged = true;
        break;
    }
    if (rx != WAIT) return false;
}

uint8_t msb = 0;
uint8_t lsb = 0;
if (!transfer_byte(NOP, msb) || !transfer_byte(NOP, lsb))
    return false;

position = static_cast<uint16_t>(((msb & 0x0F) << 8) | lsb);

The encoder returns a 12-bit position, so the value wraps from 4095 back to 0. The position comparison therefore needs to handle wraparound properly:

bool AMT20::position_is_near(uint16_t position,
                             uint16_t target,
                             uint16_t tolerance)
{
    position &= 0x0FFF;
    target &= 0x0FFF;

    const uint16_t forward = (position - target) & 0x0FFF;
    const uint16_t reverse = (target - position) & 0x0FFF;
    const uint16_t shortest = forward < reverse ? forward : reverse;

    return shortest <= tolerance;
}

That lets the firmware check whether the mechanism is close enough to the calibrated target position without getting confused near the encoder wrap point.

The target positions themselves still need to be measured on the assembled mechanism:

static const uint16_t BASE_POSITION = 2502;
static const uint16_t SHIFT_UP_STOP_POSITION = 2290;
static const uint16_t SHIFT_DOWN_STOP_POSITION = 2670;
static const uint16_t SHIFT_NEUTRAL_FROM_1_STOP_POSITION = BASE_POSITION;
static const uint16_t SHIFT_NEUTRAL_FROM_2_STOP_POSITION = BASE_POSITION;
static const uint16_t SHIFT_POSITION_TOLERANCE = 20;

At the moment, these are placeholders. The firmware structure is there, but the mechanical calibration still needs to happen before the controller can be used properly on the car.

With all that out of the way, there are still a few jobs that need doing before the shifter can be considered fully operational:

• Measure the final encoder positions for upshift, downshift, and both neutral approaches.
• Tune the shift torque values on the assembled mechanism.
• Tune the shift timeout.
• Confirm the ECU gear decoding against live CAN data.
• Test neutral-from-first and neutral-from-second separately.
• Test encoder read failures with the actuator safely unloaded.
• Test timeout behaviour.

The shift timeout is currently:

static const int64_t SHIFT_TIMEOUT_US = 200000;

That gives a 200 ms actuation window. Whether that is correct depends on the final mechanism, the ESC configuration, and how aggressive the shift can be without abusing the gearbox.

That is basically where the project is now. The firmware still needs calibration and testing against the real mechanism, but the important shape is there: the interrupt captures the request, the safety task owns the actuator, CAN provides the external gear state, and the encoder decides whether the shift actually happened.

That ties back to the original aim. The controller should not make gear shifts exciting. It should make them boring. A request comes in, the firmware checks it, the actuator moves only if it is allowed to, and the motor is stopped as soon as the shift is confirmed or something looks wrong.

As with my other articles, all the code and schematics can be found on GitHub.