Chapter 06 — Left, Right, and Why They Disagree

Part III opens here — three chapters on the conventions that decide whether the smartest model flails or the dumbest mock looks real. This one is about coordinate frames: the fact that Unity and the robot disagree about which way is “left,” and what a single wrong sign does. It assumes only that you have seen the wire carry a position or a rotation.

Two worlds that point different ways

An action can say move 2 cm to the left. A proprioception reading can say the end-effector is here, tilted like this. Both are statements about space — directions and rotations — and space is where Unity and the robotics world quietly disagree.

Every 3D system picks three axes and a positive direction for each. The robotics world, following the ROS (Robot Operating System) convention that this project matches, uses:

Unity, built for games, uses a different set:

Two differences hide in there. First, the axes are relabeled: what robotics calls forward (x), Unity calls forward too — but spells z. What robotics calls up (z), Unity calls up — but spells y. Second, and more dangerous, the two systems have opposite handedness.

Handedness — whether a coordinate system is “right-handed” or “left-handed,” which fixes how rotations and cross-products point. Point the fingers of your right hand from the x-axis toward the y-axis; your thumb points along z. If that matches the system, it is right-handed. Robotics (ROS) is right-handed; Unity is left-handed. They are mirror images.

Opposite handedness means you cannot convert between them by just renaming axes. A pure relabel preserves handedness; going from right-handed to left-handed requires flipping a sign somewhere — a mirror reflection. Miss the flip and everything still looks plausible (three numbers, reasonable magnitudes) while being subtly, or grossly, wrong: a robot that reaches left when told right, or turns a screw the wrong way.

The position map

Putting the relabel and the mirror together gives the exact conversion the project uses. Going from a robot-frame position (x, y, z) to a Unity-frame position:

unityPos = ( -y,  z,  x )

Read it component by component. Unity’s x (right) is the robot’s -y (the negative of “left” is “right” — there is the mirror flip). Unity’s y (up) is the robot’s z (up). Unity’s z (forward) is the robot’s x (forward). And the inverse, going back:

robotPos = ( uz, -ux, uy )

A worked example makes it concrete. The robot says “move forward and to the left”: (x, y, z) = (0.10, 0.05, 0). Convert: unityPos = (-0.05, 0, 0.10) — Unity reads that as a little to the left of center, no height change, forward. Left in robot-speak became negative-x in Unity, which is Unity’s left. The directions agree; only the bookkeeping changed.

Insight: the frame conversion is the same one every robotics-in-Unity project needs. This is not a bespoke hack — it is exactly the mapping used by Unity’s official ROS-TCP-Connector, the standard bridge between ROS and Unity. Getting it right once, in one small file, means every position and rotation that crosses the wire is handled consistently. Getting it wrong means chasing a “the robot moves the wrong way” bug through the entire stack. Concentrate the convention in one place and the rest of the code can forget it exists.

Rotations are worse

Positions are three numbers. Rotations are four — a quaternion (qx, qy, qz, qw), the standard compact way to encode an orientation — and the handedness flip tangles them more thoroughly. A robot-frame quaternion becomes a Unity-frame quaternion by this remapping:

unityQuat = ( qy, -qz, -qx, qw )

The components are shuffled and two of them are negated, because a mirror reflection reverses the sense of rotation about the flipped axes. You will not derive this at a glance, and you do not need to — it lives in one function alongside the position map. What matters is the lesson: rotations carry the handedness flip in a less obvious form than positions, so a rotation bug is even easier to introduce and even harder to see. When the model produces a rotation delta — rotate the gripper a little — that delta rides this exact conversion on its way to the joint.

The sign that bites

Here is the detail that proves the whole point, because it is a place where the “obvious” conversion is wrong. Most vectors crossing the frame boundary — positions, gravity directions, linear velocities — use the position map, (uz, -ux, uy). But angular velocity — how fast the body is spinning about each axis — needs a different sign pattern:

positionLikeToRobot(u)  = ( uz, -ux,  uy )
angularVelocityToRobot(u) = ( -uz,  ux, -uy )

Every sign is flipped relative to the position map. The reason is the same handedness reversal: a rotation about an axis reverses sense under a mirror, so angular quantities pick up an extra negation that positional quantities do not. This matters in exactly one place in the project — the reinforcement-learning locomotion track (Chapter 18), where the robot reports how fast its base is spinning — and getting it wrong would feed the balance policy a body that appears to rotate backwards.

Insight: “it’s just three numbers” is how frame bugs survive. A wrong frame conversion never crashes and never looks obviously broken — it produces a valid vector of the right size that means the wrong thing. The robot leans instead of reaching, turns left for right, or a balance policy fights a phantom spin. There is no error message. The only defenses are to keep the conversions in one audited place, and to watch the robot (Chapter 12) rather than trusting that reasonable-looking numbers are correct.

What you now understand

Directions are one convention the messages assume. The next is just as unforgiving and even more mechanical: the order of a robot’s joints, and why the fourteenth number in an action array had better drive the joint both sides agree it drives.

Continue to Chapter 07 — Joints in a Row.

All of this lives in one file, DevVLA/Assets/Scripts/VLA/FrameConversions.cs — the position maps RobotToUnityPos / UnityToRobotPos, the quaternion map RobotRpyToUnityQuat, and the angular-velocity variant used by RlBridge.cs. The convention matches Unity Robotics Hub’s ROS-TCP-Connector exactly, by design.