Microduck StudioIndependent project

Train your first duck on a Windows PC

Pollen's tools run on Linux, so on a Windows PC they run in WSL 2, Windows' own Linux, and train on your NVIDIA graphics card through it. This guide goes from that PC to a walking policy you trained yourself, watched, and entered in Pollen Robotics' Microduck Arena. You do not need a robot, and you do not need to code. On another machine?

The training tools here are Pollen Robotics': their code, their challenges and their Arena, built on mjlab and MuJoCo. This page is an independent owner's route to them on a home PC, through Windows' own Linux, a few free tools and, if you like, Claude, so if a step goes wrong for you, tell us which one.

What you need

You do not need Isaac Sim or Isaac Lab. Pollen trains the Microduck with mjlab, which borrows Isaac Lab's way of describing a task but simulates with MuJoCo Warp, and the workshop installs all of it for you.

Linux inside Windows

Pollen's tools run on Linux, and WSL 2 runs Linux inside Windows, so the PC stays yours for everything else.

  1. Right-click PowerShell, choose Run as administrator, and run: 
    wsl --install
    If it only prints its help, run wsl --install -d Ubuntu instead.
  2. Restart the PC.
  3. Open Ubuntu from the Start menu. The first time, it unpacks itself and asks for a username and a password, which are for Linux only. Nothing appears on screen while you type the password, which is normal. Keep it, because Linux asks for it when you install something.

Commands from here on go into that Ubuntu window, unless a step says PowerShell. Ubuntu opens in your Linux home folder, ~, and your work belongs there: folders under /mnt/c are your Windows drive, which Linux reaches much more slowly.

Let Linux see your graphics card

  1. In Windows, install the newest driver for your card from NVIDIA or the NVIDIA app; that one Windows driver is all WSL needs. Do not install any NVIDIA driver inside Ubuntu: NVIDIA's WSL guide says not to, because it would replace the one Windows provides.
  2. In PowerShell, as administrator, bring WSL up to date, then restart it: 
    wsl --update
wsl --shutdown
  3. Open Ubuntu again and check: 
    nvidia-smi
    A table with your card's name in it means Linux can use the card. If the command is not found, try /usr/lib/wsl/lib/nvidia-smi, and if that prints the table, Linux sees the card. An older card shows up here as well and still cannot train, because training needs Turing or newer.

Install git and uv

git fetches Pollen's code and records each change you train with, and uv installs the right Python and everything training needs.

sudo apt update && sudo apt install -y git
git config --global user.name "Your Name"
git config --global user.email "you@example.com"
curl -LsSf https://astral.sh/uv/install.sh | sh

Put your own name and email in the git config lines, since git puts them on every change it records and the workshop commits each change before it trains. Then close Ubuntu and open it again so that it finds uv, and check that uv --version prints a version number.

Open Pollen's workshop

Pollen's Microduck Arena races policies against a clock, and every challenge on it has a Train my own robot line. Start with the 2 m Sprint, which asks the duck to walk down a lane as fast as it can without falling. It is one of the Arena's events, and Race in Pollen's Arena says what each one tests. In Ubuntu:

cd ~
uv run https://pollen-robotics-microduck-arena.hf.space/workshop.py sprint-2m

The first time, it copies Pollen's challenges into ~/microduck-challenges and installs everything, which is a big download but only once. Then it prints a line that starts Microduck workshop: http://127.0.0.1, and if no browser opens by itself, paste that address into your browser in Windows.

Leave this Ubuntu window open: the workshop runs in it until you press Ctrl-C there, so open a second Ubuntu window for anything else.

The workshop's home has a list, Ready to train?, that ticks what is set up: Hugging Face Jobs billing, your fork of the challenges on GitHub, and an NVIDIA card. Training needs the card or the billing, not both, and the fork and billing lines say how to set them up.

Log in to Hugging Face with the button at the top right. Run the same line on another day and it reopens the same work; if it says its port is busy, the workshop is already open in another window.

The fork line's Push sends your commits to your fork, so a policy you publish links to the code that trained it. Without it, publishing still works and the link is missing.

To push, git has to be signed in to GitHub, and GitHub's own tool does that: install it, then log in, choosing HTTPS, and yes when it asks to authenticate Git with your GitHub credentials.

sudo apt install -y gh
gh auth login

Your first training

  1. On the challenge's page, choose New training, and train on this machine. Leave the knobs (Follow the speed order, Stay upright, Lift your feet) where Pollen set them for your first run.
  2. If you changed a knob, the workshop commits that change first, so every training starts from a commit. Then it trains and gives the training a ticket. The recipe runs 3000 iterations and saves a checkpoint every 250, and how long that takes depends on your card.
  3. Every checkpoint is scored with the Arena's own rules as it arrives, and the training's page replays any of them. Pick 5/5 finishes first, then the lowest time. The best checkpoint is often not the last.
  4. When one is good enough, Publish and compete puts it on your Hugging Face account, in public, and enters it on the Arena, which races it and opens its result. The time on your PC is a guide; the Arena's own run is the one that counts.

The bar to beat. Pollen's reference policy scores 4.886 s on the Arena's sprint board and finishes every run.

No NVIDIA card, or an older one

On the challenge's page, choose Hugging Face Jobs instead of this machine; you make that choice for each training. It trains on Hugging Face's GPUs and bills your Hugging Face account, which needs credit on it.

Pollen's notes give the default GPU, one NVIDIA L4, as $0.80 per hour, and the recipe as about 1 to 2.5 hours on it.

Closing the workshop does not stop a training on Hugging Face. It runs, and bills, until it ends or reaches its 12-hour limit, $9.60 at that price. To end one sooner, open the workshop again and press Stop on its ticket, or cancel it on your Jobs page at Hugging Face, but a stopped job cannot be resumed.

Claude beside you

Claude Code is Claude working in your terminal: it reads the files, runs commands and explains errors, and in its normal mode it asks before it changes a file or runs anything that could. It comes with Claude Pro and higher, and you install it in a second Ubuntu window:

curl -fsSL https://claude.ai/install.sh | bash

Close that window and open another, then start Claude Code in the challenges folder:

cd ~/microduck-challenges
claude

It asks you to log in, and if no browser opens, press c to copy the sign-in address and open it in Windows. If the page then shows a code, paste it back into Ubuntu, with a right-click if pasting does nothing.

Pollen wrote the challenges with this in mind: every challenge folder has an AGENTS.md that tells a coding agent which files to change, how to run and judge a training, and the challenge's rules. The 2 m Sprint's tells it never to start a paid job, publish or enter the Arena without your yes. Things to say:

Read what it proposes before you say yes. You decide the goal, the budget and what gets published, and it does the mechanics: that split is Pollen's rule, in the same file.

When something goes wrong

WSL will not start, or mentions virtualization (error 0x80370102)
Virtualization is switched off in the PC's firmware settings (BIOS or UEFI), so turn it on there; your PC maker's support pages say where.
wsl --install stalls before it downloads anything
Run wsl --install --web-download -d Ubuntu instead.
nvidia-smi is not found, or shows no card
Update the NVIDIA driver in Windows, run wsl --update and wsl --shutdown in PowerShell, and open Ubuntu again.
uv: command not found
The installer adds uv for new windows, so close Ubuntu and open it again.
The first install stops with a timeout
It downloads a lot at once, so give it longer with export UV_HTTP_TIMEOUT=600 and run the line again. Pollen's README suggests the same setting for its own first install.
Git says "Author identity unknown"
Run the git config lines from Install git and uv, with your own name and email.
CUDA out of memory
Train fewer simulated ducks at once: ~/microduck-challenges/src/microduck_challenges/sprint_2m/challenge.toml sets num_envs to 4096 under [recipe], so change it to 2048, or ask Claude Code to.
Everything is slow
Check where you are working: under ~, not under /mnt/c.
A MuJoCo window is slow or garbled
Under WSL, Linux windows can fall back to drawing without the graphics card, a known mjlab issue, so run export GALLIUM_DRIVER=d3d12 before the command that opens the window.

Going further

The challenges train for the Arena's courses. So what about the skills a real Microduck runs, its walk, its stand and its tricks? Those you train in Pollen's microduck_rl, which you fetch into your home folder in a new Ubuntu window:

cd ~
git clone https://github.com/pollen-robotics/microduck_rl
cd microduck_rl

No usable NVIDIA card? The quick test and the real run below both need one, so log in to Hugging Face first, as at the end of this section, then skip the quick test and add --hf-jobs --no-wandb to the end of the real run. It trains on Hugging Face's L4 instead, and the Mac guide shows how to bring its checkpoints back into Ubuntu.

Run a quick test first, which catches most mistakes before the long run and, the first time, also installs this repository's own copy of everything:

uv run train Mjlab-Velocity-Flat-MicroDuck --env.scene.num-envs 64 --agent.max-iterations 5 --agent.logger tensorboard

Then the real run:

uv run train Mjlab-Velocity-Flat-MicroDuck --env.scene.num-envs 4096 --agent.max-iterations 4000 --agent.logger tensorboard

--agent.logger tensorboard keeps the logs on your PC; without it, training uses Weights & Biases, which asks you to log in. Pollen's README puts a usable gait at about one to two hours with 4096 simulated ducks, and its notes say a gait needs 4000 iterations or more.

A checkpoint lands every 250 iterations, in a folder under logs/rsl_rl/velocity/ named for the date and time and ending in _velocity. The quick test made one too, so the later folder is the real run. To see them:

ls logs/rsl_rl/velocity/*/

In the commands below, put that folder's name in place of <run> and a checkpoint's number in place of <N>, without the angle brackets. To watch a checkpoint in your browser, run play, which prints an address to open and runs until you press Ctrl-C:

uv run play Mjlab-Velocity-Flat-MicroDuck --viewer viser --checkpoint-file logs/rsl_rl/velocity/<run>/model_<N>.pt

Then export it. export.py writes output.onnx, the one safe way from a checkpoint to a robot, because it bakes in the normalizer the policy expects:

uv run scripts/export.py Mjlab-Velocity-Flat-MicroDuck --checkpoint-file logs/rsl_rl/velocity/<run>/model_<N>.pt

Next, drive it from the keyboard. This opens a MuJoCo window and takes your keys in the terminal. --new-cmd-obs matters: this policy reads the newer layout of commands, and without the flag the script feeds it the old one and stops.

uv run scripts/infer_policy.py --walking output.onnx --new-cmd-obs

To share a gait, log in to Hugging Face with uv run hf auth login, which says you are logged in already if you used the workshop's button. If not, it asks for an access token: make a Write one at huggingface.co/settings/tokens. Then publish, with your Hugging Face username in place of <you>:

uv run publish --onnx output.onnx --repo <you>/microduck-my-walk --kind perpetual --slot walk --no-private

An owner installs it on their robot with sudo robotctl policy load walk and the repository's name, with the robot on its stand the first time, and sudo robotctl policy reset walk puts Pollen's walk back (Pollen's cheat sheet). A clip of it walking at media/preview.mp4 in the repository is what robotctl policy search links beside it (the policy manifest).

Then

The books explain what is happening while it trains. Have an iPhone, iPad or Mac as well? Microduck Studio opens an .onnx file like the one you export and, with every tool on, shows what is inside it. Stuck on a step, or found a better way? Tell us, and say which step.