How to use the world’s compute

I stopped the GPU, not the experiment

I was running an experiment on a RunPod GPU. The code worked. The data was in place. There was a useful checkpoint on the volume, but the run needed more time.

I had to leave it for a few days, so I stopped the Pod. In my head, I had paused the experiment. What I had really done was release one rented GPU while keeping its disk.

When I came back, the volume was intact. The GPU was not available. Capacity had filled while I was away, and the Pod could not get the same worker back. RunPod now calls this out in its docs: a stopped Pod can restart with zero GPUs when capacity changes.

The files survived. The setup did not.

There were other GPUs in the world, just not behind that Start button. To use one, I had to get the project out of its old home, find another provider, boot a new machine, and put the project back together there.

Copying the bytes was only one part. The new machine also needed the right image, CUDA stack, packages, paths, secrets, launch command, and checkpoint. I kept asking small but costly questions: Which files were current? Which package change mattered? Was this the last good checkpoint or only the last one written?

The model was not the hard part that day. Reconstructing the context around the model was. I spent more time making the run recognizable than improving the experiment.

A network volume can reduce this pain inside one provider. RunPod offers them now. But a provider-owned volume still does not give a job a durable identity across providers, and it does not record how to resume the work.

GPU supply is a coordinate, not a shelf

Cloud catalogs make compute look like a shelf of parts. In practice, a GPU is a coordinate: provider, region, model, memory, price, quota, and time. Change one value and the answer can change.

Google lists different GPU models by zone and marks some capacity as limited. It also says quota does not guarantee supply. AWS has an error for the same fact: a valid request can fail because a zone has no instance capacity right now.

This is why ‘supports H100’ and ‘can start my job now’ are different claims. Discovery, permission, live capacity, and a successful launch are four separate checks.

My stopped Pod had exposed the same market from the other side. The hardware had not vanished. Someone else was using it, while usable compute existed under a different set of coordinates.

That changed the system boundary

We used to let the machine define the workspace. The machine had the files, process, shell, and history, so the two felt like one object. The RunPod move showed why that boundary was wrong.

A workspace needs its own stable identity. Call it `workspace_id`. A rented GPU needs a shorter-lived identity, such as `allocation_id`. Stopping or replacing an allocation must not rename the work, fork its history, or erase its next step.

That sounds like a small data-model choice. It changes the whole product. Files have to be durable outside the worker. A checkpoint needs a trusted version. Runtime requirements need a record. Cost and provider events need to attach to the job without becoming the job.

The handoff needs more than a folder

A portable workspace needs a compact description of its current state. The large files can live in object storage. The small record says which large files matter and how a new worker should use them.

For our kind of work, that record needs at least the items below. If one is missing, the resume path depends on memory or luck.

• A manifest of file paths, sizes, versions, and content hashes.
• The last checkpoint that finished and passed its write check.
• The container image or runtime requirements needed to open it.
• The command that was running and the next action to take.
• A record of the old allocation, stop reason, spend, and open failures.

Resume is a protocol, not a button

The word resume hides a sequence. First, stop new writes. Then finish or reject the writes already in flight. Publish one final manifest. Only after that is safe should the old compute be released.

On the other side, acquire a real machine, not just a catalog result. Check the image and driver. Materialize the small files first. Fetch large data as the job reads it when that is cheaper than waiting for a full copy.

Before the UI says Ready, read the state back from the new worker. The checkpoint hash, manifest version, runtime, and workspace path should match the durable record. If they do not, the workspace is still restoring.

This order is less exciting than a clever scheduler. It is also the part that lets a scheduler change providers without corrupting the work.

Then routing becomes useful

Once state can survive the worker, the system can choose compute instead of merely remembering one provider. That choice should use the whole cost of the route: queue time, GPU time, data transfer, storage, and the risk of a weak match.

A cheap GPU can be the expensive option if 200 GB must cross a region first. A faster card can lose if its image is not ready. A listed card is worthless if the launch fails. The best route is the one that gets this workspace to its next verified state.

There are still limits. Data does not teleport. Egress costs money. Some accelerators are scarce everywhere. Sometimes waiting is better than moving. Provider independence should make those facts visible, not pretend they went away.

What I wanted was a Continue button

That old experiment did continue. I found another machine, moved the data, rebuilt the environment, and recovered the run. None of those steps was impossible. They were just work that the system already had enough information to do for me.

What I wanted that day was not a multi-cloud dashboard. I wanted to open the same workspace, see that the old GPU was gone, and press Continue. The system could find a healthy worker, show me the transfer cost, verify the checkpoint, and return me to the run.

That is what using the world’s compute means to us. The machine can change because the work has somewhere else to live.