Local or cloud models — which one wins?

A year of cloud, and the same walls every time

For a year, I had my architectural renders produced by turnkey cloud platforms. With models such as Nano Banana Pro V2, the results became genuinely striking and readily usable — given a few generations and a well-judged composite in Photoshop. But as I folded these platforms into my weekly working frame, four limits took shape, and they are what recently made me reconsider the case for training my own “over-models”.

• Standardised outputs, first: however hard you craft the prompt, the images end up looking alike.
• No real control, next, over the input geometry — and an architect does not sign off a hallucinated building; the drawing is the product.
• The black-box effect: zero visibility on the model’s parameters, and therefore no way to learn from what fails.
• And per-use costs that quietly punish iteration — which is precisely the gesture that makes design.

I had opened ComfyUI for the first time in January 2025, then drifted away from it for lack of time. In July I came back — with one precise hunch in mind: that the new AI models and agents could help me improve the quality of my own models substantially.

Three layers of control, and why they change everything

The first concrete result is a bachelor project rendered four years ago in Twinmotion. At the time the images were decent, but they told nothing in particular. I wanted to take those original images back up in a different style, one I am fond of: Post-Digital Collage. Alongside training my over-models on that aesthetic, I had to build a script for it in ComfyUI.

The script is understood in three parts, and that separation is what makes all the difference with an ordinary prompt.

Geometry stays sacred: the ControlNets read the depth, the edges and the structure of the input image and lock them in, up to a point; the masks decide where the model may reinterpret and where it must obey. The whole difficulty sits there: preserving the drawing while leaving the model enough air to invent its own language.

Style comes from the models: the open-source checkpoint sets the general hand, then my own LoRA injects the visual DNA I trained it on — the collage grain, the texture, the palette. That is what turns “an AI image” into my image.

Precision comes from the parameters: denoise strengths, ControlNet weights, sampler, steps. A handful of dials deciding how far the image travels from the input. Reading them is the whole difference between a gimmick and a tool whose workings you command.

Two models in one month: what the numbers actually say

In July I trained two style LoRAs, both on SDXL base 1.0, with Kohya_ss, on a single 12 GB laptop GPU.

Post-Digital Collage ran through seven versions. The underlying question was dataset size: my intuition said a tight set of 50 well-chosen images would beat a large heterogeneous one. Head-to-head grids at equal parameters said the opposite. v7 — 220 images seen 20 times each — imprints a stronger style than v6 — 50 images seen 90 times — at a near-identical step count, 4,400 against 4,500.

So the real variable was not the step count but the number of views per image. More unique images seen less often generalise better than a few images drilled. Practical corollary: as the dataset grows, scale the repeats down.

Architectural Hand Sketch took the opposite path — and it is the one I learned most from. v1 ran too hot: high capacity, text encoder trained, fifteen epochs on fifty images. Oversaturated. In reaction I pulled three dials down at once on v2 — reduced capacity, noise offset off, fewer epochs — and landed on the symmetrical excess: washed out, near photorealistic, style absent. With no way to tell which of the three was responsible.

v3 recovered the balance by restoring the middle, one dial at a time. It is the most transferable lesson of the whole month: change only one setting at a time. Otherwise you learn nothing.

Define the task you are training your LoRA for

A LoRA is judged in the regime where you use it. In unconstrained txt2img, v7 over-cooked at its nominal weight — scribbled skies, over-outlined trees. In img2img with ControlNet, where I actually use it, the structure holds and the defect disappears. Every new version needs its weight re-tuned: the one that suited the previous version over-drives the next.

The AI that trains the AI

Here is the real novelty of the month, and the reason for this post.

Until now, captioning was by far the most time-consuming task of a training run: describing each image to tell the model what to learn and what to ignore. Done by hand over two hundred images, that is an evening or more given to a repetitive task. An agent does it in minutes, holding to a strict nomenclature, and does it better.

Sourcing followed. Rather than widening a dataset by hand, I had Python scripts written for scraping, embedding and clustering: over a thousand images collected, then reduced by similarity to a coherent core, added to the hand-curated set. The full chain runs in seven stages, from scraping to captioning by way of a visual similarity score.

And above all, the diagnosis. I wanted to see how far a model could read my results — the output grids and the training curves — in order to propose the next round of corrections.

Put end to end — scraping, captioning, diagnosis — these methods gave me results far beyond what I was getting on the same checkpoints two years ago. The hardware has barely changed. The working loop, entirely.

So — local or cloud?

The honest answer: local isn’t universally better. It’s just better at different things.

Local wins on surgical control, and on a style that is literally mine. No SaaS can reproduce it, because it is my model and not a preset. It also wins on privacy: everything runs on a single machine, nothing leaves it, no per-image bill. For competition work, that last point is not a comfort, it is a condition.

Cloud wins on invention. Frontier models populate a scene with elements that are not in my input image: crowds, vegetation, atmosphere, context. Locally, the very ControlNets that protect my geometry forbid exactly that kind of freedom. It also wins on speed and on needing no setup at all — to say nothing of the GPU it spares.

So I stopped picking a side. The skill isn’t local or cloud, it’s orchestration: generate the controlled base locally — my geometry, my trained style — invent the impossible in the cloud, then marry the two in a compositing tool such as Photoshop.

That, to me, is the near future of AI in architecture: not a stack of subscriptions, but architects who conduct these tools instead of obeying them.