Engineering note · Synthetic training data

Synthetic rust masks: GPT Image 2.5 Sunburst vs Nano Banana 2.1

By Experiment: October 13, 20268 prompts · 16 generated samples

I generated synthetic image-and-mask pairs with GPT Image 2.5 Sunburst and Nano Banana 2.1 to prepare training data for an RF-DETR rust segmentation model. The intended model will return rust segmentation masks from inspection images. In this note, I compare the two generators’ strengths and failure modes.

I have not trained RF-DETR on these data yet. I plan to cover training and evaluation in a follow-up engineering note in a few weeks. The masks shown here come from the image generators, not from a trained rust detector.

What did this small comparison show?

Nano Banana 2.1 looked more promising on simple, isolated rust patches. GPT Image produced better-aligned masks in the boat-hull and distant-railing examples. Neither set was reliable enough to accept as training labels without inspection and, in many cases, revision. This is a qualitative pilot, not a general model ranking or an RF-DETR benchmark.

The painted-post example from each generator, with its own rust mask overlaid in magenta
Two outputs from the same scene prompt. Each overlay belongs to its own RGB image; the two scenes are not pixel-matched copies.

The experiment: the same prompts, not the same images

I reused eight complete English prompts for the Nano Banana run, with one new output per prompt. I did not select only the best results after generation. Nano Banana received the scene descriptions, not the earlier GPT images as references. The scenes cover painted posts, access caps, railings, structural beams, a mooring bollard, a boat hull and a deliberately rust-free hard negative.

Each prompt asks for one side-by-side image: an inspection-style RGB photograph on the left, and a spatially registered black-and-white rust mask on the right. White means visibly rusted exposed iron or steel. Black means background and non-rust, including intact paint, shadows, dirt, algae, water, reflections and mere runoff staining on intact coating. The prompt also asks the mask to preserve intact-paint islands as holes.

Prompt coverage
8 prompts, one saved sample per model per prompt; 16 samples in total.
Nano Banana model record
gemini-nano-banana-2.1, generated through Picsart.
Nano Banana settings
21:9, 2K, thinkingLevel: high, seed 20261013. An exact 2:1 option was unavailable.
Nano Banana output dimensions
3168 × 1344 px per full pair; 1584 × 1344 px per half.
Preparation
Midpoint split, threshold 128, 8-connected regions, minimum area 1 pixel. No automatic dilation or region merging.
Evaluation completed
Visual review and mask-format checks. No training, no independent ground truth, no IoU, Dice or mAP scores.

Model provenance: I identify the OpenAI run here as GPT Image 2.5 Sunburst. Its archived image-tool response does not preserve an exact backend model-version ID. The Nano Banana export records the model identifier above. These saved outputs support a sample-level comparison, not a fully reproducible backend-version benchmark.

The same prompt does not produce the same photograph. Resolution, camera geometry, corrosion extent and scene complexity differ between the outputs. A better-looking result can partly reflect an easier generated scene. The seed does not make samples identical across services, and eight examples cannot establish either model’s general reliability.

Results at a glance

These assessments describe the saved examples below. “Looks better” does not mean “approved for training”.

SceneVisual assessmentMain concern
01 · Painted post: isolated patchesNano Banana looks betterSmall boundaries still need review
02 · Access cap: small spots and paint islandsMixed resultVisible corrosion omitted elsewhere in the scene
03 · Close guardrail: rust, paint and a welded jointMixed resultIntact welded or painted surfaces labelled as rust
04 · Understructure: beam edges and occlusionNano Banana edge, with caveatsSmall components and unequal scene difficulty
05 · Mooring bollard: a broad patch with holesNano Banana edge, with caveatsGood main patch, incomplete or excessive boundaries
06 · Boat hull: registration around openingsGPT Image looks betterNano Banana mask shifted onto pale paint
07 · Distant railing: small targets and repeated barsGPT Image looks betterMask on the wrong bar; small-target displacement
08 · Hard negative: rust-like distractorsNo clear differenceEmpty masks still require semantic review

A clean binary mask is not necessarily a correct label

Several Nano Banana mask panels contain few intermediate-grey pixels and threshold cleanly. That is a useful output-format property, but it does not prove that the positive pixels sit on rust. A crisp outline can still be shifted onto intact paint, as the hull and railing examples show. Different output resolutions also make raw grey-pixel fractions unsuitable as a model leaderboard.

The positive-area percentages and connected-region counts above are descriptive mask statistics. A larger percentage is not higher recall, and fewer regions are not higher precision. Without an independently checked reference mask, they cannot tell me how much real corrosion was correctly labelled.

The preparation checks establish that a pair can be split, thresholded and converted consistently. They do not decide whether a brown patch is rust or repair an incorrectly placed region. I would still review the boundaries, exclusions, holes and small components before accepting a sample.

Binary semantic labels versus RF-DETR instances

The generator supplies one foreground class, rust, as a binary semantic mask. RF-DETR’s segmentation workflow uses per-object masks; in my preparation plan, each connected positive region becomes one rust annotation. That is a deliberate annotation convention, not proof that a connected region corresponds to one physically meaningful corrosion defect.

Touching patches remain connected, diagonal contact counts under 8-connectivity, and intact-paint holes must be preserved. The Nano Banana preparation report contains a trial COCO RLE export with 138 connected regions across eight images. That demonstrates export consistency, not 138 verified rust annotations, and it is not evidence that RF-DETR training or its actual data loader has been run.

In the source review, the Nano Banana painted-post and hard-negative samples remained needs_review; the other six Nano Banana examples were marked needs_revision. None was automatically approved for training.

What I would change before scaling up

I would start another controlled trial with simple close-up posts and isolated patches, where Nano Banana looked promising. Complex structures, holes and distant thin bars should be separate difficulty groups. I would not start an unchecked large generation batch with either model on the evidence of these eight prompts.

  1. Control the scene. Use fewer objects and explicitly rust-free backgrounds, so the generator has fewer opportunities to draw unlabelled corrosion.
  2. Specify the hard negatives. Restate that clean welds, end caps, intact paint, dark openings and runoff staining are negative. These are proposed prompt changes, not fixes already validated by this experiment.
  3. Review both geometry and meaning. Correct or reject shifted regions, missing corrosion and paint false positives. Inspect very small components at the intended training resolution.
  4. Repeat prompts. Multiple outputs per scene are needed to investigate consistency. One favourable sample is not a reliability estimate.
  5. Evaluate on independent real images. Keep related structures and capture sequences in the same split, and reserve a separately annotated real-image test set before judging deployment quality.

Questions answered

Which image generator was better for rust masks?

There was no universal winner in this pilot. Nano Banana looked stronger for several simpler patches; GPT Image was better aligned in the hull and distant-railing examples. Scene differences and the small sample prevent a general accuracy claim.

Can these generated masks be used as training ground truth immediately?

I would not accept them unchecked. The examples include displaced boundaries, missing rust and intact surfaces labelled positive. Format validation does not replace semantic and spatial review.

Are these RF-DETR predictions or training results?

No. Both RGB images and the initial masks were produced by image generators. RF-DETR training has not been performed on these data; it is the planned next stage.

Why are there no IoU, Dice or mAP scores?

There is no independently annotated reference set for this comparison. Scoring the two generated masks against each other would not be meaningful because they belong to different generated images.

References and related work

The observations in this note are my review of the sixteen saved samples. The following official documentation describes the models and the intended training workflow; it does not independently validate my generated labels.

Notes from the workbench

Keep an eye on what’s next.

New engines, live demos, and practical engineering notes from me, Antal. Occasional emails when there’s something worth sharing.