Large Language Models and Multimodal Generative AI in Architectural Design: A Review of Capabilities and Evaluation Methods

3.1 Requirements and Early Ideation

The earliest stage covers interpreting briefs, defining programs, and producing conceptual alternatives. One study interviewed ChatGPT as a studio collaborator and had five educators rate the transcript through a Delphi panel: the model listed plausible studio subjects and concepts, but the panel judged its answers generic enough to fit any studio, found factual errors, and noted it could not access map data Caliskan (2023). StakeSight instead simulates stakeholders, using multimodal LLMs with role-conditioned retrieval-augmented generation (RAG) over 3,017 first-person urban narratives; twelve designers rated it strongly useful for early ideation (median 4.5 of 5) while rating role plausibility lowest (median 3.5), and the authors position it as preparation for real engagement, not a replacement Qi et al. (2026). Both produce language about design, not geometry.

Retrieval grounding and parametric scripting push ideation toward deliverables. In adjacent biomimicry work, BIDARA compared four GPT-4 settings on ten design questions scored by two practitioners: feasibility was 3.45 for zero-shot without retrieval and 3.70 for few-shot with retrieval on a 0–5 scale. The comparison changes the prompt regime as well as retrieval, so it cannot isolate the latter’s effect; novelty was at most 2.00 in these settings Toukmaji and Tee (2024). For scripting, a three-agent GPT-4o system splits a prompt into intent inference, Grasshopper component mapping, and JavaScript Object Notation (JSON) emission; three case studies yielded usable scripts of low-to-moderate complexity, a bridge case exceeded the complexity threshold, and the authors report hallucinated components, JSON errors, and no user study Rietschel et al. (2024). Both systems stop exactly where outputs must be geometrically correct.

Practitioner reports describe how interactive parameters structure ideation. Wicaksana and Ardhianto illustrate an architectural Midjourney workflow: semantic prompts and example chaos/stylize settings generate initial alternatives, followed by blending or retexturing and a feedback loop for refinement Wicaksana and Ardhianto (2025). The authors describe faster ideation, but their self-run demonstration supplies neither a manual-workflow baseline nor a measured time saving. Outside architecture, a graphic-design study found requirement ambiguity and shifting requirements make creative practice nonlinear, and linear command-style tools scored lower usability than a framework offering clarification and alternatives (70.21 versus 64.17), with the AI simulated by a wizard Zhou et al. (2024). Ideation methods thus produce program text, concept language, and starting scripts; grounding in sites, dimensions, and regulations remains out of reach, on mostly qualitative evidence.

3.2 Multimodal Concept Generation

The task here is producing design imagery: text-to-image renders, image-to-image restyling, sketch-guided generation, and facade synthesis. Sketch-to-Architecture assembles the fullest pipeline, fine-tuning Stable Diffusion for conceptual floorplans, lifting them to massing models by monocular depth estimation, and rendering with mask-based editing; its authors claim the first systematic such workflow, report no quantitative evaluation, and concede immature floorplans, imbalanced proportions, and inconsistent view geometry Li et al. (2023). For urban renewal, six fine-tuned adapters trained on 50 tagged Guangzhou arcade facades, conditioned by ControlNet line drawings from site photos, produce style-consistent historical facades; assessment is qualitative, with flagged cultural misrepresentation risks Kuang et al. (2023). Both demonstrate visual control, neither spatial or technical correctness.

Sketch input also drives massing and interior generation. Vitruvio turns a single perspective sketch into a 3D-printable watertight mesh, improving reconstruction accuracy 18% and inference time 26% over its baseline, yet its authors state reconstruction metrics do not measure design usefulness, and training used only synthetic sketches Tono et al. (2022). A sketch-to-render workflow claims superiority over Canny edge detection and mobile line segment detection (MLSD) controls from two example sketches, without specifying the proposed recognition model Karadağ (2023). These examples produce plausible imagery and partly structured geometry, not verified drawings.

Evidence comparing platforms comes mostly from expert panels and practitioner reports. A study scoring six text-to-image platforms with a 30-professional panel and five multi-criteria methods found DreamStudio and MidJourney ranked highest (rankings from technique for order preference by similarity to ideal solution, TOPSIS, and analytic hierarchy process, AHP, correlated at 0.997), but only on aesthetic criteria, leaving buildability and compliance untested Hoser et al. (2026). Two reports are anecdotes rather than evidence: a Stable Diffusion walkthrough asserts productivity gains with no evaluation Pichugov and Shabiev (2024), and a ControlNet restyling application reports unquantified feedback while its data statement says no data exists Dhingana et al. (2024). A Midjourney study claims faster iteration and reports aesthetic consistency rising from 78% to 91%, but its reported completion times and percentage reduction disagree, and it has no limitations section, unreported panel details, and timings averaged over ten trials, placing it in the same stratum Aini et al. (2025). Visual quality here is real but weakly measured, and it is not spatial or technical correctness.

3.3 Floor-Plan and Spatial Layout Generation

Several plan-generation studies use quantitative benchmarks, with language, bubble-diagram graphs, and building boundaries as inputs. Tell2Design defines the language task with more than 80,000 instruction-plan pairs; its baseline reaches micro intersection over union (IoU) of 54.34 on the full test set and 55.10 versus 64.67 for humans on a matched 100-case subset. In a separate volunteer check, only 38% of generations met all instruction requirements versus 85% of ground truth, and plain text-to-image generators failed the constraints outright Leng et al. (2023). Architext fine-tunes language models on 250,000 synthetic residential plans, reaching layout validity of at least 99.5% for every model, but correctness falls from 83.73% on room-count prompts to 25.00% on distinct room locations Galanos et al. (2023). ChatHouseDiffusion couples an LLM parser with a diffusion generator; on the Tell2Design protocol it reports Micro-IoU 60.57 versus 54.34, yet the same generator reaches 85.04 with hand-annotated input, and its Llama3 parser found room locations at only 43.62% accuracy: language understanding is the bottleneck Qin et al. (2024). These residential benchmarks show that valid plan outputs can still miss language-specified constraints.

A second lineage conditions generation on graphs and boundaries with generative adversarial networks (GANs): a ten-model Pix2Pix benchmark on the RPLAN dataset scored over six million images with the Fréchet Inception Distance (FID), a distributional similarity score, and even its best models deleted essential rooms at times, deferring architectural criteria to future work Rodrigues et al. (2021). A boundary-plus-graph network won the first Computer Vision for Architecture and Urban Analysis (CVAAD) challenge with 93.9 mean IoU on the Modified Swiss Dwellings dataset, but scores 0.355 without the background class and outputs only rasters Jeon et al. (2023). A graph-transformer GAN reports realism of 0.29 versus HouseGAN’s 0.17, from 22 raters judging 75 samples, and concedes that rasterized FID does not measure wall geometry or constructability Tang et al. (2024). At an adjacent urban scale, a conditional model generates layouts for arbitrary city blocks, cutting block FID from 39.27 to 14.94 against BlockPlanner He and Aliaga (2023). A top-down graph network fills a fixed office volume with multi-story plans, raising voxel-classification accuracy from 22.81% to 74.66% when designers supply a few labeled rooms, and remains office-only Ogura et al. (2025).

The frontier removes rasterization and moves toward structured, BIM-adjacent representations. DiffPlanner casts boundary-constrained generation entirely in vector space with three diffusion models; on boundary-conditioned bubble diagrams it reports FID 1.59 against BubbleFormer’s 14.90, while a separately post-processed, boundary-plus-room-attributes and adjacency-conditioned variant reports floor-plan FID 0.25, yet rooms stay axis-aligned boxes needing post-processing to close gaps, and its perceptual study used three raters with weak agreement Wang and Pajarola (2025). In an adjacent room-interior task, BIM-native tokenization encodes wall-referenced furniture and other room entities; its constrained decoder checks clearance and BIM validity within given residential rooms, not the placement of rooms in a floor plan. It reports semantic-inventory coverage 54.9 versus 40.8 for a bridged ATISS baseline on a controlled same-data benchmark. A separate 50-room deployment-style comparison reports 98.1 for its trained system versus a 68.3 envelope over zero-shot text LLMs Ladron de Guevara et al. (2025). By contrast, prompt-template fine-tuning of Stable Diffusion raises structural similarity (SSIM, a pixel-level structural resemblance measure) of raster plans from 0.3191 to 0.4254, and its authors conclude the output suits early ideation, not technical planning Mostafa et al. (2025).

These scores should not be read as regulatory or architectural compliance, for four reasons. IoU measures overlap with a reference plan, while FID compares distributions in a specified feature space; neither measures conformance to codes; none of these plan-generation benchmarks establishes egress, room minima, or structural compliance; the room-interior BIM decoder of Ladron de Guevara et al. (2025) checks local clearances but does not validate a whole building. Overlap with one ground truth is a weak target in a one-to-many task where even humans reach only about 65 IoU. The corpora are overwhelmingly synthetic, residential, and rectangular, so validity does not transfer to other programs. Raster scores are also insensitive to dimensions and constructability, as the vector and BIM-native authors argue; compliance checking appears only as a separate task, taken up next.

3.4 CAD and BIM Documentation

Documentation tasks divide into querying and authoring BIM models and generating CAD geometry. BIM-GPT answers natural-language questions through a prompt library: intent classification reaches 99.5% with few-shot prompting versus 83.5% zero-shot on the BINLQ dataset, but value recognition stays between 62.0% and 81.7%, geometric properties were excluded, and the authors call it a proof of concept Zheng and Fischer (2023). A generalized framework extends retrieval into authoring with matching and validity-check steps; across 240 generated wall details the check step raised structural-material accuracy from 92.50% to 100.00% and minimum-thickness accuracy from 91.66% to 100.00%, with only two rules in a self-described rudimentary experiment Lee et al. (2024). Domain fine-tuning helps: a 14B model lifts an LLM-judged score from 0.689 to 0.834 overall and from 0.588 to 0.801 on design-review tasks, on injected defects and offline QA only Lin et al. (2026). A BIM-to-finite-element-analysis (FEA) pipeline shows data format dominating outcomes: CSV scored 94% or higher on its 14-question benchmark while JSON and prompt formats fell to 50% or below, on a single steel frame Lee et al. (2026).

Text-to-CAD makes the gap between producing code and producing usable geometry measurable. Prompted general LLMs fail outright: on the Text2CAD test set raw GPT-4o produced 93.00% invalid CadQuery scripts, and under the CADFusion protocol proprietary LLMs showed invalidity between 54.26 and 87.80 Guan et al. (2025); Yan et al. (2026). Fine-tuned generators improve sharply: CAD-Llama raises command accuracy to 84.72 versus 69.91 for Text2CAD on DeepCAD, and CAD-Coder lowers mean Chamfer distance to 6.54 with 1.45% invalidity on the same test set, though cross-paper distances are not comparable Li et al. (2025); Guan et al. (2025). The image-mediated Text2CAD pipeline reports a GPT-4 image rating of 8.375 out of 10 with no baselines, judging pictures rather than engineering validity Yavartanoo et al. (2024). All of these systems target single-part mechanical benchmarks, none architectural documentation.

A second cluster runs the code and reacts to the result: ToolCAD trains tool-calling agents that drive a CAD engine through a primitive library, reaching 63.9% average success versus 62.7% for GPT-4o ReAct prompting, with engine feedback beating vision-language model (VLM) feedback Gong et al. (2026). CADCodeVerify has a VLM pose validation questions about renders; applied to GPT-4 it lifts the compile rate to 96.5%, but the verifier’s own answers are only 64.6% to 68.2% correct, human feedback still wins, and its metrics miss defects like gaps between parts Alrashedy et al. (2025). An agent study across four workflows found visual inspection best (62% success), naive ask-back clarification worst (37%), zero passes on a right-angle geometry, and 17% overall on a topology-optimized bracket Schüpbach et al. (2025). PR-CAD adds multi-turn editing, lifting task success from 31.25% to 81.25% for novices and from 56.25% to 100% for experts in its human study; this required multiple turns, so the shorter mean time per turn does not establish a shorter task An et al. (2026). RA-CAD learns a post-execution feedback policy, cutting invalidity to 1.05 against baselines above 21.98, but its rewards require ground-truth references absent in open-ended design Yan et al. (2026).

The clarification studies differ in conditions rather than establish a direct contradiction. Unconditional ask-back dialogue lowered success on mostly well-defined constructive-solid-geometry prompts compared with visual inspection Schüpbach et al. (2025). On a separate ambiguous-prompt benchmark, ProCAD reports 0.9% invalidity with its own clarifier and coder versus 4.8% with both agents supplied by Claude Sonnet 4.5. With the ProCAD coder held fixed, the Claude clarifier gives 2.3% versus 0.9% for ProCAD’s clarifier; the 14.6% single-model baseline and 12.2% human-evaluation result use separate settings Yuan et al. (2026). Different tasks, representations, models, and measures prevent a head-to-head conclusion; a matched comparison remains needed.

Even the strongest text-to-CAD numbers describe valid geometry, not architectural requirements. A four-level benchmark reports invalidity rising from roughly 15% to 70–90% across its middle tiers and shows execution-only scoring overestimates capability: the lowest-invalidity model on the hardest tier scored near-zero IoU, and only about 10% of its real-world tier is architectural Wang et al. (2026). Adjacent road-engineering experiments in FreeCAD concur: a brief converted directly into a script guarantees nothing about geometric, computational, or regulatory correctness, one refinement cycle cost about 24,000 billed tokens (168,000 for seven), and the proposed validation architecture remains unvalidated Kotov (2026). Automated checking is emerging separately: a GPT-4 vision probe found missing doors and windows in 100 floor plans at F1 0.61 overall but 0.23 for doors Kumar et al. (2024), while a fire-safety checker pairing a regulation knowledge graph with layered LLM reasoning reached F1 0.863 with zero missed violations versus 0.530 for plain RAG, on one dormitory and 122 clauses Zhao et al. (2026). These examples have not tested a complete architectural generation-and-verification loop against the same building requirements.

5.1 Cross-stage synthesis of review findings

For RQ1, the reviewed examples cover requirements and ideation, multimodal concepts, spatial planning, and CAD or BIM-related tasks. These are distinct outputs, not successive steps of a demonstrated professional pipeline. The plan generators tested on reference layouts Qin et al. (2024); Wang and Pajarola (2025), the mechanical-part CAD generator tested for script execution and geometry Guan et al. (2025), and the fire-safety checker tested on one dormitory Zhao et al. (2026) do not jointly establish a route from an architectural brief to verified construction documents. An adjacent road-design study proposes a division of labor among a language model, geometric tools, checks, and engineers, without validating that complete architecture Kotov (2026). No such integrated architectural evaluation was identified in this structured, non-exhaustive review.

For RQ2, the property measured changes with the output. Expert ratings in a student ideation experiment assess originality and repetition under that task’s conditions Ge and Hou (2025); plan overlap measures resemblance to a reference, while a distributional score compares generated and reference sets of bubble diagrams or plans under separate protocols Qin et al. (2024); Wang and Pajarola (2025). Script execution and geometric distance assess a mechanical CAD output against a benchmark Guan et al. (2025). A narrow drawing-flaw detector checks doors and windows, and a fire-safety checker evaluates clauses in a single building Kumar et al. (2024); Zhao et al. (2026). None of these measures alone establishes the unmeasured properties of the same output. In particular, visual or spatial resemblance cannot substitute for a check of the brief, constructability, or applicable rules.

RQ3 concerns this mismatch between demonstrated properties and the requirements of professional architectural work. The reviewed examples provide limited evidence about integrated requirements checks, human revision time, and reliability across tasks and projects. Lack of a reported test does not show that a system would fail it. Figure 2 therefore separates illustrative reported evaluations from a proposed stage-specific verification requirement. Its six categories organize questions for comparison; they are neither a validated scoring scale nor a certification gate. Governance proposals that assign human oversight by task risk offer a related recommendation, not experimental proof of deployment readiness Albano (2026).

5.2 Scope and limits of the evidence

The 185 page-anchored records comprise 107 core and 78 contextual sources. The main text cites a smaller set to explain particular methods and comparisons; it is not a quantitative account of every record. The search was structured but non-exhaustive, and access to full-text PDFs constrained the included evidence. The six review-proposed categories have not been coded into a complete source-by-stage matrix, so this article reports no corpus-wide frequencies for them. A blank in Figure 2 denotes no example presented there, not proof that an evaluation is absent from the literature.

Tasks, datasets, outputs, and baselines also differ among the cited studies. Their scores cannot be pooled into one measure of progress toward readiness. Evidence from graphic design, mechanical CAD, and road engineering helps identify questions for architectural research but does not validate architectural documents. The proposed verification framework identifies properties to test; it has no operational pass threshold and has not been validated against professional decisions.

5.3 Implications for evaluation and future research

A useful next study would test the same generated artifact against its task metric, the input brief, and relevant expert or rule-based checks. Reporting the outputs and failed cases together would show whether a good similarity or execution score coincides with architectural requirements. For plans, adjacency, boundary, access, and applicable code conditions need separate checks; for CAD or BIM documents, the checks must address building-scale geometry, semantics, and revisions, rather than infer these from executable mechanical parts.

Clarification policies need controlled comparison too. An unconditional ask-back strategy under mostly well-defined constructive-solid-geometry prompts performed differently from the ProCAD system on ambiguous synthetic CAD prompts Schüpbach et al. (2025); Yuan et al. (2026). Those studies use different systems, tasks, comparators, and measures, so they do not establish a contradiction or a general advantage. A matched experiment could vary ambiguity while holding the benchmark and baseline fixed. Future evaluations should also record human correction time, error frequency, computational cost, and performance over repeated professional tasks. The road-design report’s token volume is a computational observation, not a measure of architect time Kotov (2026). These are proposed tests of the evidence gap, not results demonstrated by this review.