On Thursday, researchers at Carnegie Mellon University unveiled LegoGPT, an AI model that creates physically stable Lego structures from text prompts. The new system not only designs Lego models that match text descriptions (prompts) but also ensures they can be built brick by brick in the real world, either by hand or with robotic assistance.
"To achieve this, we construct a large-scale, physically stable dataset of LEGO designs, along with their associated captions," the researchers wrote in their paper, which was posted on arXiv, "and train an autoregressive large language model to predict the next brick to add via next-token prediction."
This trained model generates Lego designs that match text prompts like "a streamlined, elongated vessel" or "a classic-style car with a prominent front grille." The resulting designs are simple, using just a few brick types to create primitive shapes—but they stand up. As one Ars Technica staffer joked this morning upon seeing the research, "It builds Lego like it's 1974."
