MinecraftLM

Build 3D Minecraft Worlds from Text

Project Team: Matt Zhou, Johnathan Chiu, Preston Bourne, Avinash Jain

MinecraftLM.mp4

Table of Contents

• What is MinecraftLM?
• What You Can Build
• Quick Start
• Configuration

What is MinecraftLM?

MinecraftLM lets you create 3D structures in Minecraft just by describing them. Type "build me a medieval castle with towers" and watch as an AI agent writes code, validates it, and generates your structure in real-time. Keep the conversation going to add gardens, modify interiors, or place your castle on terrain—the agent maintains context across turns and ensures everything connects spatially.

No templates, no limits—the AI writes code to understand spatial relationships, materials, and architectural concepts, giving you the freedom to build anything you can imagine.

What You Can Build

MinecraftLM can create a wide range of structures—from realistic landmarks to fantasy worlds to functional objects.

Buildings & Architecture

Build anything from a cozy cottage to a Gothic cathedral. The agent understands architectural concepts like walls, roofs, windows, doors, and can create multi-story structures with furnished interiors.

Vehicles & Objects

Create vehicles with realistic proportions and details—aircraft with proper wing geometry, ships with deck structures, or cars with wheels and windows.

Terrain & Landscapes

Generate natural landscapes with mountains, valleys, rivers, and vegetation. Place structures on terrain and watch them integrate seamlessly with automatic foundation filling.

Creative & Abstract

The agent can interpret artistic concepts and build abstract sculptures, pixel art, or fantastical creations.

Quick Start

Prerequisites

You'll need these installed on your machine:

• Python 3.11+ — Download from python.org/downloads
• Node.js 18+ — Download from nodejs.org
• uv (Python package manager) — Install from docs.astral.sh/uv
• API Key — Get one from Anthropic, OpenAI, or Google

Installation

1. Clone the repository

git clone https://github.com/mattzh72/minecraftlm.git
cd minecraftlm

2. Install backend dependencies

cd backend && uv sync && cd ..

3. Install frontend dependencies (this also builds the 3D renderer)

cd frontend && npm install && cd ..

4. Configure your API key

cp backend/.env.example backend/.env

Edit backend/.env and add your API key (see Configuration below for details).

5. Launch the application

./run.sh

(OR in development mode ./run.sh --reload)

This starts both the backend server (port 8000) and frontend (port 5173) with hot reload enabled.

6. Open your browser

Navigate to localhost:5173 and start building!

Troubleshooting

• Port already in use: Stop other processes on ports 8000 or 5173, or edit run.sh to use different ports
• uv not found: Make sure you've installed uv and it's in your PATH
• npm install fails: Try deleting frontend/node_modules and running npm install again
• Backend fails to start: Check that your API key is correctly configured in backend/.env

Configuration

MinecraftLM supports multiple AI providers. You only need one API key to get started.

Getting an API Key

Choose any provider and get an API key:

Provider	Where to Get Key	Environment Variable
Anthropic (Claude)	console.anthropic.com	ANTHROPIC_API_KEY
OpenAI (GPT)	platform.openai.com	OPENAI_API_KEY
Google (Gemini)	aistudio.google.com	GEMINI_API_KEY

Setting Up Your API Key

1. Copy the example environment file:

   cp backend/.env.example backend/.env

2. Open backend/.env in a text editor and add your API key:

   # Add only one of these:
   ANTHROPIC_API_KEY=sk-ant-your-key-here
   # OPENAI_API_KEY=sk-your-key-here
   # GEMINI_API_KEY=your-key-here

3. Save the file and restart the application if it's already running.

Supported Models

The app will automatically detect which providers you've configured and show only those models in the selector:

Provider	Available Models
Anthropic	Claude Opus 4.5
OpenAI	GPT-5.2
Google	Gemini 3 Pro

---

Made with 💜 in San Francisco and New York City