Huge wall of text incoming for anyone interested in more detail on the tech underneath this... :)
Nearly 100% of the game code was written from a combination of Opus 5, Fable, and Gemini 3.7 Flash. Generally we used:
* Fable for high-level architecture and project setup, including planning out new features and how they interact with other systems.
* Gemini for anything visual, which it seems to excel at, like HUD/UI, animations, particle systems, etc.
* Opus 5 for the grunt-work, writing code after we've finalized design plans and implementation details.
* All static sprites other than the characters were generated in Unity with Gemini 2.5 Flash Image (cheap, fast, and consistent).
* All animated sprites (including characters) are actually generated from code from Gemini; generating pixel-perfect spritesheets (and at a consistent PPU!) is hard enough with the fuzzy pixels that image models usually spit out, especially when they go through resizing/restyling pipelines, etc.
* To get around this design limitation, all of the character parts (hair, eyes, beard, arms, legs, etc) are all generated from code (snippet) that guarantee every pixel is actually a hard-lined pixel, and lets us drive animations from data rather than images (which makes restyling characters way more maintainable). We then can define characters as a collection of parts (including animations for those parts, generated in the same way).
We have two kinds of crafting in the game. In the beginning, you're told what to draw and are awarded if you can draw it. Later, you can draw anything you want and new usable game items are generated for whatever items you draw.
For did-you-draw-this-item:
1. NPCs you meet typically have a request for you to draw some specific item.
2. We used Gemini to mass-generate some "hint" templates for every item that basically tell you how to draw it, if you want that hint. After you've drawn something, we just do a basic similarity check of "how many pixels-different are you from a template".
3. If you are close enough, we can skip all the fun AI stuff here and just give you the item immediately. This cuts down on AI wait times, token spend, etc, and also avoids nasty issues where a player might follow a template and still be told what they drew isn't actually what they were trying to draw.
4. If you went off the rails and drew your own interpretation, we convert the pixel canvas into an image and pass it into Gemini 2.5 Flash to ask "Is this a picture of XYZ?" to use AI to classify whether you accomplished the goal or not. If you did, you get the same item as if you'd drawn an expected template.
For draw-any-item:
1. Your Crafting Table has a full canvas that you can draw anything in.
2. When you submit a drawing, we convert the canvas to an image and pass that into Gemini 2.5 Flash Image and ask, in short, "What is this a drawing of?" It tells us it's a teapot (for example).
3. When we get a result back, we pass that back to Gemini 2.5 Flash again and say, "Here are all the composable systems in our world that describe how items work. Please generate the item data for a functional teapot.
4. In parallel to #3 (to minimize wait time), we also prompt Gemini 2.5 Flash Image to generate us a teapot in the style of our game items.
5. When we have both the game data and sprite back, we write that to our item database and mint the new item in the game, then reward it to the player as a usable item they just crafted.
The existing systems in the game carry the weight for what it can do. For example, you might be able to water your crops with a generated teapot if it grabs functionality off the base Watering Can item. Or if you craft a Mega Torch, it might have the functionality of a base Torch but with higher brightness values. Or if you craft a Glowing Teapot, you might get a tool you could water your crops with but also gives a slight glow around you, and can be placed in your farm for more permanent light around it. Or maybe you just draw a neat little chair and table that you want to decorate your farm with.
One other gameplay note on this: we use the farming system so players can harvest plants that they turn into different-colored dyes in order to use when painting, so farming feeds back into what colors you can use in your art, which also affects how the AI interprets your art (good luck trying to draw something wet without blue paint!).
Lastly, we also have open-ended harvesting (using tools on in-game objects). The tools you craft don't only copy existing systems functionality. Whenever you use any tool on any object in the game, we query the recipe database to see if there's a known recipe for what happens when, say, you use a shovel on a tree.
1. If a recipe already exists, you just get that item (easy).
2. If a recipe doesn't exist, we dump into another multi-step generative process:
3. We ask Gemini 2.5 Flash "What happens when you use [Tool] on [Object]?" to classify into distinct interactions: modifying the object, modifying the tool, harvesting a resource, and/or triggering other in-game effects.
4. Based on the type(s) of interactions, we then pipe that back into Gemini 2.5 Flash and ask it to generate the data for that interaction, based on our existing pre-coded systems that objects support.
5. For example, using a crafted Superfiery Tinderbox on a Tree might set the tree on fire, and turn it into a Burning Tree. This'll probably add a big light glow to the tree, and since it's a new object there's new recipes to be found again (for example, using an Axe on the tree might now give Burning Wood, or using your Watering Can on the tree might turn it back into a regular Tree).
5b. Other examples: a shovel on a tree might harvest worms, which you could then turn around and use on dirt to transform the dirt into rich soil, which you could then use your shovel on to get fertilizer, etc.
6. Finally, we save whatever recipe(s) we created from this tool-object pairing so the interaction can be instant from then on in the game.
Huge wall of text incoming for anyone interested in more detail on the tech underneath this... :)
Nearly 100% of the game code was written from a combination of Opus 5, Fable, and Gemini 3.7 Flash. Generally we used:
* Fable for high-level architecture and project setup, including planning out new features and how they interact with other systems.
* Gemini for anything visual, which it seems to excel at, like HUD/UI, animations, particle systems, etc.
* Opus 5 for the grunt-work, writing code after we've finalized design plans and implementation details.
* All static sprites other than the characters were generated in Unity with Gemini 2.5 Flash Image (cheap, fast, and consistent).
* All animated sprites (including characters) are actually generated from code from Gemini; generating pixel-perfect spritesheets (and at a consistent PPU!) is hard enough with the fuzzy pixels that image models usually spit out, especially when they go through resizing/restyling pipelines, etc.
We have two kinds of crafting in the game. In the beginning, you're told what to draw and are awarded if you can draw it. Later, you can draw anything you want and new usable game items are generated for whatever items you draw.For did-you-draw-this-item:
1. NPCs you meet typically have a request for you to draw some specific item.
2. We used Gemini to mass-generate some "hint" templates for every item that basically tell you how to draw it, if you want that hint. After you've drawn something, we just do a basic similarity check of "how many pixels-different are you from a template".
3. If you are close enough, we can skip all the fun AI stuff here and just give you the item immediately. This cuts down on AI wait times, token spend, etc, and also avoids nasty issues where a player might follow a template and still be told what they drew isn't actually what they were trying to draw.
4. If you went off the rails and drew your own interpretation, we convert the pixel canvas into an image and pass it into Gemini 2.5 Flash to ask "Is this a picture of XYZ?" to use AI to classify whether you accomplished the goal or not. If you did, you get the same item as if you'd drawn an expected template.
For draw-any-item:
1. Your Crafting Table has a full canvas that you can draw anything in.
2. When you submit a drawing, we convert the canvas to an image and pass that into Gemini 2.5 Flash Image and ask, in short, "What is this a drawing of?" It tells us it's a teapot (for example).
3. When we get a result back, we pass that back to Gemini 2.5 Flash again and say, "Here are all the composable systems in our world that describe how items work. Please generate the item data for a functional teapot.
4. In parallel to #3 (to minimize wait time), we also prompt Gemini 2.5 Flash Image to generate us a teapot in the style of our game items.
5. When we have both the game data and sprite back, we write that to our item database and mint the new item in the game, then reward it to the player as a usable item they just crafted.
The existing systems in the game carry the weight for what it can do. For example, you might be able to water your crops with a generated teapot if it grabs functionality off the base Watering Can item. Or if you craft a Mega Torch, it might have the functionality of a base Torch but with higher brightness values. Or if you craft a Glowing Teapot, you might get a tool you could water your crops with but also gives a slight glow around you, and can be placed in your farm for more permanent light around it. Or maybe you just draw a neat little chair and table that you want to decorate your farm with.
One other gameplay note on this: we use the farming system so players can harvest plants that they turn into different-colored dyes in order to use when painting, so farming feeds back into what colors you can use in your art, which also affects how the AI interprets your art (good luck trying to draw something wet without blue paint!).
Lastly, we also have open-ended harvesting (using tools on in-game objects). The tools you craft don't only copy existing systems functionality. Whenever you use any tool on any object in the game, we query the recipe database to see if there's a known recipe for what happens when, say, you use a shovel on a tree.
1. If a recipe already exists, you just get that item (easy).
2. If a recipe doesn't exist, we dump into another multi-step generative process:
3. We ask Gemini 2.5 Flash "What happens when you use [Tool] on [Object]?" to classify into distinct interactions: modifying the object, modifying the tool, harvesting a resource, and/or triggering other in-game effects.
4. Based on the type(s) of interactions, we then pipe that back into Gemini 2.5 Flash and ask it to generate the data for that interaction, based on our existing pre-coded systems that objects support.
5. For example, using a crafted Superfiery Tinderbox on a Tree might set the tree on fire, and turn it into a Burning Tree. This'll probably add a big light glow to the tree, and since it's a new object there's new recipes to be found again (for example, using an Axe on the tree might now give Burning Wood, or using your Watering Can on the tree might turn it back into a regular Tree).
5b. Other examples: a shovel on a tree might harvest worms, which you could then turn around and use on dirt to transform the dirt into rich soil, which you could then use your shovel on to get fertilizer, etc.
6. Finally, we save whatever recipe(s) we created from this tool-object pairing so the interaction can be instant from then on in the game.