---

I made a 3D rendered browser based computer craft controller, letting you inteface with a minecraft world from the comfort of the browser.

Computer Craft

So a bit of context, Computer Craft is a mod for minecraft that adds computer and turtles that can be controlled with lua code. The turtles have the ability to interact with the world by moving around, mining & placing blocks, playing audio, etc. I’m using a modern fork called CC:Tweaked as well as the Advanced Peripherals extension to get additional features.

Over the years I’ve made plenty of things in computer craft, a FTP drive that stores it’s data in a minecraft world. As well as 4bit colour palette partitioned video player.

However, this is the 3rd time attempting this specific project. Previously, I’ve failed each time due to lack of experience or not planning properly. My last attempt failed since I hadn’t correctly modelled how computer craft works, making it impossible to implement correctly meaning I had to start over.

This time however, I decided I was going to plan how it would work.

Design

My approach this time required me to actually plan it out. I’ve

I’d settled on a 3 device arrangement:

• A Browser UI
• A Websocket proxy
• A Computer Craft Turtle

The websocket proxy was needed to connect to the turtle. It acts as a server, allowing the browser and computer craft to interact.

Locks

The main thing I implemented this time around was device locks. In my previous attempt I had made it so only one task could execute at a time. This was to ensure you couldn’t disrupt a current task by performing an action when it was unexpected, such as moving during a calibration.

However, this meant that I couldn’t ask about the device state whilst it was doing a long running task. This approach was inoperable, it made things too difficult to work with.

In this verison, anything that can’t be disrupted has a lock attached, when a request is made it waits until this lock is free before running. This meant I could still respond to queries and inventory actions whilst a movement action was ocurring.

Networking

Implementing this initial design only took a day, by end of it I had a rough prototype

For a ‘lil flair, I decided I was going to style the UI off the minecraft design system.

During this prototype, I was using eval to perform action, this would run the lua code using load and return the result as JSON.

I had underestimated how powerful this was going to be. My original design was going to implement every action as a seperate packet type, but using eval I could implement 90% of what I needed just on the client.

Rendering

When rendering, I wanted to experiement with React Three Fiber. This is a react layer of Three.JS, so you can tie it into the react render cycle. This helped me simplify the rendering and add listeners to meshes so adding tooltips and interactions was simple.

One of things that made this project possible was mc-assets, this is a package for access minecraft assets. This was a package made for a seperate website by Zardoy, but was seperated out when they realised how hard a problem this was.

However, once I implemented the ability to scan large amount of blocks this wouldn’t work anymore. I was creating a mesh for every block, even after to culling blocks it still wasn’t enough.

I had to switch over

Polling vs Event

One of the main improvements I made was moving away from polling and towards events.

Originally, I was agressivng polling the device for a heartbeat every 0.5s to keep it in sync. This worked, but was going to cause issues on large scales. I started moving updates by monkeypatching the turtle’s APIs.

This meant whenever a block was mined or scanned, an update packet was sent. This helped to prevent desync from the turtle’s knowledge.

The Result

I slowly toiled away at this, improving the UI, rendering and fixing bugs. Until I had a fully rendered, controllable turtle interface:

The Future

I’ve taken a break from this project as I’ve lost motivation to continue it, although I’ll probably pick it up again at some point. There’s a few ideas I want to use this force, one being to create a publically accessible WebUI that would let people control their own turtle on a world.

I’ve been working on a JavaScript API for it, I managed to the API to this point:

const turtle = await findTurtle("Ben");

while (true) {
    await turtle.digFront()
    await turtle.goForward();
    await turtle.scan(1);

    const surroundingBlocks = world.getAdjectBlocks(turtle.position);
    const ores = surroundingBlocks.filter(block => block.name.includes("ore"));
    for (const ore of ores) {
        await turtle.mineBlock(ore.position);
    }
}