Lego Art Remix

Easily remix Lego Art sets into custom mosaics using computer vision
Made with ♥ by DebSource Code

You can also change the available piece count after selecting the input image

Supported image formats are dependent on your browser's compatibility

Due to the nature of the Lego Art sets, images with transparency aren't fully supported

Example use case - this is a full Hogwarts crest created by the algorithm with just 1 copy of set 31201

These are some articles and videos featuring Lego Art Remix

Some are quite interesting even outside the context of this tool in particular, since they go into the history of Lego mosaics

Note that some were made when the tool was older

Made with ♥ by DebSource Code
What is it?

In 2020, The Lego Group released the Lego Art theme, which allows people to create a predetermined image using lego studs.
Lego Art Remix lets you upload your own image, and then uses computer vision to use the studs from a Lego Art set that you already have to recreate the image.

This project is not affiliated with The Lego Group

Performance and Security

The computer vision techniques used are pretty inexpensive (with the exception of optional depth map generation), and the resolutions being dealt with are naturally quite low, so as of the time of writing, the algorithm runs quite quickly. This allows for it to be run on the client, and on the machines that I tested, it ran in near real time.

The most computationally expensive part of the process, apart from depth map generation, is generating the instructions, since even pdf generation is done client side.

Since it runs almost entirely within the browser (see the source code), no image data is sent to a server and so it's very secure. This also makes it much easier for me to maintain and host. The only server code consists of simple increments to anonymously estimate usage for the purposes for tracking performance in case the static deployment needs to be scaled up, and for the counter in the about section.

Even the deep neural network to compute depth maps is being run entirely within the browser, in a web worker, using a modified version of ONNX.js. I've compiled a version of the library based on this pull request, with a small additional change I made to support the resize operation in v10. The model used is MiDaS - more specifically, the small ONNX version which can be found here. It's incredible that ONNX.js allows us to use amazing work such as this within a web browser.

Citation for Model Used Ranftl, René, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun. "Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer." (2020). IEEE Transactions on Pattern Analysis and Machine Intelligence

Bugs, Feature Requests, and Algorithm Improvements
Direct any concerns or ideas for improvements here

As of the time of writing, I don't have all of the sets, and I haven't had much time to test. As a result, there's probably a few bugs, so let me know if you find any.

Algorithm improvement ideas are always welcome. Improvements that maintain the efficiency to within a reasonable degree would allow the algorithm to keep running on the client, which I really like.