Emmet – Brixx    |    Minifigure-Finder

This project is an extension of our previous work in the area of terminal building block classification and shifts the focus to minifigure recognition and classification. Through the use of modern AI technologies the project aims to develop a reliable and efficient pipeline capable of accurately classifying minifigures. A particular feature of this project is the ability to identify minifigures that have been can be assembled in different ways. The main goal of the project is therefore not only to identification of such diversely assembled minifigures, but also to design the system in such a way that it can identify the correct construction method and the respective building blocks.

System requirements

Recognition of minifigures

The system should be able to recognize minifigures in a variety of scenarios and lighting conditions.


Special attention is given to the classification of the torso of the minifigures. This is important for the accurate identification and sorting of the figures.

Finding the associated components

The system must not only identify the minifigure as a whole, but must also be able to assign the specific parts such as the head, arms, and legs that belong to the figure.

Linking minifigures to sets

Another requirement is the ability to automatically link identified minifigures to their associated sets. This could be done through a database that contains information about the affiliation of minifigures to specific sets.

Practical application

Automatic recognition and classification of minifigures is a challenging task that exceeds in its complexity far exceeds that of interlocking toy bricks classification. Minifigures are composed of a large number of different components and can be combined in numerous ways. With over 15000 ( Source: different minifigures and a of possibilities for the assembly results in an exponentially growing number of combinations, which have to be classified.

Data set

Our dataset includes a diverse collection of 8,000 different minifigures that come from a variety of sources. These include photos from collectors as well as images from publicly available databases and websites. It is remarkable that the dataset originally consisted of only 100 minifigures and grew to its current size within half a year. This broad database allows us to address a diverse range of classification problems while achieving robust and reliable classification performance.


Our powerful infrastructure allows us to efficiently train even sophisticated deep learning networks internally. The combination of high quality data and our state-of-the-art hardware ensures that we can develop accurate and reliable models.


The Mean Average Precision is a metric for measuring an object detection model and includes the following statistics: Confusion Matrix, Intersections over Union, Recall and Precision.

Recognition accuracy is defined as the quotient of the sum of true positive and false negative test results. In our model, we arrive at 99.2 % correct predictions.

The current data set allows the classification of 8,000 different minifigures.

The average processing time for classifying a minifigure is 800 milliseconds. It should be noted that no specific optimization attempts have been made to reduce this processing time. This leaves room for potential improvements in future development phases. The processing is done server-side, which provides the opportunity for continuous updates and improvements.

Future work

Although our results to date are promising, we recognize that there is always room for improvement and further developments. In the upcoming phases of the project, we plan to take the following steps:

We are currently developing an in-house app that will allow users to conveniently use our AI-driven minifigure recognition system via mobile devices. The app development is done exclusively in-house to ensure both control over the project and security of user data.


Emmet Software Labs GmbH & Co. KG
Hertzstr. 6
32052 Herford

Phone: +49 5221-763 999-0