Gorilla Shop
AI that recognizes materials in photos
An R&D project funded by a BRIdge Alfa grant: a neural network recognizes building materials and interior fittings in a photo, then points to matching products. Gorilla Shop is a research and development project delivered with a BRIdge Alfa grant worth PLN 700,000. Funding came from the Smart Growth Operational Programme (POIR), Measure 1.3.1. The goal was to build an AI system that recognizes building materials and interior fittings in photos. The system matches the recognized object to product pages in an online store.
- Customers do not know the product name. Someone sees a tile at a friend's place, a faucet in a hotel or a panel in a photo online. They have a picture but no idea what the product is called or who makes it.
- Text search falls short. A classic store search works on words. If users do not know the right word, they will not get a relevant result.
- Browsing categories by hand. Without image search, all that is left is paging through categories one by one. Construction catalogs are large, so customers give up quickly.
- Huge visual variety. Building materials and fittings differ in texture, color, pattern and shape. Phone photos come with varying light, angles and backgrounds.
- A neural network was needed. Simple rules and filters cannot cope with that much variation. The task required a model trained on images.
- A research project by nature. There was no ready-made solution to copy at the outset. The project had to go from hypothesis to working prototype.
- The photo as the query. We started from the premise that the search input is an image, not text. The user uploads a photo and the system returns a list of matching products.
- Computer vision instead of rules. We chose a neural network approach. The model learns to recognize the visual features of materials and objects from examples, not from hand-written conditions.
- Two tasks in one process. The system first recognizes what is in the photo. It then compares that with the catalog and points to the most similar products.
- Linked to the product page. The recognition result leads straight to a product page. Customers do not get just a label, but a place where they can buy.
- Iterative research work. We tested the model on successive sets of photos and improved it based on its errors. That is the typical path of an AI R&D project.
- Grant accountability. We ran the work to meet the requirements of the BRIdge Alfa program. Every stage had a defined goal and documentation.
- An industry where images say more. We chose building materials and interiors because customers here usually start from visual inspiration. A verbal description of a tile or cladding is rarely enough to find the product.
- Search by image. Users upload a photo instead of typing a phrase. It is the only sensible way to search when the product name is unknown.
What we built it with.
Defining the research problem
We described which objects the system should recognize and under what conditions. The scope was building materials and interior fittings.
Data preparation
We collected and organized product photos for training the model. The quality and variety of the data determine how well the model handles phone photos.
Building and training the network
We designed a computer vision model and trained it to recognize categories and visual features. We compared results across successive versions.
Catalog matching
We connected the recognition output to the product database. The system points to the product pages closest to the object in the photo.
Testing and project reporting
We tested performance on photos outside the training set. We documented the results for the grant program.
What changed.
See other projects.
Want customers to search for products by photo?
We will assess your catalog and data, then propose a path to rolling out image search. In industries with large catalogs, customers often know what a product looks like but not what it is called. Image search turns a phone photo into a query the store understands. A project like this does not have to be funded entirely from your own budget, as R&D programs support work on AI. The prerequisites are a well-defined problem and data that a model can learn from.
Let's talk about AI in your store →