NFQ software developers took a challenge to build a platform that could successfully handle growing amounts of data (at the start of June 2019, there were 1 million products, now, in 2021, there are around 20 million) and to do it in 6 months before the marketplace's launch. Thus Smarketer was born.
Smarketer.Shopping is a well-known German price comparison website. Our team, programming in the Kotlin language on an Elasticsearch search engine, built a marketplace on AWS, suited for rapid growth where end-users can quickly and effectively find desired items and compare their prices across different suppliers.
And it worked. The system's unique visitors are growing by an average of 66% every 6 months and continue to grow despite the COVID-19 situation in the market during 2020-2021.
"Smarketer was built with a considerable emphasis on stability and scalability. In the not-so-distant future, we seek to boost the platform's monetization capabilities and make it possible to increase the total product count radically. The level we aim to reach eventually is 100 million products," says Vainius Venclovas, Team Lead at NFQ.
While developing the Smarketer.Shopping platform, we tackled an important problem. The large file sizes of high-resolution product images were slowing down the load speed of the web platform. Considering that the planned number of products was huge, image processing had to be managed delicately. To import millions of product images every day, the NFQ's engineers took advantage of Amazon Web Services.
More specifically: when product images are imported, their size is reduced in real-time using the AWS Lambda compute service. Once processed, images are stored in the AWS S3 data warehouse.
And that's just a single small process in a well-designed ecosystem. To mention just a few others:
"In developing the project, the biggest challenge for me personally has been learning how complex the mechanics can be under what from the outside may look like something very simple. And thinking through all the details and situations that could occur," – says Paul-Willem Thum, Senior Team Lead at Smarketer.