A lot of our eCommerce customers use the nopCommerce platform, and they often have questions. One of the most common is, How many products can a nopCommerce store handle? Unfortunately, the answer isn’t really cut and dry. But after reading this post you will know how to successfully integrate high-product SKUs into a nopCommerce store.
We’ve used nopCommerce successfully for many years and have found it to be an excellent ecommerce solution for the majority of our clients needs right out of the box. Most of the stores we built initially had around 200-1,000 SKUs and NOP out of the box shines with a store like this.
About four years ago a large reseller/distributor client approached us wanting to set up a store with up to 800,000 products. We had a hard time finding any documentation or anyone with personal experience to tell us exactly how many products a nopCommerce store could handle. So, we got to work by loading in products at a clip of 4,000 at a time, waiting a week between each batch to see how the store performed all the way up to 800,000.
Along the way we discovered that NOP needs some “enhancements” to handle stores with larger product SKUs. Since that time we’ve stood up additional stores with over 100K SKUs and we continue to refine and enhance our processes. This article is meant to provide some general information if you're looking to build an ecommerce store with a large amount of SKUs based on the lessons we learned along the way. It’s not an all-encompassing list of what needs to be done but it will certainly guide you on what you need to do to launch a large store in nopCommerce.
General Guidelines for a nopCommerce Site
A number of factors impact how many products can be listed. Site traffic and customizations are two of the biggest. At Foremost Media, we’ve developed hundreds of nopCommerce stores in the last nine years and have certified in-house nopCommerce developers on staff. The information below has been compiled from our development and hosting team, who have been involved in setting up eCommerce stores that carry over 800,000 products.
Here are some general guidelines our team has compiled:
1-5,000 Product SKUs — Smooth as butter
Specs and Requirements:
- Decent quality shared hosting
- Code base pretty much out of the box with custom skin
- 200 or fewer visitors per day to the store
5,000-25,000 Product SKUs — Needs more server resources
Specs and Requirements:
- Some minor customizations to store
- Moderate amount of traffic; 200+ visitors per day
- Small dedicated server or high-end shared hosting
25,000-100,000 Product SKUs — Needs custom coding and dedicated server resources
While nopCommerce 4.50 offers significant improvements in performance and efficiency of database calls, we’ve found that a site of this size will have serious performance issues without some customizations to the out of the box code.
Suggested changes to the site to hosting infrastructure with 25K products or more:
-
SQL Standard Database (Not the SQL Express that comes with Azure hosting, but full-blown and licensed SQL Standard). Here is why:
- SQL Express editions of SQL Server have a RAM limitation of just over 1GB per instance, while the Standard edition can use up to 128GB. The amount of RAM available greatly affects the performance of SQL Server. Loading more database data into RAM improves speed, but once the RAM limit is reached, data retrieval from the slower hard disk becomes necessary.
- While your dedicated web server has multiple processors with multiple cores each, SQL Server Express has a maximum compute capacity limitation of one processor and up to four cores per instance. This impacts overall performance. If the processor becomes overloaded, instructions will queue, resulting in slower performance. While this limitation may not pose a problem for smaller applications, higher traffic sites with lots of SKUs can become a bottleneck.
- SQL Server Express has a maximum database size limit of 10GB. This restriction determines the total amount of data that can be stored in the database. While 10GB is a considerable capacity for smaller stores with fewer products, we’ve found it typically creates timeouts and performance issues on larger stores.
Additional Issues with High-SKU Stores
There are a few other issues that need to be addressed when setting up a nopCommerce site with lots of SKUs, such as:
-
Theme needs to be optimized for performance and asynchronous database calls.
With the open nature of nopCommerce, there are many ways to achieve certain functionality within the theme. Some are more optimized than others. We review these methods and ensure we are only fetching the data required and that the call is not interfering with other site functionality. This ensures the increased number of categories or brands associated with the large product count does not hinder page performance.
-
Store sitemap out of the box is dynamically built when called by a bot or visitor.
In nopCommerce, the HTML crawler site map is created dynamically on the fly when it is accessed by a bot. When a site has a lot of products, the dynamic site map will time out or severely bog down the database every time it is hit. We typically build a cached version of the site map that updates on a scheduled task during off hours. The core nopCommerce functionality is designed to dynamically build the sitemap when it is called by a bot or visitor of the site. With a significant amount of content (blog articles, categories, products, etc.), the site will struggle to maintain performance during the generation of the sitemap. To work around this, we create a scheduled task that produces a cached version of the sitemap for larger content sets and have this run during off-peak hours to maintain performance. We then link these to the sitemap and disable it from the dynamic generation so the sitemap loads quickly and keeps the site in a performant state.
-
Create custom pages for Category/Manufacturer bypassing core nopCommerce and building out in our own way for speed purposes.
Similar to the theme, core category functionality references code that calls back too much data in a method that requires other functionality to wait. The larger categories work better with a custom page that allows us to optimize the calls being made and the timing of them, keeping the site loading fast.
-
Review maximum numbers of products in a specific category.
Break products down into better categories as some blanket a large number of products, which corresponds to a large number of ACL record/spec filtering SQL lookups. No matter how efficient the calls are, returning thousands of products can still have an effect on the site. In these instances, we review the total number of products and look to sub-categorize them, allowing us to offload much of the data to a different view. Sometimes this might equate to a shuffling of the categories we have in place, depending on our customers’ needs.
-
Image optimization and storage in the database.
nopCommerce has the ability to save images as a site file or as data within the database. Images stored in the database will be slightly slower to load initially, and will increase the amount of storage your database requires. On sites with a lot of products, this can cause issues with speed for all data access. More importantly, understanding image optimization can save significant strain on the site. Using an image that is sized properly for what you need and compressed can save a significant amount of data transfer for your site. Putting proper images on the site will inevitably help every page load faster, which is compounded when you factor in a large product count.
Is 100,000+ products in one store the right strategy for you?
I still question if a giant store is the right solution for most companies from a marketing strategy perspective. I could make a strong case for the idea of building smaller stores focused around a specific industry or theme being easier to get traction on search engines. However, this might be a good topic for another blog post. Clearly there is demand for large stores, and if you need help we would love to discuss your project.
Collaboration Drives Innovation and Success
nopCommerce is a flexible and enterprise level open-source ecommerce shopping cart solution that has revolutionized online selling since 2008. As a Gold nopCommerce Development and Solutions Partner, Foremost Media has launched hundreds of sites on this robust platform and has been through the certification process. Our collaboration with nopCommerce ensures tailored customizations, cutting-edge designs, and unwavering support throughout the development cycle. Together, we drive innovation and success in online commerce, providing outstanding solutions that exceed client expectations.