Tech Talk: Scaling WordPress With DevriX & Pagely

Hello, dear readers, and welcome to the second episode of the series “Webmasters Unleashed: Empowering Websites with Pagely and DevriX”.

Today, we will be focusing on scaling WordPress for high traffic websites, and how it can cater to both startups and enterprise sites. 

We will be discussing strategies for optimizing website performance, managing increased traffic and user loads, implementing efficient caching mechanisms, and utilizing cloud enterprise hosting solutions.

Before you continue to read today’s Tech Talk, head over and check out the first episode, if you haven’t already.

Pagely Q&A

From a hosting perspective, what are the key infrastructure considerations and resources required to support the scaling needs of a WordPress website transitioning from a startup to an enterprise-level solution?

For a small site, you need a web server and a database server – they might even be the same server. As you scale up, you need more robust web and database servers. Depending on your site, you may need a caching layer; you may need a CDN; you may need to decouple your web and database servers.

You will need to pay more attention to the software, as your business-specific themes and plugins will become more of a performance issue as your traffic volume increases. A managed WordPress host, that has domain-specific expertise in this, is a wise investment, as they can help with every part of the above.

Can you provide insights into load balancing, auto-scaling, and server configurations that help accommodate increased traffic and ensure high availability for enterprise-level WordPress websites?

The first insight I’d like to give is, “are you sure you need this?” Load-balancing and auto-scaling are one way to enable a site to handle higher volumes of traffic, but they add a lot of additional complexity. Once you introduce auto-scaling, you now must ensure your content is synced between servers in near-real-time or introduce a shared-storage component. 

A load-balancer is a potential single-point-of-failure, and the health checks it needs to know which back-end servers are available may increase the server load on those servers. For many use cases scaling up (making your servers larger) to handle your peak traffic, rather than scaling out (adding more servers), is more reliable.

If you do really need this, the best piece of advice I can give is “test early and often.” Make sure every piece can run independently. Make sure every component can handle failure, both of other copies of itself and of pieces in other layers of the stack. Randomly turn pieces off every so often to ensure that nothing has changed.

Use configuration management tooling, to keep all the components identically configured. Be sure to check recovery as well – that when one of your replicas comes back from a failure, that its configuration and content are properly synced back from the good copies. 

Yes, this is a lot of additional work, which is why I asked first to be sure that you need it – not everyone has a business requirement for HA.

What are some recommended mechanisms that hosting companies can offer to enhance the performance and scalability of WordPress websites? 

In my experience, caching almost always is the most important performance boost. There is the obvious fact that you must have a cache, of course, but configuring and tuning your site to take the best advantage of that cache is where you’ll really get the most benefit.

Caching content longer, and only purging the cache when necessary, helps. Make sure your site is cacheable – for WordPress, anything that involves user-unique content, even something as simple as a tracking cookie, can prevent content caching and drop performance drastically.

How can hosting providers assist in managing traffic spikes and handling increased resource demands during periods of high user activity for scalable WordPress websites?

A hosting provider can work with you before your high-traffic events, to be sure your server is right-sized for your anticipated volume of traffic, and also can help you make your site more suitable for that traffic volume, such that you might not need to make any hardware changes. By adding a caching layer, or identifying ways to make your site more performant, you might not need to do anything special for your event.

What are some security measures and protocols that hosting companies can implement to protect enterprise-level WordPress websites and ensure data integrity and privacy? 

Most hosts offer a WAF (Web Application Firewall) feature, that will prevent the most common attacks, and which can be tuned for your specific site. Some also offer DDoS protection, though many choose to focus on hosting specifically and can help you integrate with a third-party solution for that. (Many DDoS protections double as CDNs, which will also make your site perform better as well.) 

A host that specializes in WordPress will have additional protections specific to that application, and can help you keep WordPress and your themes and plugins up-to-date, often automatically.

What’s the key benefit of using an enterprise-level hosting solution? 

The most important benefit, by far, is experience. Even if your internal IT teams have expertise in all the different technologies involved in hosting a website, outsourcing that function allows them to work on projects more directly tied to your business. For an enterprise host like Pagely, that is our business.

DevriX Q&A

As a WordPress agency, what are the key considerations and best practices when it comes to scaling a WordPress website from a startup-level solution to an enterprise-level solution?

When launching a new website, site builders can be scrappy. Putting up an MVP is an easy feat, an exercise that often involves a premium theme, a powerful builder, and heavy components that leverage the power of the plugin ecosystem.

This often works well enough in the first months (or while the website generates several dozen/a couple of hundred daily visits).

As brands grow in popularity and gain following – or start to introduce paid traffic, turnkey solutions become threatened by three factors (we call them the 3S): Speed, Stability, and Security.

A minor bug – or a rarely visited website – is prone to regressions or slow load times, but this doesn’t pose severe business risks due to the minimal traffic. 

However, accelerating further inevitably impacts the user experience due to excessive SQL queries or dozens of scripts and styles loaded across the site, outages coming up every now and then, and plugin updates causing regressions among each other.

Part of the scaling process is identifying the bottlenecks and incrementally optimizing them one at a time. Minifying and combining assets being a standard procedure for improving load times. 

Offloading image assets to S3 and performing lossless compression. Hooking into filters to optimize SQL queries. Rewriting some heavy components into simplified, lightweight plugins that solve the core problem.

WordPress can truly scale – as we’ve proven dozens of times, generating hundreds of millions of monthly views across our larger publishers and B2B magazines. But performing professional engineering practices is a hard requirement – which is the norm for any WordPress enterprise solution, regardless of its core technical stack.

Can you provide insights into building modular and extensible WordPress architectures that can easily accommodate growth and scalability as the website’s traffic and user base increase?

There are different aspects to scaling a solution.

First and foremost, in-depth understanding of computer architecture and operating systems is pretty helpful. WordPress is a CMS built with PHP, HTML, and JavaScript, running on a web server (like nginx or Apache) with core modules (php-fpm or mod php). 

Content (anything from posts and pages through options and users) is stored in a MySQL database – or an RDS instance in AWS’s case.

User sessions run on the corresponding processes and take a certain volume of resources – including processing power and data volume pulled from the database.

Growing platforms start to hit certain limits. But this could be caused for different reasons, for instance:

• High traffic (concurrent users on site)
• Heavy SQL queries (suboptimal data pulls fetching tons of data or a number of JOIN statements causing a heavy load)
• Too many queries (plugins fetching data through a long set of queries that aren’t necessarily optimal)
• Excessive non-cached sessions – due to heavy traffic, bots, DDoS attacks, maxing out PHP processes
• Browser overload due to too many scripts or timing out assets
• General sluggishness due to large page sizes, lack of lazy loading and critical CSS, degraded CSS animations, etc.

Work with your agency to establish the weak links. Are you maxing out the CPU usage? Or running out of RAM? Are there excessive I/O operations on the disk? Is the MySQL instance powerful enough to accommodate that load?

Professional architectures are designed in a way to mitigate these outages. In certain cases, reengineering the data layer could help normalize the database and running more optimal queries. Or moving certain data points to a different storage. 

Separating the web server instance from the database one is a standard best practice that regular hosts often don’t present – but is a pretty common procedure for our Pagely hosted customers.

What are some recommended approaches for managing large databases and optimizing database performance on WordPress when dealing with high traffic volumes?

Running a benchmark of your current build can be a great starting point.

One of the simplest (and fastest) ways to assess problematic queries for every site is using a profiling plugin like Query Monitor. The app runs on every page load and compiles all SQL queries run on the page (both front-end and back-end). This could outline some expensive queries generated via a premium theme or heavy plugins.

If possible, enable MySQL’s slow query log. Site owners and developers can set a threshold logging all expensive queries taking a certain amount of time to run (say, over 2 seconds or 5 seconds). Keep an eye on the log and optimize accordingly.

Integrating a better profiling and monitoring system such as New Relic provides deeper intelligence into run times, the number of callbacks through each system (including function calls), and other areas for improvement.

WordPress is a robust system designed to solve every case. This necessitates a loosely defined database able to accommodate any data types (leading to the early introduction of custom post types in the core CMS). 

At scale, creating custom database tables and running lightweight database layers may be a preferred strategy. In some cases, building micro apps and fetching data asynchronously – or via the REST API – may yield better results for high-scale websites, too.

Understanding the triggers causing excessive load is the first step. Refining the database layer and optimizing SQL queries can alleviate the database further. And implementing the right caching layers will get rid of constant calls pulling the same data set all over again.

Can you highlight the strategies and techniques employed for WordPress websites that successfully scaled from startups to enterprise-level solutions?

WordPress websites are no different from other real-life examples.

There are $30 chairs on the market and $5K ones with ergonomics designed for people spending 12 hours a day working professionally.

NASCAR and Formula 1 don’t run races with the mass-produced cars on the market.

A Target screwdriver used twice a year isn’t up to the standards of professional construction workers and mechanics. 

Bottom line, shifting from a hobbyist website to a professional build requires a professional level of attention, expertise, and the right architecture – hosting environment and code base. 

And working with the right providers experienced in scaling enterprise-grade solutions is the first step to success.

How can agency expertise contribute to scaling WordPress websites effectively?

Apart from computer science know-how and understanding the core principles of web solutions, in-depth WordPress expertise is integral to making the most out of a WordPress build.

One aspect of scale is understanding the weak spots of WordPress. Certain outdated vectors such as xmlrpc.php can lead to brute force attacks or taking excessive server load if left unattended. 

Directory indexing can expose sensitive files unless the host has disabled indexing. Using prepared statements and the right functions to create queries (instead of raw statements) prevents SQL injections and other loopholes. 

Many shortcomings of WordPress arise in most scaling projects – and experienced teams often know what to expect and how to mitigate them. 

Avoiding bloated plugins, moving to a lightweight theme specifically designed for the project, creating custom database table when it makes sense, improving the editorial workflow with core components and the recent Gutenberg updates are just a few of the common optimizations veteran engineering teams perform. 

The core Pagely team additionally optimizes the server layers for maximum performance, introduces redis caching opportunities for database queries, patches zero-day vulnerabilities on a regular basis (as reported), and works closely with the agencies on offloading assets externally to cloud systems or managing resources across multiple instances. This is what enables startups to grow consistently for years to come.

---

David (he/him) is a devops engineer at Pagely, with over a dozen years of experience with WordPress (and far more years than he cares to admit working in IT generally). He specializes in high-availability and resiliency for WordPress sites and is very bad at video games.

Mario Peshev

I work with SMEs, agencies, and individual consultants on increasing revenue and securing long-term business opportunities. Having built my first website in 1999 and with over a decade of experience in software engineering and business consulting, I’ve dedicated my career to leveraging technology to solve complex business problems and drive growth. I’m passionate about open-source software and its potential to change the world.

As an active contributor to the WordPress community, I’ve not only coded, but also offered strategic advice to businesses navigating the digital landscape with WordPress. I try to bring omnichannel customer experience that expands the traditional business-to-consumer journey into a myriad of additional channels. In the context of eCommerce, it’s not just about the single flow of a customer browsing a catalog and making a purchase, but longer pathways from support interactions to post-purchase automation.