When the Script Comes Full Circle

A look back to where my journey as a software engineer in e-commerce began.

Hi, I’m Daniel — a software engineer who’s spent the past five years working in the e-commerce space — from a small startup (which this article is about) to two of the biggest retailers on the African continent.
Lastly, welcome to my first blog — hope you enjoy!

Disclaimer: This is a bit of a brief reflective write-up, looking back at where it all started.

Where it began

My journey in the tech space all started in May 2021, when I was hired as a Junior Software Engineer for an on-demand grocery startup based in Stellenbosch called Quench. To put it into context, I joined at a time when the world was under lockdown and Checkers’s Sixty60 wasn’t a household name yet (launching just under two years prior). I started at that time in a space that I was soon to see take off and change the way millions shop for everyday goods.

Interestingly enough, about a year before — during lockdown level 5 (May 13, 2020, to be exact, based on the repo’s initial commit) — I started hacking together my own version of an e-com platform called Graple.

The aim of Graple was to scrape product data from major retailers — Woolworths, Checkers, and PnP — and present it for smart price comparison of similar items. The original idea began as a POC to aggregate grocery data and build a platform like pricecheck.co.za, but focused on everyday groceries, with eventual plans for delivery — similar to what OneCart and Quench later offered.

Of course, at the time, I thought e-commerce was just about building a website with products and a cart. But stepping into the real world of e-commerce quickly proved otherwise — it’s far more than just an online platform. Behind it lies an entire ecosystem: logistics, payments, operations, integrations, customer support, and more.

But anyways I continued on my pursuit to create this MVP to smartly compare like and simlar products across retailers. I managed to scrape 14k+ products from Checkers and PnP so far, still busy tweaking and working around their rate limiters and DDos protections.

But then it was Woolworths' turn. 👀
I decided to include them too — and that’s where things got interesting. I built my script to maximise the number of products I could scrape in one request, i'd often push the page size limits to see what's the max the API would allow. If memory serves me right, I worked my way up to page size of 1 million products.

The result?

I started seeing a consistent Nginx 502 Bad Gateway response — and saw the same error page across all my devices, whether on Wi-Fi or mobile data (in the hope I was maybe just blocked and paired with an incorrect status code/response)🫣 The uniformity of the 502 across networks rules out client-side issues, and Nginx acting as a reverse proxy typically throws this when it can’t reach the upstream server — often due to a backend service timeout, a crashed container, or resource exhaustion (CPU/memory). My assumption was that the database most likely fell over, causing the application server to become unresponsive and triggering the 502 errors from Nginx.

Anyway — the project ended with a working MVP and a dataset of 14k+ products scraped from the various retailers. Just like many of my side projects, it was soon abandoned and i moved onto another (at the time i think it was a Covid19 visualisation and tracker for SA) which you can find here.

Joining Quench

Back on track — when I joined Quench, it was a small company with an even smaller dev team (six developers, to be exact).

For extra context, Quench started out as an on-demand liquor delivery service. I joined at the time when the team was getting ready to go live with their re-platforming — with still lots to do!

The new platform included the concept of Silos — one being Groceries (Woolworths), and the other two being Dis-Chem (the second-largest pharmaceutical retailer in South Africa), Liquor (Ultra Liquor), and much later, Marketplace. The same backend also powered Dis-Chem’s DeliverD offering — a mobile-friendly webview within the Dis-Chem app, built by our team. This backend was part of a multi-tenant architecture shared across both the Quench and DeliverD experiences.
The architecture was modern:

• GitLab CI/CD
• Flutter (SuperApp + PickerApp)
• Golang Microservices (Catalogue, Fulfillment, Payment, Tracking, Driver)
• Angular
• K8s (GKS)
• Managed Google PSQL database instance per microservice
• Some Ruby on Rails jobs

The infrastructure itself was ready for multi-tenant scale. This was set up by what I believe was a contractor. Looking back at the infrastructure now, it was pretty overkill — we could’ve probably gotten away and achieved massive cost savings with some minor changes, at the expense of a tiny bit of resilience, I guess (which is worth it if you’re running a cash/devops-lean startup, IMHO). The team was of an intimate size, which meant that everyone had to wear multiple hats and learn quickly on the job. This environment was ideal for a junior developer like me, eager to gain hands-on experience by building in the fast-growing e-commerce space. I often wore a bit of a devops hat — figuring out CI/CD problems and troubleshooting infra issues.

My First Task

The first challenge that landed on my desk was to implement a Woolworths price scraper (round 2 😂). With the go-live for the re-platform date fast approaching, the business/CTO could no longer wait for Woolworths to send us the SoH and price exports. Those were CSV dumps of their SoH and prices for the products. The scraper script would hit Woolworths over the internet with the SKUs we had on record, i.e. at the time:
GET - https://www.woolworths.co.za/prod/A-<sku>

After spending a few days learning Go and getting familiar with the codebase, I made my first professional commit:

Fast forward another two or three days — the scraper that I put all my sweat and tears into was abandoned, as Woolworths finally sent through the product and SoH CSV exports.
Nonetheless, it wasn’t that deep, and that first project served as a good challenge — the first of many to come:

• 🚚 Built a tracking microservice from the ground up for in-house analytics, streaming data to BigQuery
• 🛠️ Made infra changes using Terraform — from provisioning Elasticsearch to Google log alerts
• 🧺 Built a Customer Product Backup/Replacements feature
• 📅 Implemented Order Slot Scheduling (affecting both the SuperApp and PickerApp) alongside the CTO
• 🔍 Integrated Elasticsearch, to drastically improve product search across platforms

The Memories

From early coffee runs and last-minute Builders Warehouse missions to grab chairs, to setting up our office space — only to be hit with a Level 5 lockdown a few days later. Weeks pass, we return... and find out our office isn't even our office anymore (all that hardwork putting the chairs and desks to together for another team 😂)

To troubleshooting Terraform state issues on Meet/Slack calls to jumping on production bugs affecting orders and ops — these moments stick with me. They weren’t just tasks; they were high level learning experiences. A reflection of the culture set by our CTO — one that valued curiosity, ownership, and learning by doing.

Reflecting

After a few intense Black Friday seasons, working on two more e-commerce platforms, and gaining some distance and clarity — I realized I wouldn’t make many of the same technical or same questionable business decisions I've seen in the past today.

And that’s okay. That’s the point — to grow past short-term fixes and reactive scaling, and start thinking in terms of long-term resilience, observability, and simplicity. I’ve seen how brittle things get when foundational choices are rushed. These days, I think more about trade-offs: how a slightly slower path upfront can save weeks of firefighting later. How latency budgets, circuit breakers, and good metrics are not “nice to have.” How infra decisions echo into developer velocity. The tech lessons stick because I’ve lived through the consequences, those so called scars now guide better calls.

What I’m most grateful for is the environment I got to grow in. One that lived by the motto:

fail fast, fail often, but always fail forward.