Building shop.app at scale with Remix and AI

00:00 First, I'll start with a little bit of background to give you an idea of where we came from.

00:22 And then I'll share with you my five key takeaways on how we shipped at scale within a relatively short time frame. So you'll all know Shopify has a commerce platform for millions of merchants worldwide to set up and run their online store.

00:40 What the Shopify platform will also do though is allow merchants to go to where their customers already are and sell on additional and complementary sales channels such as Instagram, Facebook, Shopify Inbox and now also Shop.

00:58 Taking a little step back in time, Shop launched in the spring of 2020 and back then its primary purpose was as a way to track the packages you purchased online. You may remember at that time there was a small worldwide pandemic that meant we all bought a lot of stuff online.

01:16 I know I certainly did. Bought everything from Toilet Roll to Pasta. And Shop at that time was perfect because it allowed you to track the packages that you bought from store to door. I wasn't with Shopify at the time but it was around that time that I first became aware

01:35 of Shop as I purchased from various Shopify merchants and spotted the same track with Shop called to action on the post checkout page. This bot will then appear. This post checkout page is seen by millions of users on a daily basis and then that is

01:50 how I became aware of Shop before joining Shopify in 2021. When I joined the Shop team, the team were already starting to think what the next logical step would then be where you would not only track the packages that you bought but you

02:09 would also go to the Shop app to purchase what you wanted in the first place. The obvious flywheel being Shop, Pay, Track, Repeat.

02:27 And for the first two years of its existence, the Shop app was a mobile native only experience and these apps were and still are built using React Native. This allowed us to leverage JavaScript and React experience that already existed within

02:44 the company, ship features to both platforms with the same cadence and of course maintain one platform, one code base instead of two. Joining Shopify and being placed on the Shop team marked a bit of a shift for me personally.

03:01 I'd only ever previously worked on the web and it was a bit of a change for me to move to a calendar based release cycle. The challenges of this are you fix a bug but it can be maybe up to five or seven days before that fix actually gets out to users.

03:19 The same with new features and the feedback loop is very long and that can be tough because it can take a while before when you push out a new feature to know whether it's doing what you think you want it to do.

03:39 So while the arc of time is long, thanks to Ben's words, the web, and when we also discovered that our users were starting their purchase journey on mobile but completing it on the web, we started to think what Shop would look like on the web also.

03:58 And aside from anything else, the web is woven through the fabric of Shopify. Toby our CEO described Shopify itself as a love letter to web devs. So in some ways, it feels somewhat inevitable that Shop would end up on the web.

04:16 And moving Shop to the web has had many advantages. The open nature of the web means that if you have a web browser, you can access Shop. We can also now allow the user to break out of the constraints of a mobile form factor to view the products they want on a much larger screen.

04:33 We can also now allow the web to work in tandem with the app. So things like your cart, your favorited, and your recently viewed now sync between the two platforms. And now we can adopt CICD practices, continuous integration, continuous deployment.

04:53 Our feedback loop is much tighter. We can almost immediately squash any bugs that we find. We can try things out, monitor their success, and walk them back within days if we need to. And due to this feedback loop now being much tighter, we can also try things out on the

05:09 web that we think might work well on the native app and backport them if successful. Additionally, the majority of users who purchase through a site on the Shopify platform are

05:24 funneled through our centralized checkout flow and will see as their first option in their express checkout, Shop Pay. And whether they know it or not, when they check out with Shop Pay, when they then come to Shop Web in the future, they're already signed in, which is very powerful as it means

05:43 that we already have their shipping address, their billing address, and their QR details. So we knew we wanted to move to the web, but what would our stack be? Shopify is traditionally a company with a big investment in Ruby on Rails, and indeed

06:00 the first version of Shop.app to market native apps was a traditional Ruby on Rails app. However, this was in the spring of 23, and Shopify had acquired a shiny new toy that we were all very excited about in the web space, and that is, of course, Remix.

06:17 Remix was the obvious choice for us as it prioritizes many things that we feel are very important, namely server-side rendering, speed, and of course, it makes sense when Remix joins Shopify to dog food our own product.

06:36 So while that timeline was progressing along, there was another one on a different track, and that was the emergence of natural language processing and large language models. The most well-known example of natural language processing at this stage is ChachiBT, and a large language model is the artificial brain that allows you to interact with artificial

06:55 intelligence using natural language. So we seen an opportunity here. We knew an opportunity to do something cool, which was twin the power of natural language processing with a huge product data set that is the Shop product catalog to see could we

07:11 surface relevant products which are offered by our merchants to those users who are searching for them. So bringing all these threads together, we knew the opportunity to do something cool, but we didn't have a huge amount of time to do it as the biggest day in the e-commerce

07:26 shopping calendar was fast approaching, that is, of course, Black Friday. So looking back on that period, these are the five insights that I believe allowed us to deliver this product in a relatively short time frame.

07:43 Sorry, I will go through them, I'll just go back a slide. Yeah, start small, reuse what already works, lean into the platform, validate your thinking, and expect the unexpected.

07:59 In terms of starting small, a smaller subset of our team had actually launched a one-page site at Shop.ai in March of 2023, and that was powered by natural language processing and ChatGPT.

08:17 And it was this initial AI landing page that kicked off the genesis of what is now Shop on Web, as we needed a product detail page, or PDP in e-commerce parlance, to complete the conversion funnel to Shop Pay.

08:32 However, at this stage, it didn't make sense to keep Shop.ai as a separate landing page, so we decided to bring everything under Shop.app, with the AI product as the powerhouse behind

08:44 Shop Web Product Discovery. So starting small with this under-the-radar approach of soft launching and then building

08:58 on that foundation brick by brick, this initial PDP, with this initial PDP, served us really well when we got to Black Friday, as we could be confident that when we scaled up, that

09:11 we had a strong converting funnel as our foundation. Reuse what already works. We had a product market fit with native apps, and we also had a lot of tooling and infrastructure

09:29 already set up to support that. So we needed to move fast, and by utilizing a mono-repo structure, we were able to take advantage of many things, such as code convention, tooling, design tokens, and deployment pipelines

09:42 that were already in place and battle-tested. So we tried not to reinvent the wheel as much as we could. And although React Native is React, I found that a lot less of the codebase could be shared

09:59 than what you initially might think, due to React Native primitives, such as view and text, which actually make it quite difficult to share UI components. But if we stick to the business logic, using a shared codebase actually gives quite a lot

10:16 of consistency between many aspects of the product, by sharing things like dependency versions, design system tokens, currency conversion, translation strings, and crucially, the API components between shop server and the LLM.

10:35 So typically, users to an e-commerce product can be split into two camps. Those who know exactly the product that they want to buy, and those that have the use case for their product, but not yet the actual product that they want to buy. And it's that latter use case, when given a large and diverse product catalog, that

10:53 AI really begins to shine. And this is because using vector embeddings, we can now provide semantic search. What that means in practical terms, is rather than explicitly needing to search for a term

11:08 like, say, barbecue, I can search for cooking outdoors in summer, and the LLM will return results to me related to barbecue, without me actually needing to specify that term. So how does it do this?

11:27 AI can be a bit daunting if you have not worked with it, but it might be more simple than what you might think. So when you train a model, what we did was take every product in the shop product catalog,

11:43 pass it through an embedding model, and turned each product into a specific numerical vector, which is just a number. Then when you search for something, the search term is processed by the same embedding model,

11:59 and that search term is also turned into a numerical vector, also just a number. So the search results that you see are those, according to the model's calculations, have numbers nearest to the numbers of your search term.

12:16 That is to say, the nearer the number of your product to the number of your search term, the more relevant that product is. This approach also allows us to perform similarity searches in much the same way.

12:34 Your original product is used as a search term, and then we stack rank all the other products based on how near the numbers of their numerical vector is.

12:50 This allows us for much more nuanced searches than just matching keywords. I also want to say that this is quite deep into the black box. At SHOP, we had teams much further upstream who dealt with this type of thing.

13:07 So if you're thinking of experimenting with AI, there's probably already products out there which will provide an API to do that, so you don't need to be thinking about things

13:21 like numerical vectors. For further refinement, we also introduced a sidebar shopping assistant. This allows users to chat using natural language, and the really cool thing is that it is context aware.

13:40 It knows the product you're currently looking at, and it also knows products you've previously viewed. We also allow the user to provide feedback on the responses, and the mechanism for doing

13:55 this is either thumbs up or thumbs down on the individual chat responses. This is very important with using an LLM, as these signals will improve the model over time, and they also implicitly signal to the user that perhaps these responses are not

14:13 infallible because they are not. When dealing with new technologies and AI in particular, I find that it was very easy to kind of over-engineer what you think the solution might be. With this example, what we had to do was keep the manual filters along the top in sync with

14:32 the chat bar at the side, and when I first thought about how we might do this, I was chatting to my team and I was saying we should track everything that the user does, and every single change we could send back to the model, and it would come back, and no, too complex. It doesn't have to be as complex as that.

14:52 Don't be this guy. Lean into the platform. For the current state, we simply use the URL as a single source of truth. Both manually changing the filter or asking the assistant will update the search terms

15:08 and will change the URL. Contacts for previous searches are stored in the browser via local storage. These are two native browser features, but twinned with the magic of AI, that become

15:25 much more than the sum of their parts. Validate your thinking. When scaling up to millions of users, we need to be able to ship changes with confidence.

15:39 We do this in shop by using experiment flags for A-B testing. This allows us to test changes when we're unsure of what the impact may be, and then

15:53 move forward with confidence. Related to experiment flags are feature flags. We use feature flags extensively in shop, and we use these right from the very start.

16:09 We guard new features in a feature flag so we can land them on main to stop long-running feature branches, and that allows us to be able to ship those changes out confident they're guarded by a flag.

16:25 We can then swap the feature flag for a staff flag when we want to do internal testing within the shop team, which increases the confidence in what we're doing. And laterally, even when a feature is live, we'll redesignate that flag as a kill switch,

16:44 which means we can turn off a feature if needs be. You'll particularly want this if you're relying on a third party, as you will be if you're using an LLM. If you've ever been woke up at page duty at three in the morning and you're in your pants,

17:00 the first thing you'll want to reach for is a feature flag. So to recap, start small.

17:10 Start small, fly below the radar as long as you can, soft launch, and build upon that. Reuse what already works. We were able to lean on existing design systems and patterns that were already in place for the native apps.

17:30 And if you can do that and not reinvent the wheel, that will make you move a lot faster. Lean into the platform. Lean into the platform where you can. Do not over-engineer things, as the more you abstract away from the platform, the more

17:48 chance that you increase the complexity and increase the chance of bugs. Update your thinking. You can't have success if you can't measure what success looks like. Use A-B testing tools so that you know the changes that you are pushing out are doing

18:06 what you expect them to do. And finally, expect the unexpected. Use feature flags, if possible, so that when your page duty goes off at 3 a.m., you have tools to reach for.

18:24 So there are my five insights which allowed us to build Shop.App at scale using Remix on a dash of AI. Thank you for your time.