App Development

Why should I use CI for a website?

17 December 2020 • 12 minutes

Written by Jörn Guy Süß

image for 'Why should I use CI for a website?'

When Codebots rebuilt codebots.com authors happily became coders. In this article, we will tell you how and why we made this change and how you might do the same!

We want better and faster.

Websites today are quite miraculous. In fractions of a second, in unfavorable circumstances (think spotty train network) with varying output devices (screen-reader, watch-size display, radiant 1040p half-square-mile living room TV) they provide information and navigation to us. Often ‘design’, ‘look-and-feel’, and ‘brand’ are words that appear quickly in a discussion about the quality of a web-site. But think of Google. What did the white box with the one button to its right have that the other ones did not? It was better: It gave us what we wanted. It was faster: It did that in almost no time. This is possible only because underneath a lot of machines perform a lot of fiddly, precise optimizations. We knew that this was true for making software using compilers, build tools, and everything that comes with it. But so far the idea that the ‘creative’ work of producing a website is at odds with precision machinery to make it still reigns in many places. It did in Codebots. No more.

Stop panel-beating your web pages.

If you see a Rolls-Royce Silver-Cloud and a Tesla Model S P100D sitting next to each other, you may be tempted to see them as similar. They both have four wheels after all. However, once you compare the numbers, you know which car is better and faster. The Tesla leaves the Rolls Royce behind in every numerically comparable category, other than price. Sure, the Silver-Cloud is pretty; each surface panel has been carefully beaten by hand, and it comes with a hotline that will pick you up when it breaks down.

You can even have the same Rolls Royce quality for your website. Carefully handcrafted HTML and CSS, bespoke systems of links and JavaScript, and a 24x7 hotline that helps when things do not work. You can have this quality if you have the money for this, but I would posit that most of us will be happy with the Tesla. Wanting the Tesla means engineering the web-design, not panel-beating your web-pages.

Works for me works for no-one.

Needless to say, putting together machinery so complex that it builds a super-website reliably is not something people relish doing. We know that static-site-generators can build sites reliably. However, let’s investigate what could go wrong with having that installed on author 1’s (call him Jack) computer. In this story Jack writes a few files, and then sends them to author 2 (say her name is Jill), or maybe they even use a version control system. Jill might have the SSG as well, or maybe she just edits blindly. Regardless, as the files get copied up to the webserver: the site breaks. Some Windows-Mac-Unix finery. Some timestamp issue. A version of something somewhere being just a bit off. Anyway. Site down. Business down. “Worked for me,” says Jack. “Can’t see it until it’s up” says Jill.

We will never know how badly this story ended, but we do know that “works for me” works for no-one. We need a place where the machine tests every change for everyone, all the time, and delivers them to the client if and only if, they are ready. This is the concept of Continuous Integration and Continuous Delivery (CI/CD).

Becoming Alice.

You can buy services that give you the experience of CI/CD for a modern website. Netlify is a prime contender in this area. However, this packaged deal is comparative to a trip to the convenience store to buy your dinner. It can become quite costly if this is your regular staple. It also restricts you if you want anything but the standard freezer meal. If your business website interacts with other business functions or has specific behavior that the range of plug-ins and extensions in a Web CI provider like Netlify cannot provide, you are building a convenience store dinner.

The alternative route is to become Alice and slide from the solid world of authoring content down the rabbit-hole to the Wonderland where the engineers live. Trust me, engineers may appear strange, but they will help you in your quest. They will help you to develop your operations (speak “DevOps”) in the way you need. They will ask questions about the order in which you do things; because some of the things you might want to do differently, but you never could, because it was too much manual labor.

Parts of the machine.

So what will a CI-solution for your website usually entail? Below is a list of requirements:

Issue capture: What you want. It is good to have a certain amount of agreement in writing before you make a change. Issues are where you will capture these agreements. You should treat issues as part of your CI machinery. Do not put your issues in meeting notes or spreadsheets, or they will not connect to what you do later.
Branching: Where you work. You want to contribute something new, so you make a copy of the original for this purpose. This copy is known as a branch and it is usually kept by a version control system. Git is the most commonly adopted one these days.
Build: The machine works and checks for you. When you have added some things to grow your branch, you store it on the version control server. This will prompt the build server that monitors the version control server to build your new version for you, and perform every automatic check you and your engineers could require. If a check fails, you will get feedback. And you can collect the outcomes over time to build a picture of what aspects need the most care.
Review App: Visibility over what you are doing. If everything is fine, your branch will immediately start a website. Yes. You heard correctly. Every changed version gets its own copy for review.
Merge Request: Is this what we wanted? When you are happy, it is time to discuss if the content can be added to the actual site proper. This is a human decision. Merge Requests are an online forum where the content is discussed.
Rinse, Repeat: What else do we want? If your website is so wildly successful that each team member can now buy an island next to a friendly software tychoon, you are finished. Otherwise, discuss your next issue.

At Codebots, it took two engineers and four team members about two months to port a library of content from an old site and set up this flow. However, this number is probably not indicative for others, as we also reworked the site completely for an 8-second to 120-millisecond speed-up and increased its quality, while still adding new content.

Foundations

What do you need to make this happen? Your human capital is the most important. You will probably need two engineers that have practice in building software with a CI-loop and are truly passionate about automating everything. If you have to replace this part of the setup with outside expertise, brace for steep markup in time and cost. Any hired hand or consultant will need considerable warm-up time before they understand your business. If you ask someone with a ‘method’ to assist you, you will likely end up with a pre-fabricated solution that may not fit.

The second component is a well-integrated CI-system that uses Docker containers with small vendor lock-in. Well-integrated means that you have a database that can take you from issues to branches to builds to reviews and so on. Without these connections, you do not know where you are. Using Docker means that you can pick from an incredibly large number of pre-prepared computer installations as environments to run your build. Without that, your choices in doing things your way are limited. Small vendor lock-in means that you can have your own or decide to walk away when the party offering the system becomes commercially problematic. You want to avoid this type of Hotel California, where “You can check out any time you like, but you can never leave”.

At the time of writing, there is only one contender that meets these criteria, which is Gitlab. It can be installed locally - which will be more Alice in Wonderland - or used in a hosted fashion. Gitlab salespeople will try to sell you all the bells and whistles, and then it quickly becomes pricey. For the purposes described here, you should very well get by with the free version. The recommendation for Gitlab is not gratuitous. I have tried hard to find alternatives. See the list at the bottom for competing approaches and how they rank with the three criteria.

Building your site on foundations

There are just three must-have parts to build a high-quality web application:

A static site generator (SSG): This is a tool that takes in a high-level description of the site in contents and design and turns it into a very large number of detail files that make up the site when a browser downloads them. Look at this list to pick a candidate. At Codebots, we use Jekyll. The SSG can optimize navigations, media formats, and search metadata, and check links, accessibility, security, and spelling.
A staging site: This is a webserver that gets filled with the files produced and then offers them under a specific address. This address is the address of the review app shown previously. Your developers will know where to source this.
One or more site checkers: These are tools that you provide the address of the staging site, and they can perform all the checks to automate for you and provide you a report. At codebots, we use Google’s Lighthouse CI which can check aspects of response time, security, and anything else that can affect your search engine optimization.

These three steps can be imagined as a set of three computers running one after the other, with a disk inserted, written, and carried to the next on each step. Of course, technological refinement on so many levels wraps this process up, but at the core, this is what happens.

But Design is not Code!

All of the sections above may have made creative staff quite queasy. It sounds like the issue already spells out what to do, and then the discussion comes later. What about all those scribbles, proposals, wireframes, prototypes that accompany the design process? In Gitlab, there is design management that is integrated with the issues. It is for keeping all the doodles and will-not-be-includes that arise while you are discussing your issue before you commit to working on it. One company called Figma has taken this one further and integrated their design tool into this stage, allowing remote staff to collaborate on designs and committing them to Gitlab issues once finished.

Are you succeeding yet?

Of course, a website can tick all technological boxes and follow all formal guidelines, if the content is not interesting, and improvements do not happen, then the website does not fulfill its function. To this end, three more facilities that can help the team to produce a well-liked and well-managed website are worth mentioning:

Lighthouse CI server is a web server that collects information from the lighthouse site checkers and displays them as trends. This allows developers to see improvements and regressions of quality over time. This helps the team to direct effort where it is needed.
Internal Value Stream Analytics is a Gitlab facility that uses data captured to compute information on the time it takes to respond to a need. This helps the team to understand how well items flow and whether there is a consistent stream of novelty. In its most minimal form, it is based on time-stamps and captures the following:
- Time to schedule an issue (by milestone or by adding it to an issue board)
- Time to first commit
- Time to create a merge request
- Time it takes GitLab CI/CD to test your code
- Time spent on code review
- Time between merging and deploying to production
External Value Stream Analytics is using Google’s analytics information to create reports that show how the website is performing in practice. This helps the team to focus content on topics that improve the status of the website. In Codebots implementation, the identifying information required is added to the website by the Jekyll SSG.

Observing these input sources and explicit feedback can help produce the issues that drive the improvement loop.

How soon is now?

It is easy to look at a desirable technology and then shy away from it, we almost did this at Codebots. It inevitably took a while to go through the steps of connecting creative staff and our engineers but we now see the benefits in our search rankings and customer experience which leaves us asking the question ‘why did we not do this earlier?’.

So ask yourself: How soon is now? Taking the plunge may bond creatives and coders in new and exciting ways. When it happens the changes will shine on your website - your companies’ beautiful face to the web.

Appendix

Overview of alternative build systems and their short-comings

Unless noted, none of these systems offer first-class integration of workflow and issue tracking.

GitHub Actions is an extension to the Github online platform.Github Actions aims to create a marketplace of premanufactured actions to build your automation. This focus on the market means that creating a Docker container action is a convoluted process. If you opt for GitHub actions, you are locked into the Github online platform and Github actions and their commercial providers.
Azure Pipelines is part of the Microsoft Azure Cloud Platform. It is highly integrated with Microsoft offerings and only stack overflow offers information on how to run a docker container in Azure pipeline?. This is major vendor lock-in.
CI/CD on Google Cloud is a highly efficient CI solution aimed at building code. While its architecture is fairly open, it is limiting your scope, because it is only able to use specific Docker containers that implement the Cloud Builder interface.
Oracle Developer Cloud Service is Oracle’s ‘me also’ version of Gitlab, based on their now-defunct version 1 cloud architecture. It does not use Docker, but Build VMs which are based on a proprietary VM system. This is a vendor lock-in on all levels, coupled with technology that is deprecated by the vendor itself. Oracle has done a good job at copying the functionality of Gitlab that was current at the time the product was designed.
IBM Cloud® Continuous Delivery is a mix of technology assembled by IBM for its cloud services. It does not allow general docker, only a limited number of build statges. It has issue tracking based on a forked Gitlab. So this is another old Gitlab clone linked to another proprietary account.