A/B Test.

Introduction

I wanted to make sure that the proof of concept could make the pages faster than a developer like myself. To do this I decided to run an A/B test where I compare the time it takes for the POC to create a page against the time it takes for me to do it manually. I thought this would be a straightforward way to see if the POC could really save time and effort.

Abstract

I started by timing how long it takes for a page using a template and three images to be generated by the POC. To get accurate results I measured the entire process so also including the time spent selecting and setting options within the POC. I repeated this process three times and took the average of those times to get a measure.

Next I timed how long it would take me to create the same kind of page manually, also using a template and three images. To make it easier on myself I generated the titles and descriptions. This setup was designed to give me the best possible chance of creating the page quickly, so keep in mind that in a real world scenario manual development would likely take much longer.

The average time for me to manually create the pages came out to be 4 minutes and 19 seconds.

The POC was able to create the pages in an average of 1 minute and 38 seconds.

Conclusion

The proof of concept is definitely faster at creating a temporary theme page, even when I am given the most optimal conditions. This shows that the POC can save time in the creation of temporary theme pages. Even though I used a template and generated content to speed up the manual creation time, the POC was still faster. Another point is that during the creation of the page in the POC, the user does not need to do any work. Whereas in the manual process, the developer would be working the full duration.