Category
Document
Reading Guide.
The Assignment
Main Question
How can generative AI improve the efficiency of creating temporary theme/promotional pages, so that clients can streamline its workflow, reduce manual labor, and quickly deploy temporary content?
The Company
iO Digital is a company that actively breaks down barriers between the agency, technology and consulting sectors. It operates at the crossroad of strategy, marketing, innovation and digital. By using its expertise in these areas iO positions itself to effectively address challenges for clients.
Problem/Opportunity
The main problem iO and its clients face is the time consuming task of manually creating temporary "theme pages" for promotions or events. These theme pages are needed periodically, and the process of making them from scratch is a bottleneck. Because iO’s clients have digital platforms with content lasting for years, the need to create temporary pages for specific promotions is a problem.
The opportunity here is to develop a tool that uses generative AI to make the creation of these temporary theme pages more efficient. Instead of building them from scratch every time, the tool should generate these pages. This would save a lot of time and effort and allows iO's team and its clients to focus on other important tasks. This tool would greatly improve their workflow and would also make it easier to handle multiple promotions or events at the same time without being held back.
By using generative AI the tool could turn this repetitive task into a more easy and straightforward process. This would both benefit iO and improve the overall experience for their clients who rely on having engaging digital content.
Goals
- Research and analyze existing AI frameworks and tools: The first step in this project is doing thorough research on the different AI frameworks and tools that are available. This means looking into the latest technologies, reading about recent developments, and understanding how these tools work. The goal here is to create a list of options and noting their features, strengths, and weaknesses. This will be done by doing a lot of reading, testing, and comparing to make sure that I have a good understanding of what's out there and how these tools can be helpful for the project.
- Assess the quality, adjustability, and compliance to guidelines of the identified tools: Once I have created a list of potential tools the next step is to evaluate each one. I need to determine how well they perform in terms of quality, which includes looking at the quality of the code they generate and how well they comply to best practices. Adjustability is also important, I need to see how easy it is to customize the output to meet specific needs.
- Select a set of tools to be tested in the proof of concept phase: After assessing the different tools the next goal is to narrow down the list to the best few that show the most potential. These tools will be put through more testing in the proof of concept phase. The selection will be done by weighing the pros and cons of each tool and considering factors like ease of use, flexibility, and performance. The goal is to choose tools that both meet the requirements and also have the potential to deliver the best results.
- Integrate content retrieval from API-based data sources, specifically StoryBlok (headless CMS) and Magento (webshop platform): Another important goal is to integrate content from external sources into the generated theme page. Specifically I will focus on retrieving data from StoryBlok, a headless content management system (CMS), and Magento, a popular e-commerce platform.
- Generate and deploy a theme page on AWS: In this phase I will use the tool to create an AI generated theme page and deploy it on Amazon Web Services (AWS). This will be done by setting up the needed infrastructure on AWS, generating the theme page using the tool, and deploying the generated page so that it is live and accessible by anyone.
- Test POC pages and the entire pipeline against WCAG guidelines: Finally I will test the proof of concept pages and the entire AI pipeline against the Web Content Accessibility Guidelines (WCAG) to make sure they are accessible to all users.
The Process
Planning
Before I started the project, I made a detailed time plan to make sure I could meet all the goals during my internship. This plan was divided into five parts of sprints. I used a method called the double diamond to help guide the project. The double diamond breaks the project down into four phases: Discover, Define, Develop, and Deliver.
In the Discover phase, I focused on doing all the needed research for the project. This meant gathering information, studying related topics, and properly understanding what needed to be done. Next in the Define phase, I focused on coming up with ideas and concepts. I also started experimentation for the different features that came out of the ideations and research.
The third phase was Develop. During this time I worked on creating a more put together version of the proof of concept using the prototypes and research done previously. Finally in the Deliver phase, I tested the project thoroughly and made any needed changes to it. This phase was all about making sure the project was complete and ready.
Summary
Before starting the first sprint, I had a few weeks to properly set up the project. During this time I created my Project Plan, which holds all the steps and goals for the project. I also identified and gathered the stakeholders, which are the people involved in the project's progress and outcome.
With the stakeholders I arranged weekly standup meetings. These meetings were around 15 to 30 minutes and took place every week. The goal of this was to keep the stakeholders updated on the project's progress and to get their feedback. This way everyone involved stayed informed, and I could make improvements based on their suggestions throughout the project.
Discover
During the first sprint I wanted to build a strong foundation for the project. To do this I gathering as much information as possible about AI and related tools and services, as well as understanding the end users and stakeholders.
Because my project heavily relies on AI, I began by researching the different AI tools and services available and testing their capabilities. This helped me understand which tools would be the most useful for the project.
I also looked into AI prompting to better understand how to generate pages effectively. This research was important as it helped me understand how to use AI to create content. Lastly I researched branding consistency, which would be important later during the concepting and development phase when creating the proof of concept to help to integrate branding styles.
To make sure I was addressing the end user needs, I also interviewed end users. These interviews helped me understand their needs and challenges, specifically of the creation of temporary theme pages. By understanding their struggles I could make the project better fit their needs and make it effective.
Define
In this sprint I had gathered all the information I needed to start making decisions and narrowing down my options. My first step was to properly visualize the end user. To do this I created two personas based on the needs and insights I gathered from the interviews with the end users. These personas helped me understand who I was creating the tool for and what their specific needs were.
Next I took the ideas from the small brainstorming sessions that happened during the weekly standups and combined them with the needs of the end users. From this I created a list of functionalities. This list holds the features that the proof of concept would need to have.
Besides the functionalities list, I also created user stories. User stories are short descriptions of the features from the perspective of the users involved. They helped me make sure that each feature was in line with the users needs and gave a better understanding of how each functionality would be beneficial in real world scenarios. These user stories were important in keeping the project user focused and making sure that the final product would be practical and beneficial for the end users.
To create a good overview of all these requirements of the proof of concept, I created a POC requirements document, which holds all the requirements on different aspects of the POC.
With the functionalities in place, I wanted to start experimenting to make it easier to realize the most important features during the development stage. To do this I began looking into how to use OpenAI with Azure.
I also started working on creating prototypes for HTML generation and the HTML viewer. By developing these prototypes early on I could test and refine the features and making sure they worked as needed. This experimentation helped me find any potential issues early on and allowed me to make improvements before moving into full development. This way I could make sure that the most important features would be well developed and functional when the time came to build the final proof of concept.
Using the functionalities list and the user stories, I then started creating the first designs for the interface of the tool. These first designs were created by keeping the needed features in mind and were meant to address the needs and preferences of the end users. This step was important in moving the project from the concepting phase into actual development.
Develop
Starting this phase, I first had a return moment. For this event, I created and held a pitch for my assessors and fellow students about the progress of the project. The goal of this event was to get feedback to help improve the project and to reflect on the work I had done so far. I received good feedback with a few points to keep in mind. The two most important points were to setup Success Criteria to help know when the project has succeeded, and do an Ethical Analysis to get a better understanding of the dangers and problems of AI. The feedback I received during this event as well as the feedback from my company mentor helped me do a 360 midterm review.
With the designs and list of functionalities in place, I was almost ready to start further development. However before doing that I needed to understand the technical requirements like coding standards, required services, and programming languages for the proof of concept.
To get a good understanding of these technical aspects, I held two expert interviews. One was with the chapter lead, who gave me important information on the overall technical direction and standards with a focus on the backend. The other interview was with a frontend developer, who shared information on the frontend aspects of implementing the functionalities, like the best framework, libraries and tools to use as well as the coding standards and guidelines that I needed to follow.
These expert interviews helped me get the needed technical information to make sure that the development phase would go smoothly, and so it would be easier for them to carry on the development of the project once my internship was complete. This step was important in making sure that all technical requirements were met and that the project was better set up for success.
I could now go ahead with the further development of the proof of concept. With the base created during the define stage, the next logical step was to develop the page styling generation. This step was important because it would allow the pages to be styled which made them look more modern and professional.
Next, images needed to be generated and included. I expanded the tool so that users could let the AI not only produce the text of the page but also generate the images. This feature made the pages more lively and the images helped convey the theme of the pages. From the expert interview I did with the chapter lead, it was also mentioned that images should be hosted on AWS and thus I made an integration with S3 and CloudFront. A problem that came up with this image generation feature was that it took longer the more images the user wanted to generate. I therefor implemented concurrent image generation which significantly sped up the image generation process if a user wanted more than one image.
There was also the need for having a tone of voice for the content. I looked into the possibility of tone of voice cloning to personalize the content more effectively. Unfortunately the cloning was not effective and didn’t produce the results I wanted. Because of this I slightly changed this feature into a predefined list of tones of voice that the user could pick from. This way users could still choose a tone that best suited their needs.
Because these and the yet to develop features did not yet have a proper place inside the interface of the tool, I updated the UI to accommodate for them. This way the features could be used properly and more could be added later.
With the new UI I could now add a feature allowing users to generate pages using templates, which was another need from the end users. By implementing this feature users could create pages based on predesigned templates, which allows for easy branding consistency.
Another need from the end users and stakeholders was the ability to make changes to pages and images. To do this I created two prototypes: page section regeneration and image regeneration. These two features would allow users to quickly make changes to a generated page.
With all these new features added to the UI, I noticed that the current layout would not work if more features were to be implemented. During a meeting with my assessor, he told me about the interface of ChatGPT for plus members. This layout would fit a lot more options and would be a strong base as ChatGPT is already used by a lot of people. Discussing this with my stakeholders, they agreed to changing the layout to make the design more future proof.
Something else that came up during the interview with the chapter lead, was the recommendation of keeping the tool as independent and flexible as possible. Because different companies use different styling methods/frameworks, I added to ability to let the user decide on how they want to style the page.
With the main core features added, I needed to implement the integrations that the end users needed. The two most important ones were a Storyblok integration and a Magento integration. By integrating Storyblok, users can embed links to their other content inside the generated page, which is great for SEO. The Magento integration allows users to connect their webshop and add their products to the generated page which is useful for promotional events.
Deliver
With the develop phase complete, I now had a working proof of concept that was ready to be tested. The first test I did was a WCAG test to see how well the generated HTML complied with WCAG standards.
I then moved the first round of user tests which was done with two clients of iO who I also had the first interviews with: Jordi ter Woord from IGM and Bart Verdouw from Prominent. Both had never seen the tool before which was great for the user tests. From these user tests I got very valuable feedback for both improving the user experience as well as functionality. The main priority for now was to improve the user experience, and therefor used the insights from the user tests and made UI Iterations. I also implemented a yet missing but required feature: page publishing along with page title editing.
Having made the user experience improvements, I wanted to further validate the design and workflow of the proof of concept. I planned and held a co-reflection with a senior UX Designer. The UX designer was very positive and only had minor improvement to make the design look more interesting.
Now with the somewhat validated design, I did a final round of user tests to fully validate and see if the improvements made were actually effective. I held two separate user tests with marketeers from iO: one with Anouk Wendling and one with Wieke van Ditshuizen. Both had never seen or used the proof of concept which was good for the final test. Both user tests went super smooth with no confusion on how to carry out the scenario. This showed that the proof of concept was now truly easy and straightforward in use.
I could now test if and how much faster the tool is in creating temporary theme pages. I tested this with an A/B Test. The test showed that the even in the most optimal conditions for a developer, the tool is still faster in creating a theme page.
Lastly to finalize the project, I deployed the final POC, and created an advice report containing all the insights and recommendations to help the further development of the project after my internship.
Research Questions
I will show all the questions that were setup for this project and explain what steps I took to to solve the problems.
Main Question
How can generative AI improve the efficiency of creating temporary theme/promotional pages, so that clients can streamline its workflow, reduce manual labor, and quickly deploy temporary content?
To fully answer the main question I came up with several sub questions. Each of these sub questions helped me focus on different parts of the project. This way I could look at the whole problem but in detail.
Sub Questions
What are the marketing team’s (end-users) requirements so that the POC can be effective?
In the first phase (discover), I did interviews with the end users to get a good insight into their struggles and needs. The information I gathered from these interviews were defined and made visual in the second phase (define) by creating two personas and user stories. Based on these two deliverables, I also created a list of functionalities and the first designs used to develop the proof of concept. To have a complete overview of all the requirements I also made a POC requirements document.
In the third phase (develop), I created success criteria using the research and documents made in the previous phases. Lastly, I did four rounds of user tests (User Tests, Final User Tests) to validate and iterate the design and proof of concept, in which I got even more insights into any new needs of the user.
How can the efficiency be measured so that improvement can be validated?
During the end user interviews in the first phase (discover), I asked the end users about the key indicators that they would use to measure the effectiveness/efficiency of the proof of concept. In the third phase (develop) I also setup success criteria based on my research to help with the validation of the proof of concept as whole. Lastly, in the fourth phase (deliver) I did A/B Testing to see how effective the proof of concept is when comparing it to the manual development of a temporary theme page.
What are the requirements to make the POC usable and implementable?
I started in the first phase (discover) by getting the needs of the end users by doing end user interviews. After this, in the third phase (develop), I did two expert interviews, one interview with the chapter lead and one interview with a frontend developer. With these expert interviews, I got insight into the usability and implementability requirements of the frontend, backend and output of the POC. These too were included in the success criteria and the POC requirements document.
What are the content quality requirements for the POC pages and how can the POC adhere to it?
In the end user interviews during the first phase (discover), as wel as the expert interviews with the chapter lead and frontend developer during the second phase (develop), I got insights into the needs of the output of the POC. During the fourth phase (deliver), I got even more insights into this during the user tests. Another requirement was that the pages follow WCAG guidelines, and to validate it I did a WCAG test.
What AI tools can be used for generating visual elements and how can they be used in the POC?
In the first phase (discover), I started by doing research into the different AI tools and services out there. Using this research I concluded to a set of tools and services that were going to be used for the proof of concept. Before the development however, I did more research that was needed before starting with the prototypes: research into AI prompting. Using the research I started with creating the first designs as well as the prototypes. The proof of concept was made out of a bunch of different prototypes with each individual one also being a research for a specific functionality. Later in the fourth phase (deliver), I did user tests, a co-reflection with a UX designer, and final user tests. I used this to iterate on the proof of concept. Lastly, I made an advice report to help further improve the POC after my internship.
How can we make sure that the branding/feel stays consistent throughout the AI generated pages?
In the first phase (discover), I did testing of the theme understanding of different AI models during my AI tools and services research. I also did research into branding consistency which concluded to possible solutions that could be integrated into the proof of concept. Some of these solutions were used to develop prototypes related to branding consistency during the fourth phase (develop). In particular prototypes: Page Styling, Templates, Tone of Voice, Styling Frameworks, Storyblok Integration and Magento Integration.
Proof of Learning Outcomes
Learning Outcome 1: Professional Duties (You carry out the professional duties on a bachelor level resulting in professional products in line with the IT-area you are working in.)
During my internship project, I created professional products on all the different IT Layers within the domain User Interaction. Because I was responsible for everything in this project from research, design, development, to testing, I had a lot of professional duties. Below I shown how each product fits into an IT Layer within User Interaction.
Analysis
Research Tools & Services, Interviews End-Users, Expert Interview Chapter Lead, Expert Interview Developer, Personas, User Stories, POC requirements, Research AI Prompting, Research Branding Consistency, Pitch, Midterm Review, Success Criteria, Ethical Analysis, Brainstorm Sessions, Functionalities List, WCAG Test, A/B Test, User Tests, UX Designer Co-Reflection, Final User Tests
Design
Initial Interface Designs, Iteration Interface Designs, Updating the UI, Changing the Layout, UI Iterations
Realize
Using OpenAI with Azure, HTML Generation, HTML Viewer, Page Styling Generation, Image Generation, Tone of Voice Cloning, Updating the UI, Integrating S3 and Cloudfront, Concurrent Image Generation, Page Generation with Templates, Page Section Regeneration, Changing the Layout, Image Regeneration, Styling Frameworks, Storyblok Integration, Magento Integration, Page Publishing, Page Title, Final POC, POC Deployment, UI Iterations
Advise
Advice Report, Stakeholder Meetings
Manage & Control
Project Plan, Reading Guide, Stakeholder Meetings, Git Version Control
Learning Outcome 2: Situation-Orientation (You apply your previously acquired knowledge and skills in a new and authentic context to deliver relevant and valuable results for the project and company.)
The knowledge I gained during my previous semesters is what greatly helped the success of this project. Starting off, because each semester was a new project, I learned how to properly set up a project. In my internship, I started by gathering the right stakeholders, planning weekly standups with them and creating a project plan. For the project I made use of the double diamond, which is something I learned during all my semesters. I looked at how each research question could be triangulated using different CMD methods. I also made a time plan where I had to estimate how long each sprint would take. I was able to pretty accurately make these estimates because of my previous experience with projects.
I then started doing the research needed for the project, I created several different research deliverables for different aspects of the project. I also did multiple interviews with end users and experts to get the proper requirements of the project. With my research I created concepts and designs for the proof of concept. I also used it to create small prototypes of individual functionalities. Using these prototypes I was able to create, test, and iterate a full proof of concept.
Because of my previous knowledge, the project ran smoothly from the start, and I was able to create a researched, iterated, and validated final product that I am very proud of.
Learning Outcome 3: Future-Oriented Organisation (You explore the organisational context of your project, make business, sustainable and ethical considerations and manage all aspects of the execution of the project.)
Before I started the project, I made sure to gather the right people/stakeholders for the project. This helped to keep the project on the right track by keeping them updated and receiving feedback. During all my meetings throughout my internship, I’ve made notes in order to save valuable information for the project.
As part of my project plan, I also looked at the company, its clients and how this project can be beneficial for both of them. I also interviewed some of their clients as part of the research which helped me also better understand who iO works with.
Because my project relies very much on AI, it was important to also look at the ethical aspects of AI. I did an ethical analysis on the AI models used within the project. This helps both me and my internship company better understand the potential ethical dangers of AI.
By really understanding the project from the start, updating stakeholder, and keeping up with new information during my internship, I was able to build a product that can benefit both the company as its clients.
Learning Outcome 4: Investigative Problem Solving (You take a critical look at your project from different perspectives, identify problems, find an effective approach for solving them and arrive at appropriate solutions.)
Before doing any work, I created a project plan. Here I looked at the problem and created ways to properly research it. I started with a main research question and split them up into multiple sub-questions. I then linked CMD methods to each sub questions to look at it from different perspectives (triangulation).
This allowed me to do very specific research for example into the different AI tools and services out there. Based on the research I could then start defining aspects of my project like end user needs through the creation of personas.
Each deliverable I created throughout my internship had a specific goal that helped to answer the main research questions. The combination of all the deliverables helped to create a final product.
Learning Outcome 5: Personal Leadership (You show an entrepreneurial attitude in both your project and personal development, you pay attention to your own learning ability and keep in mind what kind of IT professional and/or what type of positions you aspire to.)
During my internship, I fully took the lead of the project. I kept stakeholders up to date by doing weekly standups, arranged interviews and user tests, and asked for feedback when needed. I also paid attention to my own (personal) learning goals, for example to further improve my user experience skills, I did a co-reflection with a senior UX designer.
I’ve also learned new skills like working with Microsoft Azure and AWS. Both are used in almost every tech company and it’s really valuable to know how they work. Another thing I learned a lot more about is AI, which I believe will become very big in our day to day lives. I did thorough research on the AI tools out there and what they are capable of, and I did an ethical analysis to find more about the potential dangers of AI.
I feel proud of the way I worked on this project because I managed to plan and balance my tasks well.
Learning Outcome 6: Targeted Interaction (You determine which partners play a role in your project, collaborate constructively with them and communicate appropriately to achieve the desired impact.)
For my project, I identified and gathered stakeholders for different aspects of the project like technology, marketing, and development. These stakeholders allowed me to keep the project progressing by keeping them up to date and providing me with feedback.
Together with my stakeholders I also identified the right target audience, and made sure to get insight into their struggles and needs by doing interviews. This also allowed me to do user tests and find potential issues or suggestions from the target audience. This made sure the project is tailored exactly to the people who will also use it.
This targeted way of working helped the project solve problems specifically for the end users.
Proof of Personal Learning Outcomes
Personal Learning Outcome 1: Further improve my user experience skills
Throughout my internship I did a variety of tasks that significantly helped me improve my user experience skills. As I was responsible for almost every aspect within the project, I had to do my own user research, design, and testing. These three aspects were really important in improving my UX skills.
For user research I learned how to gather data from users effectively, using interviews, and usability testing. I looked into understanding user needs and struggles, which helped me create more user centered designs. Designing involved using Figma to create wireframes and to create prototypes. I also spent time iterating on designs based on the user feedback which was also very learnful.
Testing was another important area where I grew. I did usability tests to see how real users interacted with the POC. This experience allowed me to spot usability issues and improve the overall user experience.
Lastly, I did a co-reflection with a senior UX designer within iO. By looking together at the project, he not only gave me feedback but also explained the "why" behind his suggestions. This helped me better understand the reasoning behind certain design choices and think more critically about my own decisions. His insights helped me think in similar ways and improve my approach to UX design.
Personal Learning Outcome 2: Further developing my personal vision
Another important learning outcome was to develop my personal vision, specifically around artificial intelligence. In my previous semester Creative Technology, I had already created a personal vision but I wanted to go even further during my internship.
During my internship I created an ethical analysis that helped me see and show the potential ethical dangers of AI. In this analysis I looked into different case studies to understand the possible bad sides of AI in society. It made me more aware of the ethical considerations that need to be taken into account when developing and publishing something using AI.
Because my project involved a lot of AI, I did thorough research into its capabilities and limitations. This research not only expanded my knowledge but also gave me a better view of what AI can do on both the good and the bad.
By working a lot with AI I got a close look at how they can be used to solve complex problems but also how they can be misused. This experience was important in developing my personal vision and understanding the possible impact of AI on society.
Personal Learning Outcome 3: Get more experienced working with emerging technologies
Another important learning outcome for me was to get more experience with new and emerging technologies. Technology is always changing/improving and keeping up to date with new developments is super important. The most upcoming technology currently is definitely AI, and my project was mainly focused on this area.
Throughout the project I made a lot of use of AI, which had me working with services and tools that I had never really used before. These were platforms like Microsoft Azure and OpenAI. Using Microsoft Azure I learned how to deploy AI models and manage cloud resources. It was a bit confusing and challenging at first, but I became more and more comfortable using the platform.
With OpenAI I looked at their SDK and tools to integrate AI into my project. This experience was important as it gave me practical skills on how AI can be used in real world scenarios. I learned how to instruct models, finetune these instructions, and implement them in applications.
Reflection
Midterm Review
For the midterm review, my company mentor graded my performance on all the different learning outcomes. He was really positive of the way I work and the products I created. During the midterm review, I received feedback from my assessors and fellow students. I used to this feedback to also do a self evaluation on the learning outcomes. This created a 360 feedback that I could then use to improve both myself and the project.
Overall Reflection
Finally, I want to reflect on my internship. Since the first day I started at iO, I’ve always really enjoyed working there. The people at iO are truly lovely and helpful people. People there were always willing to give a helping hand, for example doing user tests or a co-reflection. The people involved in my project have also meant a lot for this project, and have truly helped in making it a success. So a big thank you to my stakeholders and the people of iO.
Of course, this internship did also come with its challenges. One of the main challenges I faced was integrating the different tools and services as this was quite new to me. However, after familiarizing myself with them more, I was able to successfully carry out the integrations. Another challenge was making sure the branding consistency of AI generated pages. I researched branding consistency and implemented solutions like page styling and templates to maintain a consistent look and feel. However, because AI is very unpredictable it was hard to convey a certain brand without giving the AI an helping hand like templates. The same goes for tone of voice cloning to personalize content, AI in its current state is just not capable enough of achieving this. However, these challenges were also lessons to me that helped me improve my skills and understanding of AI.
This project greatly contributed to my personal and professional growth. I improved my user experience skills by doing user research, creating designs, and user testing the POC. I also developed a better understanding of AI technologies and learned how to deploy and manage AI models using platforms like Microsoft Azure and OpenAI. Lastly, I gained experience in project management, from planning and stakeholder meetings to executing and delivering a final product.
My internship at iO was a super valuable and I learned a lot of new things. Working on this project helped me gain practical skills and knowledge that will be really beneficial in my future career. I am proud of the final proof of concept and the positive feedback it received from users and stakeholders.