Popularity of LLMs

Large language models (LLMs), capable of analyzing extensive datasets to comprehend and generate text, have revolutionized the artificial intelligence (AI) industry.

Coding Obstacles for Personalized LLM Applications

Despite the growing demand and enthusiasm for LLMs, users often find themselves obstructed by coding obstacles when attempting to develop personalized LLM applications tailored to their specific needs and datasets.

An Intuitive LLM Application Builder for Non-Coding Users

We designed sandbox.ai, an LLM application builder that empower less-experienced users to craft customized LLM applications without extensive knowledge.

Who Are The Users?

We started by delving into social media such as Reddit, identifying both novice and expert users based on their exposure toward LLM. We decided to cater the needs of the novice users who struggle most, while supporting expert users.

We looked into the market to examine how existing products meet the needs for the novice users.

We first used a survey to: 1) help us recruit participants for the interview and contextual inquiry. 2) quickly learn about users’ background and preferences. We received 25 responses in total, and found that:

Insight

Our users have diverse backgroud and exposure to LLMs, but all show interests in low-code builder and prefer simplicity

We conducted the semi-structured interview with industry experts and contextual inquiry with novice users, ultimately aiming at enhancing AI utilization and user experience.

Identifying Core Functionalities

Based on the insights collected from user research, we brainstormed separately first, then convened to share all the ideas and settled on the following core functionalities:

Innovative Solution Sketching

Armed with the core functionalities, we leveraged various creative techniques such as moodboard, metaphor, and SCAMPER, to bring up innovative solutions, and translate them into sketches.

From the plethora of sketches, we meticulously narrowed down to two initial variations that best aligned with our design vision and user needs.

We conducted feedback session with 4 novice users and our industry contact from Databricks. Key findings include:

01 Both variations were deemed clear

• All novice users gained a good understanding of the LLM application and how to build it after a brief explanation.

02 Both variations have pros & cons

• Variation 1 was perceived as approachable but with difficulties in visualizing the overall flow due to its step-by-step structure.
• Variation 2 initially caused confusion due to its simplicity, but users expressed a preference for it once familiarized, especially those with basic knowledge.

Merging Variations: Basic & Advanced Modes

Based on user feedback, we integrated both variations into a dual-mode design. This refinement includes:

• Basic Mode: Designed to minimize the learning curve, catering specifically to new users by providing a straightforward navigation experience.
• Advanced Mode: Offering a more open layout, empowering users with relevant background or familiarity with the platform to explore freely and make the most of it.

With the introduction of Basic and Advanced modes, we have finalized the wireframes that encapsulate these design decisions.

We held another feedback session with 4 different novice users and our industry contact. We made iterations based on their feedback:

#1 Progress Indication

#2 Mode Switch

#3 Education Page

Goal: To assess

Participants:

Process:

2. Survey: Invite both users to fill out a post-task survey.
3. Analysis: Code the transcript, categorize codes, and analyze data.

Here are the data visualization of the surveys:

From the result of the evaluation sessions, we found that:

• Intuitive Interface for Novices: The interface intuitively serves novice users to create LLM applications, especially the basic mode.
• User Needs Addressed: The final prototype effectively resolves user pain points, particularly in data upload capabilities.
• Accessibility Standards Met: The prototype meets accessibility standards across multiple domains, with room for auditory and speech enhancements.
• Seamless Mode Transition: Users enjoyed the smooth transition between modes, enhancing their progression and platform comfort.

While there have been successes, our evaluation also pinpointed improvement areas:

• Clarifying Progress Tracking: Users occasionally struggle to ascertain progress status and the progress saving effectiveness , indicating a need for clearer feedback.
• Simplifying Technical Jargon: Some technical terminology still presents a barrier to entry for users, requiring further simplification to enhance accessibility.
• Enhancing Inspiration Function: While the inspiration section proves useful, it lacks engagement; efforts are needed to make it more captivating for users.

Designing with Accessibility in Mind

We've embraced designing with accessibility foremost to guarantee usability for all from the start. Continuously verify compliance with accessibility standards, such as color contrast and keyboard navigation, and regularly solicit user feedback to promptly identify and address any clarity concerns, ensuring a seamless experience for diverse users.

Navigating Conflicting User Feedback

During the design process, we noted user feedback often presents conflicting viewpoints. As designers, we actively sought diverse perspectives to gain a more comprehensive understanding of user needs. With this awareness, we strive to strike a balance between simplicity for ease of use and the necessary complexity to cater to diverse users.