A design method for exploring and understanding vision language models
Discover “material exploration,” a new-to-you process that has been trusted for decades to design New Things
Learn how to use vision language models as a design material. Understand how to work VLMs into your designs through live, collaborative experiments with Perceptron Isaac 0.1.
- A two-hour public workshop
- Live exploration of a vision language model
- Plan tests and experiments with other professional designers
Watch my sales pitch in under two and a half minutes
Legible learning to design peers and stakeholders
For orgs that value formal, tangible learning, you’ll be issued a digital certificate for completing the public workshop.
Repeatedly vetted in the real world since the late 2000s
The design techniques in this workshop were formalized in the late 2000s, when designers and academics developed methods and practices for non-engineers to make sense of technologies.
For over a decade, I’ve used these methods to teach designers how to explore sensors and prototype smart devices, without learning to code or becoming an electrical engineer, and now I’m broadening the scope to include any technology, including the current models of generative AI.
What you’ll learn in this workshop
In this practical, hands-on training, you’ll learn:
- the basic principles of treating a vision language model as a design material,
- ways to conduct a material exploration of a vision language model,
- an example of these practices in use on a real project, and
- time and materials to practice planning similar work with other participants.
Workshop agenda
Workshop notes and level-setting (5m)
Introduction to vision language models (15m)
- Introduction to vision language models
- Survey of AI assistants with vision capabilities
Group exercises (30m)
- Map vision language models to AI use cases
- Identify existing products and technologies with similar capabilities
- Identify additional use cases for vision language models
5m break
Material exploration of VLMs (20m)
- Introduction to material exploration
- Introduction to Perceptron Isaac 0.1
- End-to-end example of constraint exhaustion and documentation
- Constraints of VLMs to exhaust
Group exercises (45m)
- Planning constraint exhaustion
- Planning documentation
- Practice with Isaac throughout
- Instructor critique throughout
Why do designers need to explore technology as a material?
In the long series of steps of bringing a highly-technical product to market, whether that’s a “smart” or “connected” device or an AI-enabled feature or app, designers with primarily 2D design experience — interaction designers, UX designers, visual designers, artists — can find themselves left out of significant parts of the process.
Sketches and storyboards, animatics and videos, UI prototypes and industrial design mood boards: all of these things are design artifacts. These artifacts can explain ideas to other designers, maybe even to laypeople, but when a complex technology is a core driver for the feature or product, these artifacts don’t help the design get built: there are too many open questions, too many things for an engineer to make assumptions and bad tradeoffs about, too many things for the business to not recognize the value of and cut in the name of optimization.
Designers in this situation need to demonstrate a deeper, empirical understanding of the underlying technology to stay in the room when engineering and industrial decisions are being made, and they need different ways to communicate their design and its evidence that are better suited for an audience that is not other designers.
Those tools are material exploration, and boundary objects.
I was surprised to see how this applies to industrial design — and humbled by the amount of considerations it takes to launch a physical product.
Takeaways
- An understanding of the principles of material exploration of vision language models
- Plans and tests from your group project
Alternative scheduling
Please contact me via email, phone, or text to discuss custom scheduling for private groups of 4+, including teams, corporate, on-site in-person, and full-day (8+ hour) trainings: , .
Prerequisites
No technical experience necessary.
Ensure your Zoom app is updated to the latest version.
Visit training.tertile.one from the computer and internet connection you will be participating from ahead of the workshop, and contact with any issues for troubleshooting steps: .
Target audience
Designers (especially those with solely 2D design experience), researchers, product managers, and other non-engineering professionals who need to design a hardware or software product where vision language models are a critical aspect, and who want to learn a framework for doing so.
A public workshop from Tertile, LLC principal Vitorio Miliano
Tertile, LLC conducts research, prototypes, and training for new works. Recent projects include workflows for local, offline, text-based audio editing; implementation of LLM precursors for literary/creative works; and design and development of custom AI tools for conference workshops.
Vitorio Miliano has over a decade of experience in product management and software development, and fifteen years in research and design. He applies interdisciplinary design and research methods to rigorously solve business and product problems; and has a track record of successful, evidence-based, outcomes-focused decisions at the program and product levels.
Prior to Tertile, Vitorio Miliano has launched emerging technology applications such as a healthcare news briefing on the Amazon Alexa platform, and a 3D environment used by NASA to visualize the International Space Station. He founded the user research practice at The Advisory Board Company; and the industry-leading developer relations program at Epic Games, makers of Fortnite. He conducted a multi-year technical and legal analysis of the risks of reverse-engineering, presented at an industry security conference; and a multi-year analysis of the professional needs of design and development communities, presented across two issues of an academic journal.
Frequently asked questions
Billing
Public workshops booked by credit card are billed immediately. Refunds are available by request until one business day before the workshop. Contact for refunds or alternative billing arrangements: , .
Materials
All original materials provided to participants are licensed by Tertile, LLC for their private, personal use, and are not to be shared or redistributed. Licenses are revoked in the event of a post-training refund or chargeback, and all materials must be destroyed.
Terms of service and privacy policy
Payment for and participation in training is governed by Tertile, LLC's terms of service (PDF) and privacy policy (PDF).