Do Androids dream of 'real' sheep? - Agentic AI Explorations
Organisation
Museum of Solutions
Team
Self Initiative
Role
Agentic Designer
Introduction
What started as a curiosity-driven dive into running open-source LLMs locally soon became something more agentic—an attempt to make AI not just a tool I query, but a creative partner in my design process. In a basement corner of the Museum of Solutions, amidst spatial maps and exhibit storyboards, I asked myself a speculative question:
Can a local LLM trained on our internal docs become a collaborator—an ideator, a first drafter, a co-imaginer?
I had no background in backend systems or AI infrastructure. But with a Mac Studio (M1 Ultra, 64GB RAM) and a weekend’s worth of YouTube, I began exploring.
Setting the Stage: Local Models, Real Capabilities
I started with Ollama, a tool that makes it surprisingly seamless to host LLMs like LLaMA 3, DeepSeek, Qwen, and Phi locally using Apple Silicon’s GPU acceleration. Paired with OpenWebUI, an open-source ChatGPT-style interface, I created a self-contained sandboxed chat environment—private, responsive, and modular.
Following a methodical walkthrough, I was able to:
Install Ollama and run multiple models simultaneously
Build the UI using Node (SvelteKit + Vite) and Python backend services
Integrate these models into a usable interface with account handling, prompt storage, and system prompt configuration
It wasn’t just about running the models. It was about turning them into responsive agents—capable of contextual responses rooted in MuSo’s internal language and logic.
And the biggest benefit of all was that the files were not going online. There was no computation that was happening on the cloud. So this way of doing things was essentially safe and did not risk any online threat over data privacy leaks.
Feeding the System: Context as Culture
Once the environment was set up, I began feeding foundational MuSo documents into the LLM’s prompt memory: project briefs, space design philosophies, mission structures, even tone-of-voice documents. While this wasn’t “training” in the traditional fine-tuning sense, it was a method of contextual priming—turning the model into a MuSo-aware design partner.
I soon found I could ask the model:
Generate five narrative formats for a spatial experience about climate justice
Suggest variations on our Coral Mission for a traveling exhibit
Draft a quiz structure for a water conservation challenge
These weren’t just templated outputs—they carried the tone, cadence, and conceptual language that my team had spent years refining.
Designing with the Machine: Agentic Interaction
This was more than prompt engineering. I was exploring what Analytics Vidhya recently called agentic design patterns—where the designer creates not the object but the conditions for meaningful interaction between the user and an adaptive agent.
Rather than asking “What can I make with AI?”, I was asking:
“How can I design the system such that the AI knows how to create, with me?”
Every prompt became an invitation. Every output became a sketchpad. The model wasn’t answering; it was collaborating—scaffolding the early stages of design where ambiguity is highest and constraints are still forming.
Making Two AIs Talk: LLMs —> Stable Diffusion
To extend the experiment, I connected Stable Diffusion (via the Automatic1111 API) to my OpenWebUI setup—essentially allowing the local LLM to generate prompts for image generation. It was my first successful attempt at making two local AI systems “talk” to each other, without the internet in between.
The LLM would craft a visual prompt based on a concept or idea I typed, and the API would forward this to Stable Diffusion for rendering. While the pipeline worked and responses were generated fairly quickly, results varied. Sometimes the LLM wrote prompts that were too abstract or not SD-friendly, leading to uncanny or irrelevant visuals. But when it clicked, it felt like magic—a co-pilot typing ideas, and another sketching them into existence.
Expanded Reflective Agentic framework
Designing Agentic Personas: From Content Strategist to Visual Prompter
Instead of relying on a general-purpose assistant, I began creating role-specific agents within the OpenWebUI—each with its own persona, system prompt, and task orientation. I built one for content writing, another for prompt engineering visuals, and a third that mimicked a design researcher.
This approach gave me more control and focused creative outputs. For instance, the Content Writer agent would format project descriptions with clarity and tone, while the Visual Prompter knew how to adapt ideas into clean, Stable Diffusion-friendly prompts. The more I iterated, the more I saw how agent design itself is an act of interface design—just invisible.
Impact
1 out of the 4 Missions that are active in the Investigation Zone at the DISCOVER LAB, MuSo, has been first drafted by this Trained Model. Operating in the newly designed and operational Investigation Zone Game System (one of my projects).
The first draft of the newly launched Lost Code of Play also has been done by this model. This system also being manage by a new Vibe-coded Score & Leaderboard System created by me. All of this is part of the Summer@MuSo program.
Reflections: What Worked, What Didn’t
What worked:
Speed: Iteration was lightning-fast, allowing me to go from thought to draft to prototype rapidly
Contextual Relevance: With smart prompt scaffolding, the model mirrored internal logic impressively well
Agency: The system wasn’t passive—it became an active provocateur in ideation
What didn’t:
Memory limitations: Long, layered queries lost context across threads File reading and analysis was sometimes unreliable.
Context Irrelevance: While a lot of the times, the context training worked perfectly, many a times it didn't.
Why This Matters
Most conversations around LLMs still orbit around scale, latency, or compliance. But in my microcosm—a small museum design lab—what mattered was companionship in creativity. Not automation. Not speed. But the ability to have a partner that asked “what if?” when the post-its ran dry.
In that sense, agentic design itself is an interface between the self and the ontology of self. We know what we are doing. That itself is the beginning of the question of how long is the known, unknown and vice versa, especially when the process of thinking is already defined.
Can AI recreate Van Gogh? Yes.
But, can AI be the next Van Gogh?…that is not a Straight No anymore.
That is the direction of the world now, according to me.
Where This Might Go
None of this was flawless. The prompts failed. But this project became a personal proof-of-concept: that I can work hands-on with the latest tools in AI and agentic systems, bridge them through design intuition, and keep learning endlessly. Not bound by skill labels, not waiting for permission—just exploring. System by system. Prompt by prompt.
—
This isn’t a case study with a neat outcome. It’s a design experiment in autonomy, context, and co-authorship. It asks not just what AI can do for us—but what we might do with AI if we’re willing to share the sketchbook.