The best things I build with AI start without it.

I know, that sounds like a contradiction, especially coming from someone who builds AI tools for a living. But I keep finding that the quality of what I make with AI depends on the quality of the thinking I do before I touch it.

The hard part is never the building. The hard part is knowing what to build and why. And that part, I’ve learned, goes better without assistance.

Why this is true

There’s a recent study from MIT where researchers put EEG caps on 54 people and had them write essays in three groups: brain-only, with a search engine, and with an LLM.

The brain-only group showed the strongest neural connectivity. Widest networks, most engagement, highest memory recall. They could quote from their own essays afterward. They reported the highest satisfaction and the strongest sense of ownership over what they wrote.

The LLM group scored fine on quality. Both AI and human judges gave decent marks. But the writers themselves couldn’t recall essays they’d written minutes earlier (see Figure 11). The thinking was less deep and diverse. The perceived ownership was low. When the researchers swapped groups in a later session, people who went from LLM-assisted to brain-only showed their neural connectivity jump back up. Unassisted thinking re-engaged their brains.

The takeaway isn’t “don’t use AI.”

It’s that thinking and executing are different cognitive modes, and AI helps with one while quietly undermining the other. My note in the margin when I read this: “It’s about engagement. Not disengagement.”

How I practice this

I do most of my serious thinking on a reMarkable tablet. No notifications, no browser tabs, no Slack. Just a stylus and a screen that looks like paper. When I’m working through a hard problem, whether it’s strategy for Prototyper or architecture decisions or just figuring out what I actually believe about a topic, I write it by hand. It forces me to slow down. I can’t copy-paste my way through a thought.

That’s the “think without AI” part. And it works.

The problem was the gap between thinking and building.

My notes would be stuck on the tablet. Disconnected from everything else (Claude Code, Prototyper, Codex). I’d have ten pages of handwritten strategy or design files and no way to say “Claude, read what I wrote last Tuesday and help me turn it into a plan.” Sometimes I’d take photos of pages and paste them into Claude, which worked but felt ridiculous.

The thinking was good. The bridge to action was missing. So I built one.

What I built (and how)

I built an MCP server that connects Claude directly to my reMarkable library. It took about a weekend. The build itself is a good example of what “build with AI” looks like in practice, including the parts that are annoying and tedious.

Saturday evening, 8pm: first working version in about an hour. Cloud API client, notebook and PDF extraction, full-text search in SQLite, multi-layer caching, page rendering. It could pull any document from my reMarkable cloud and render pages as images Claude could see.

The first version had three OCR backends: Tesseract, Google Cloud Vision, and MCP sampling. I stripped out two within the hour. MCP sampling means Claude reads your handwriting using its own vision. No API keys, no external services, no setup. Removing complexity made everything better.

I tried to stick with that through the project.

Come Sunday morning I realized the tool interface was wrong. It worked, but it was designed for a human browsing an API, not for an AI navigating a library. Claude kept hitting pagination walls and had no way to tell handwritten notebooks from typed PDFs.

So I rebuilt the interface from Claude’s perspective. This is the “build with AI” principle applied to design: if the AI is your user, design for how it works. For example, I implemented a multi-page read so it doesn’t request pages one at a time. I built a grep tool that searches within a document and jumps to the matching page. Auto-OCR that detects handwriting without being asked.

Sunday evening: the setup problem. This was actually quite tricky.

A great MCP server that nobody can install is useless. reMarkable’s auth requires a tricky one-time code from their website. The MCP config is a JSON file in a specific location. The dependencies include a C library for converting SVGs to PNGs that most people don’t have.

They are tiny things, but they all add up to built enough friction so that it becomes hard for non-technical people to set this up.

So I decided to try building a one-line installer:

curl -fsSL https://thijsverreck.com/setup.sh | sh

This was quite new for me. The trickiest thing that I found is making sure that it doesn’t just work on your own system. But I ended up with a neat results.

Monday: the last mile. Claude Desktop has a tiny quirk which is a limited PATH and couldn’t find uvx, so the setup script resolves the full binary path at install time. The config writer merges into existing config instead of overwriting it, because blowing away someone’s other MCP servers would be hostile.

I ended up with a result that I was quite happy with.

One design decision I want to highlight, because it ties back to the principle: the entire server is read-only. Every tool is idempotent.

Claude can read my notes, search them, OCR them, but it cannot edit or delete anything on my tablet. The tablet is where I think. I don’t want AI anywhere near that process. The MCP server is the bridge from thinking to building, and it’s a one-way bridge on purpose.

The principle, fully earned

Think without AI. Build with AI.

I thought about this problem on my reMarkable, without assistance, until I understood what I actually needed. Then I built the solution in a weekend with AI, fast and messy and real. The thinking was slow and careful. The building was fast and aggressive. Different modes, different tools.

This is what “build with AI” means for me. Not “let AI do everything.” Not “avoid AI for purity.” You use your brain for the parts that need your brain. Use AI for the parts where leverage matters more than depth. And be honest about which is which, because the EEG study suggests we’re not naturally good at telling the difference.

I’ve been building Prototyper (an AI engineering tool) and working with AI coding assistants for over 2 years now. Claude Code, Droids, Cursor, Codex, I’ve tried them all. And I kept running into the same issues with each of them.

The tools and models are good. Surprisingly good. But my workflow was a mess.

The loop I couldn’t escape

Here’s what kept happening: I’d start a session, give Claude Code a bunch of context about my project, work on a feature for an hour, then close the laptop. Next day, new session. Claude has no idea what we did yesterday. So I paste in the same files. Explain the architecture again. Describe where we left off.

By the third day I’d lost track of what was actually done versus what I’d only talked about doing.

It got (much) worse when I started running multiple agents from different providers. One agent working on the backend, another on frontend. I’d juggle instruction files for claude, other agents, Cursor and so on. Of course, Claude Code couldn’t access Cursor plans, and Cursor couldn’t access Claude’s task list. They’d make conflicting decisions. One would refactor a function the other was depending on. Nobody told anybody anything because there was no way to tell anybody anything.

I spent more time managing context than writing code.

Why “better prompts” didn’t fix it

For months, I thought the answer was prompt engineering. I did a lot of research and I built elaborate system prompts. All why trying to get them to work with different models and different tools.

None of it addressed the core problem. Prompt engineering is about writing effective instructions. My problem was different: I didn’t have control over what context the model was seeing in the first place.

There’s a term floating around now called “context engineering” that describes this better. It’s about deciding what information goes into the context window, when it goes in, and how it’s structured. Anthropic has a good write-up on it. The Manus team wrote about their lessons too. The core insight is that as context length grows, models get worse at recalling information buried in the middle. Transformers compute relationships between every pair of tokens, so more context means more noise competing for attention.

Many of the recent innovations such as MCP can actually make this worse, because every MCP that you make available to Claude Code (or another agent) actually reduces your available context. And can significantly decrease the amount of tasks that you can get done before you’ll run into problems.

Reframing the problem

An engineer friend said something that reframed the problem for me: “You’re treating these things like magic oracles instead of junior developers.”

Junior developers need great onboarding docs. They need to know where to find things. They need a way to ask questions when they’re stuck. They need task descriptions that don’t assume context they don’t have.

I wasn’t giving the AI any of that beyond scattered markdown files, and agent instruction documents. So I spent some time coming up with a framework to solve this issue.

And I’m happy to say that after using this for about a month, my “agentic” engineering productivity has increased massively.

How I solved it: the three folder solution

I decided that I wanted to give my agents access to three things: a shared context (where I can place screenshots, design files, etc), a centralised planning system and most importantly a way to communicate with each other. I solved that in this way:

`context/` is where I put everything an agent needs for the current task. Not the whole codebase. Just the relevant parts: the API spec for the endpoint I’m building, the database schema, the design mockup. Before an engineering session, I drag files in. The agent reads them. Now we’re looking at the same thing.

This is basically manual retrieval. Production AI systems do this automatically with vector databases and embeddings. But for a solo developer or small team, dragging files into a folder works fine and costs nothing. I also like how easy it is to swap things around, and the mental load of using this system is extremely low.

`plans/` solves the multi-session problem. Long-horizon tasks fall apart when context resets between sessions. The Manus team found that their agents average around 50 tool calls per task. That’s a lot of state to lose. What I noticed especially is that I’d already have APIs for certain tasks, but because there was no shared state it was a big trap for agents to recreate APIs because they were not aware of their existence.

My solution is very low-tech: each project gets a folder with a README explaining the goal and a series of numbered prompt files. Prompt 01, prompt 02, prompt 03. An agent runs one prompt, finishes, and the next session picks up at the next number. The numbered prompts act as checkpoints. They also force me to think through the approach before any code gets written.

`messages/` handles coordination between agents. When an agent hits a blocker, it writes a markdown file. Another agent (or me) reads it and responds. Delete the file when it’s resolved.

This is crude compared to how production systems handle inter-agent communication. But it works, and I can see exactly what’s happening by listing the directory.

Why files

Files are the lowest common denominator. Practically every AI tool can read files. Every human can read files. Git can track them. You can grep them. You can edit them with any text editor. It’s a way to make it possible to work with agents from any provider. Whether you like to run Claude Code and Codex, or Cursor and Droid, I can guarantee you that this setup is bullet-proof. It simply works.

Files also externalise memory. The Manus team talks about this: treating the filesystem as unlimited context that agents can read and write on demand. You can’t predict which piece of information will matter ten steps later, so you store everything and let the agent retrieve what it needs.

The filesystem is also reversible. You can drop detailed content while keeping a reference (a URL, a file path) that lets you retrieve it later. The context folder is just manual retrieval. But manual retrieval that I control.

And fourth: the cognitive load is very low. What I like about this system is how simple, yet powerful it is. You don’t have to understand RAG or semantic search. And don’t get me wrong, those tools are extremely powerful and useful. But sometimes, a simple foundation is the best.

What I learned

A few things became clear after using this setup for a while:

Simplicity is king. After many years as an engineer, I’m always surprised by how often it boils down to this. A simple system with very little cognitive overhead often outperforms a super complex over-engineered solution.

Explicit context beats implicit assumptions. When context lives in a folder, I know exactly what the agent sees. There’s no guessing about what it remembers from earlier in the conversation.

Structure prevents drift. Numbered prompts force sequential execution. Agent A does step 1, agent B does step 2. Nobody skips ahead. Nobody redoes work.

Errors should stay visible. I used to delete failed attempts to keep the context clean. Bad idea. When the agent sees that something failed, it updates its behaviour. The Manus team found the same thing: keeping error traces in context improves recovery.

Variation prevents loops. If I use the exact same prompt format every time, the agent falls into predictable patterns. Adding small variations in phrasing keeps it from going on autopilot.

What I’m still figuring out

The message system works but it’s clunky. Agents don’t naturally check for messages unless you tell them to. I’ve been adding “check /messages for any blockers” to my prompts, but it’s manual.

Plans work great for features I understand upfront. They’re less useful for exploratory work where I don’t know what the steps are yet.

And the whole thing assumes you’re comfortable with files and folders. If you’re building in an environment where that’s awkward (some cloud IDEs, for example), this might not fit.

Try it if you want

I put the template on GitHub because other people seemed to have the same problem. Clone it, and try building something with it using your agent of choice. I hope it works as well for you as it did for me!

And if you find ways to make it better, I’d like to hear about them.

The repo can be found here: Github

Further reading

- Context Engineering for AI agents) (Manus team)

- [Effective Context Engineering](Effective Context Engineering) (Anthropic)

The Surface Talk

I’ve noticed that discussions often center around buzzwords like semantic search, "agents," or the integration of chat interfaces. But do these conversations truly translate into a profound understanding within product teams?

While teams can identify customer problems, propose solutions, and collaborate seamlessly with design and development, I think the current challenge lies in their ability to draft a comprehensive and future-proof Product Requirements Document (PRD) for an "AI-powered" feature.

Non-deterministic PRDs

From what I’ve seen, it’s not unusual for 9 out of 10 PMs to find themselves floundering when tasked with the challenge of articulating a PRD for an "AI-powered" feature.

They struggle because generative AI systems introduce a layer of non-determinism, demanding a consideration of additional requirements beyond the surface-level demo applications.

Beyond Surface-Level Understanding

When confronted with generative AI concepts, there is one overarching question that has to be solved: "How can we ensure a consistently high-quality user experience across a spectrum of use cases?"

When talking to PM’s, their typical response, deflecting responsibility to engineers and trusting them to navigate the intricacies, falls short. This approach disregards the fundamental truth that, similar to any other product feature, PMs must articulate details encompassing goals, outcomes, and user experience.

Develop an understanding of ‘hard skills’

So how do product managers proceed? Well, I think that they should develop a very good understanding of what it takes to build an AI product or feature. Let’s dissect the essential components that are the foundation for AI applications, extending beyond the constraints of conventional PRD templates:

1. Prompt Strategy, Writing, and Testing:

This is about figuring out how to talk to the AI. What words or questions will make it understand and respond the way we want? It's like teaching the AI the right language to get the best results. Writing good prompts is crucial, and testing them ensures the AI understands and gives the desired output.

2. Scoping Agent Tasks:

Here, we decide what jobs the AI should do. It's like telling the AI what's on its to-do list. We need to be clear about the tasks we want it to handle, setting the boundaries so it doesn't get confused or go off track. This ensures the AI focuses on what it's meant to do.

3. Retrieval Strategies:

The ability to efficiently fetch pertinent information distinguishes a proficient AI system. Understanding and implementing effective retrieval strategies ensure that the AI doesn't merely respond but does so with a depth of knowledge, enhancing the overall user experience.

4. Context Management:

This is like helping the AI remember what it talked about earlier. It's about keeping track of the conversation so that the AI doesn't forget important details. Managing context ensures a smoother and more coherent interaction with the AI over time.

5. Feedback Strategies:

Feedback serves as the compass for AI improvement. Establishing effective feedback loops is not just about receiving input; it's about structuring mechanisms that enable continuous learning. Whether from user interactions or system evaluations, strategic feedback strategies are the conduits for refining AI performance, fostering a cycle of iterative enhancement.

6. Unit Economics:

In the domain of generative AI, unit economics extend beyond traditional business metrics. It involves evaluating the cost and value associated with each AI interaction. Understanding the resource allocation, computational expenses, and the value derived from AI outputs is essential. Unit economics in AI is a perspective that product leaders must adopt to ensure efficiency and sustainability.

7. Security & Prompt Injection Hardening:

This is about making sure the AI is safe and protected. It's like putting a strong lock on the door to prevent any unwanted access. Ensuring security and prompt injection hardening safeguards the AI from potential risks, making sure it operates in a secure environment.

8. Model Selection:

Choosing the right AI model is like picking the best tool for a job. Different models have different strengths. Model selection is about understanding which one suits our needs the most, ensuring we get the best performance from the AI.

9. Identifying and Structuring Sources for Fine Tuning:

Fine-tuning is the sculpting of AI proficiency. It involves identifying and structuring diverse data sources to enhance the model's capabilities. Product leaders must develop an eye for relevant data, understanding how to structure it to refine AI behaviour. Fine-tuning is an ongoing process, a continuous refinement that separates a proficient AI system from the ordinary.

10. Chains & Agent Orchestration:

In the symphony of AI interactions, chains and agent orchestration play a pivotal role. It's the art of choreographing the sequence of actions, ensuring a harmonious flow of responses. Product leaders must master the orchestration of AI agents, structuring their interactions in a seamless manner. Chains and orchestration are the keys to orchestrating a symphony of AI responses that resonate with user expectations.

11. Analytics & Quality Management:

Analytics is like keeping track of how well the AI is doing its job. It's about measuring its performance and identifying areas for improvement. Quality management ensures that the AI consistently delivers high-quality results, maintaining a standard that meets our expectations.

Hands-On Learning

To rapidly acquire these essential skills, a hands-on approach is essential. It's not just about building things; it's about thinking how to guide your team through the development process. What are the core requirements? How do you quantify success for your team? How do you communicate specifications for an experience that is inherently non-deterministic? How do you design for trust?

Looking Ahead

As I wrap up, I find myself leaning towards a personalised perspective. What we should do as product leaders becomes clear-embrace the challenge, develop a deep understanding of generative AI, and personally equip yourself with the hard skills that I think will shape the future of product development for years to come.

Anyone who works in technology can't help but notice the unstoppable flow of artificial intelligence (AI) startups over the last year. It feels as if a floodgate has opened, and AI applications and startups are popping up everywhere. Let's step back and see what happened over the last year and what impact OpenAI’s developer day will have on the future of the AI startups.

A Glimpse Into the Last Year

Starting about a year ago, AI and Large Language Models (LLMs) held the promise of a revival for the startup ecosystem. The potential of this technology was promising in a hardening climate for startups, with high interest rates and lower valuations. It seemed poised to rejuvenate an industry that had faced its fair share of trials. Companies like Jasper were at the lead of this technological resurgence, leading the charge.

But in tech, nothing remains the same for long. That horizon has evolved clearly now, casting a dark shadow on the earlier optimism. With OpenAI’s developer day introducing more trouble. Jasper, once the poster child of AI startups, is now slowing down. It’s reminding us that in the tech world, even the best companies can stumble.

I think that after a year the AI startup ‘realm’ can be divided into three categories: the (I) losers, (II) the winners, (III) and the bold ones.

1. The Losers

Let’s start with what I call “The Losers”. In this category, we find the likes of Jasper - and the venture capitalists who bet big on them. And with OpenAI introducing their Assistant API even ‘moat' companies such as Pinecone are in trouble. Jasper's quite lofty valuation, built upon a tech foundation that is essentially a wrapper around OpenAI’s API is now facing stiff competition.

Yes - their user experience is solid, their brand is reputable, but these components can only carry a company so far. Competition from companies crafting specialised, high-value niche products is putting immense pressure on Jasper's growth. The uncertainty is high, and I think that VCs may be in for a rough ride.

Another set of losers includes VC-backed teams that joined the chatbot goldrush, raising sums ranging from $250,000 to $25 million from ~ December to March. Their dream was to cater to later-stage and enterprise companies. These startups, while more focused than Jasper, also lacked the "tech moat" that sets them apart. Their products, though promising, often lack the unique elements necessary for survival in this competitive landscape.

I think that the underlying issue here is the misjudgment of the startup community. They believed that enterprise executives, excited about AI, would be promising first customers.

Yet, what happened is that executives and enterprise customers, when given the choice between developing their AI solutions using open-source tools or buying from unproven startups, they chose the former.

And sadly for many startups, this choice has been extremely common for executives amongst different skill levels. It's a clear testament to how large companies are building their own AI success stories, stopping startups from achieving the growth metrics that they need to raise their next round.

This shift in the way technology is adopted is part of a larger trend that deserves more exploration, but that's something I will focus on in another post.

2. The Winners

But amidst the unfolding drama, there are winners who emerge in two distinct groups:

The first group comprises established companies and market incumbents. These companies, often accused of being slow to adapt, have, to much surprise, smoothly integrated AI into their products. These are the common use-cases, where they have build internal applications like "chat-your-docs”. I think that the reasons for their success are two-fold.

Firstly, these companies recognised AI adoption as a life-or-death proposition. Failure here would mean a slow decline, which they can not afford. They'd rather lead projects internally to ensure everything goes as intended.

Secondly, there's a breath of fresh air going through the C-Suite because of a harder environment, which has created the potential for ambitious projects. This change in attitude owes a debt of gratitude to people like Elon Musk, who have shown what's possible when a small group of highly motivated individuals set their minds to it. Reduce red tape, increase personal responsibility, and watch the magic unfold. Lean teams are in.

The second group of winners are: indie developers and solopreneurs. These small, often one-person enterprises don't raise outside capital or build extensive teams. Their advantage lies in their speed and ability to adapt quickly with low overhead. They create niche products for niche markets and often dominate them. Their goal is to develop a SaaS product generating approximately $10,000 per month in relatively passive income, a strategy sometimes known as "micro-SaaS."

They operate in the intersection of software development and content marketing, answering only to the market and their own intuition. They don't need to aim for billion-dollar exits or $100 million in annual recurring revenue (ARR). They iterate rapidly, ruthlessly shutting down underperforming products.

The synergy between large language models and text-to-image models, such as Stable Diffusion, has breathed life into these entrepreneurs. The success stories are common, with a lot of apps launched within the last six months. The lifestyle and freedom these companies afford to those that perform well are alluring, and thus attracting more talented builders to this category.

This group of winners is poised to grow, creating real value with this transformative technology. As we move forward, their impact is likely to be profound. We will see much more products being built by small - yet powerful teams.

3. The Crazy Ones

The final group to consider are ‘The Crazy Ones’. These companies are reimagining entire industries from the ground up. They are venture-backed, creating products that could redefine how highly skilled individuals interact with and are assisted by technology. While it's too early to determine their success, early prototypes have been intriguing. This is the most exciting segment to watch, and it holds immense potential for the future.

In this category, we find ventures like Cursor, an AI-first code editor that could revolutionise software development, Harvey.ai, an AI-powered solution for legal practices, and RunwayML, an AI-powered video editor, among others. The potential for these companies to disrupt their respective industries is immense, making this space a captivating one to observe.

Getting Ready for Round 2

As we think about the next phase of AI startups, I think that we should acknowledge that this landscape is in its infancy. It will continue to evolve as more foundational models emerge and toolchains improve. Those will be adapted for niche use-cases. Open source models also are recapturing ground quickly on OpenAI and Anthropic. And yes - counter examples will certainly emerge, challenging this narrative.

As I wrap this up, it's clear that the AI landscape will be extremely dynamic for at least a few more years. The Winners, Losers, and the Crazy Ones are navigating an unpredictable landscape, with the promise of tomorrow's possibilities being even crazier.

In the realm of artificial intelligence, OpenAI is embarking on a journey that transcends the boundaries of traditional computing, redefining the very essence of what a computer can be. This vision extends far beyond the scope of Large Language Models (LLMs) and touches upon several foundational pillars, each poised to revolutionise the way we interact with technology.

A glimpse into the future:

OpenAI's ambitious vision begins with the profound understanding of user preferences. The ultimate goal is for AI to know exactly what a user desires, down to the most specific details. This is the moment when technology transforms from being intimidating to becoming genuinely helpful. It's a vision that traces its roots back to October 2011, when Steve Jobs showcased the vision behind Siri. Although it has been unattainable for years, we are finally closing in on that decade old vision.

The power of real-time data:

While a significant portion of an AI's utility is derived from its foundational training and refinement through human feedback, the true potential lies in its ability to tap into real-time, external data sources. Collaborations with platforms like Zapier are the initial steps, but the real change lies in integration with third-party applications and data pipelines. The scope extends beyond minor tasks like "chatting with a PDF."

Unleashing AI's computing potential:

To break free from the constraints of contextual limitations, AI providers must empower AI to execute tasks within virtual environments, similar to Python or Node/Deno virtual realms. This approach allows AI to consume vast amounts of data, akin to traditional computers. While these virtual environments are presently used by data analysts and professionals, they are poised to evolve into long-term data processing hubs, which will transform data analysis and cross-file inference.

The art of agent task and flow planning:

Effective planning hinges on the accurate understanding of user intent. Unraveling intent has been a longstanding challenge, and LLMs have finally cracked the code, providing the keys that were sought for years, with the help of NLP techniques (a technique that now seems incredibly dated!).

Once intent is accurately grasped, the orchestration of an agent planner begins. This process calls for seamless integration with user preferences, third-party data sources, and a profound understanding of computational capabilities.

An ecosystem of expertise:

OpenAI's current focus on ChatGPT is merely the tip of the iceberg. As AI continues to evolve, a diverse array of specialised assistants is set to emerge. Builders will soon have the power to combine multiple tools into complex workflows, while AI itself will learn from the pioneers, heralding a new era of creative integration and innovation.

Enriching memory and experience:

AI embeddings and vector databases offer a foundation, but they lack essential elements such as context switching, conversational centroids, summarisation, and enrichment. The future involves embedding history and persistence, unlocking the potential for long-term memory enriched with pointers to critical subjects, emotions, tone, and more.

Core memory is only the starting point; the ultimate goal is to capture the intricate structure of information that our minds conjure when reminiscing about past experiences.

Redefining time-bound tasks:

The term "agent" may carry varied interpretations, but its essence lies in tasks that can be scheduled and autonomously completed, regardless of the timeframe. Tasks such as "Let me know when flights from Amsterdam to New York are less than €500" necessitate intricate coordination across API providers and virtual environments in the cloud.

The future of user interfaces:

While text-based chat remains a cornerstone of human-AI interaction, it does not represent the ultimate evolution of user interfaces. Elements such as buttons, date pickers, and images in applications simplify and clarify interactions. AI is poised to become an adept co-pilot, customising its approach to cater to the unique needs of each user. The future of UI is inherently dynamic, adapting to individual optimisation requirements.

AI as a tool composer:

AI is ushering in a future where users can construct their own workflows and combine APIs, circumventing the need to wait for startups to provide front-end solutions. This transition reduces the dependency on apps and startups to generate front-ends, empowering AI to compose a diverse arsenal of tools and APIs, coupled with a gas fee or tax.

A symphony of assistant interactions:

In the foreseeable future, an ecosystem of specialised assistants is set to develop, each contributing and collaborating with other assistants to achieve objectives. In this collaborative environment, assistants must adapt to diverse communication modalities, spanning text, APIs, file systems, and other channels embraced by agents, startups, and humans, as integration evolves to a deeper level.

The future of plugin and app stores:

Specialised assistants come to life through the combination of tools, APIs, prompts, data, preferences, and more. The current OpenAI plugin store is merely the beginning; expect more refinement and expansion as these plugins evolve into essential components of the ecosystem.

This exposition is a glimpse into the major challenges that are still ahead of us. Behind the scenes, a set of problems awaits, encompassing internet search and data scraping, community involvement, dynamic API generation, and integration via various devices such as glasses and earbuds. The realm of AI remains far from saturated, offering boundless prospects for innovation and creativity as we continue to iterate and expand our horizons.

We are at the beginning of a remarkable journey. The future of AI promises to be a thrilling one, brimming with opportunities for those who embrace the ever-evolving landscape. OpenAI is leading the charge for now, but we're just getting started.

In this short note, I will try to walk you through integrating Retrieval-Augmented-Generation (RAG) into any web app under 5 minutes. Retrieval-Augmented Generation is a blend of text retrieval and text generation that empowers your web app to deliver dynamic, context-aware content to its users.

Setting the Stage

Before we start with integrating Retrieval-Augmented-Generation into your web app, let’s attach the necessary packages. In this case, we need to harness the following crucial dependencies:

• dotenv: For managing environment variables.

• langchain/document: To work with language documents.

• langchain/embeddings/openai: To utilize OpenAI embeddings.

• langchain/vectorstores/supabase: For storing and retrieving vectors.

• @supabase/supabase-js: To establish a connection with a Supabase database.

• langchain/text_splitter: To split text into manageable chunks.

In addition to these specialised packages, we'll also make use of Node.js built-in modules, namely fs and path, for handling file system operations.

Here's how you can begin by importing these dependencies:

import dotenv from "dotenv";
import { Document } from "langchain/document";
import { OpenAIEmbeddings } from "langchain/embeddings/openai";
import { SupabaseVectorStore } from "langchain/vectorstores/supabase";
import { createClient } from "@supabase/supabase-js";
import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";

import fs from "fs";
import path from "path";

dotenv.config({ path: `.env.local` });

Preparing Your Documents

A critical aspect of this integration process is ensuring you have some nice data to work with. For this example, let's assume you possess a set of text documents stored within a specific directory that are written in Markdown. The initial step involves loading and splitting these documents for further processing.

const fileNames = fs.readdirSync("your_documents");
const splitter = RecursiveCharacterTextSplitter.fromLanguage("markdown", {
  chunkSize: 1000,
  chunkOverlap: 50,
});

const langchainDocs = await Promise.all(
  fileNames.map(async (fileName) => {
    const filePath = path.join("blogs", fileName);
    const fileContent = fs.readFileSync(filePath, "utf8");
    const splitDocs = await splitter.splitText(fileContent);
    return splitDocs.map((doc) => {
      return new Document({
        metadata: { fileName },
        pageContent: doc,
      });
    });
  })
);

In this code, we perform the following steps:

• Retrieve the names of files within the "your_documents" directory.

• Employ the RecursiveCharacterTextSplitter to segment the content into manageable chunks suitable for processing.

• Create Document objects for each chunk, complete with metadata, such as the filename.

Integration with Supabase

With your documents ready to go, the next phase of the integration journey involves incorporating them into a Supabase VectorStore. Supabase is an excellent choice for this purpose.

const auth = {
  detectSessionInUrl: false,
  persistSession: false,
  autoRefreshToken: false,
};

const client = createClient(
  process.env.SUPABASE_URL,
  process.env.SUPABASE_PRIVATE_KEY,
  { auth }
);

await SupabaseVectorStore.fromDocuments(
  langchainDocs.flat(),
  new OpenAIEmbeddings({ openAIApiKey: process.env.OPENAI_API_KEY }),
  {
    client,
    tableName: "documents",
  }
);

In this code, we follow these steps:

• Configure Supabase authentication for secure access.

• Create a Supabase client to establish a connection with the database.

• Utilize SupabaseVectorStore to store your language documents, ensuring they are readily accessible.

• Harness OpenAIEmbeddings to embed your text, preparing it for retrieval and generation.

Adding It as a Build Script

To streamline this integration process as part of your web app's build cycle, I’d recommend implementing it as a custom script within your project's package.json file. Running this script allows you to update or add new content to your database automatically.

Here's an example of how you can include this script in your package.json:

"scripts": {
  "build-content": "node path/to/your/embeddings.js"
}

Be sure to replace "path/to/your/embdeddings.js" with the actual path to the script containing the previously discussed code. This modification enables you to initiate the integration process by executing the command npm run build-content.

Summary

In this concise note, I've delved into the process of seamlessly integrating Retrieval-Augmented-Generation into any web application.

We initiated by importing the essential dependencies, meticulously organised our text data, and efficiently integrated it into a Supabase database to facilitate storage and retrieval.

This approach empowers you to provide your users with dynamic, context-aware content that elevates their experience with your web application. And all of that in under 5 minutes :).

After starting a conversational AI company in 2018, I’ve experienced a lot variations in language models. I've run quite a few A/B tests with various model combinations and APIs. Through this journey, I've learned some valuable insights for engineers and product builders that want to work with AI.

First and foremost, it's imperative to remember that end-users are more interested in the experience you offer rather than the specific model you employ. This is key to understand when building an AI oriented product. The model is merely a tool to deliver that experience seamlessly.

Building your own models might seem tempting, but it won't necessarily secure your position in the market. If GPT-4 can provide the same results as your in-house model, your competitive advantage is merely an illusion (and creates a LOT of overhead).

So, for businesses contemplating the deployment of Large Language Models (LLMs), the key considerations are:

1. Can you achieve the latency/speed and quality of output that your users demand?

2. What is the budget you are willing to allocate for achieving that level of quality?

Of course, the decision-making process involves a complex interplay of cost and time factors. Yet, here are some practical and quick guidelines for determining when to wrap GPT-3.5/4 or fine-tune your own model:

You are still searching for Product-Market Fit (PMF):

• For rapid iterations and quick feedback, almost always go for closed APIs or a pre-trained hosted open-source model.

• If your budget allows and you have the data, consider fine-tuning. However, remember that it extends the time needed to discover PMF.

Congrats! You are Post-PMF:

• Once you've identified your product's direction, your focus should shift towards optimising the combination of cost, latency, and quality.

• If GPT-3.5/4 is cost-prohibitive or too slow, consider training your own model.

• If GPT-3.5/4 aligns with your budget and latency requirements, continue using it.

• If your task is beyond the capabilities of closed APIs, training your own model becomes the obvious choice.

Additional points to consider:

• Assess your margins; using GPT-4 may make sense even if it's costlier than fine-tuning, as it allows for faster service improvements.

• Evaluate whether a fine-tuned model will truly outperform a competitor's use of GPT-4, particularly if you lack extensive data.

• Keep an eye on future trends; the costs of models like GPT-4 will likely decrease over time, and new models will emerge.

It's important to note that many at-scale use-cases thrive on a combination of closed APIs and in-house models.

In my exprience, people often lean toward training and fine-tuning models unnecessarily. While it can be fascinating, it may not always be the most efficient path. In the end, the product experience is the ultimate goal, and the model is just a tool to achieve it.

In today's fast-paced world, staying informed about current events has become an essential part of our daily lives. However, as I reflect on my own habits, I've come to a realisation—I might be consuming too much news. In this post, I'll explore the reasons why excessive news intake can be detrimental and why finding a balance is crucial.

Defining the Problem: News and Excess

Let's start by defining our terms. News, for the purposes of this discussion, refers to information published by news outlets like The New York Times or The Wall Street Journal, aimed at documenting and informing us about current events. Excess, in this context, refers to the considerable amount of time I spend reading the news—far more than two hours a day, in fact.

The Time Argument: A Stark Reality

When we break down the numbers, the reality becomes stark. Two hours of daily news consumption equate to 728 hours per month or 30 days in a year. If I assume I have approximately 64 years left to live, that's roughly five years spent solely on catching up with the news. It's a realization that underscores the extent of my obsession.

The Information Relevancy Argument: Quality Over Quantity

One of the drawbacks of news consumption is its lack of filtration and its statistical validation based on a sample size of one (n=1). While news provides a lot of information, it often falls short of revealing the truth. Contrast this with a book, which becomes more valuable over time. A book that's two years old has been tested for relevancy, truthfulness, and entertainment factor by numerous readers. Its worth is proven.

News, on the other hand, is frequently just hours or minutes old. Its long-term relevance remains unproven and unpredictable, making it risky to invest too much time in. It doesn't offer a holistic view of a subject; instead, it provides a single data point.

The Societal Incentive for News Organizations: The Business of Eyeballs

News organizations are driven by a societal incentive—more eyeballs equal more money, whether through subscriptions or advertisements. This incentive encourages them to continuously report on events, further inundating us with information.

Even respected outlets like The Economist, which publishes weekly, are not immune to this pressure for constant updates.

Finding a Balance: Benefits and Considerations

While there are undeniable benefits to staying up to date with current affairs—improved communication with peers and more informed decision-making—it's vital to strike a balance. Operating in a different context window than the rest of society can lead to challenges in decision-making.

In conclusion, my news obsession has led me to reconsider the quantity and quality of information I consume. While it's essential to stay informed, it's equally crucial to find a balance that allows for a healthier relationship with the news. In an age of information overload, mindfulness and moderation are keys to navigating the world of news consumption successfully.