Skip to main content

Deployment Architectures

See Also

AI.JSX runs in both NodeJS and the browser, so you can choose an architecture that meets your needs. Here are some options, listed in order of most to least client logic:

  1. Run entirely on the client Great for Hackathons
  2. UI + AI.JSX on the client; API calls on the server
  3. UI on the client; AI.JSX on the server Recommended
  4. Run entirely on the server In Progress
  5. Headless AI (no UI)

The key tradeoffs are:

  • Whether you need to protect API keys
  • Where you want to put serialization boundaries
  • Latency (as a result of roundtrips)
  • Whether you want to integrate AI and UI components
tl;dr

If you're starting a new project, we recommend using the UI on client; AI.JSX on server pattern with NextJS.

Where can AI.JSX run?

In addition to the client, AI.JSX can run in serverless/edge functions, traditional standalone servers, or any other NodeJS process.

Run Entirely on the Client​

With a pure client-side approach, you run your UI and AI.JSX logic on the client. However, you still need to call out to external APIs for hosted services like model providers (e.g. GPT-4 from OpenAI) or databases (e.g. Pinecone).

Pros:

  • Simple to implement
  • Easy to get started
  • The serialization boundary is at the API layer. The only way to access these APIs is through a serialization boundary, so your architecture isn't adding any new boundaries.

Cons:

  • Your API keys are exposed to the client. (Not recommended for public apps.)
  • Performance will suffer if you need to do many roundtrips

Example​

In the ai-jsx monorepo, run this command from the repo root for a demo of this architecture:

OPENAI_API_KEY=my-key yarn turbo run dev --scope create-react-app-demo

UI + AI.JSX on the client; API calls on the server​

This is just like the Run Entirely on the Client pattern, except you add a proxy for API calls. You'd traditionally set this proxy up as a serverless or edge function. The proxy stores your API keys, keeping them safe from the client.

Pros:

  • There's still only one serialization boundary.
  • API keys are protected from the client.

Cons:

  • Compared to pure client, it's a little more complicated because you need to set up a proxy.
  • Performance will suffer if you need to do many roundtrips

Example​

In the ai-jsx monorepo, run this command from the repo root for a demo of this architecture:

OPENAI_API_BASE='/v1' yarn turbo run dev --scope create-react-app-demo

See packages/create-react-app-demo/backend/index.ts for a sample proxy server.

Alternatively, use this template repo for an example that uses a Vercel edge function.

UI on the client; AI.JSX on the server​

Recommendation

This is the most stable way to run and deploy an AI.JSX powered app at the moment.

In this approach, your AI.JSX logic runs entirely in your backend environment.

Pros:

  • You can access your AI.JSX logic from multiple clients (web app, native app, etc).
  • API keys are secure.
  • Performance will be less sensitive to roundtrips between AI.JSX and external APIs.

Cons:

  • This introduces a new serialization boundary, which sits between the UI and your AI.JSX. This limits how seamless your UI/AI integration can be.

What's the downside of the new serialization boundary?​

When AI.JSX and the UI are both on the client, you can do things like:

const dataPromise = getMyPromise();

<div>
  <AI.jsx>
    <ChatCompletion temperature={1}>
      <UserMessage>Write me a poem about {dataPromise}</UserMessage>
    </ChatCompletion>
  </AI.jsx>
</div>;

In this example, we have a value from our UI logic, dataPromise, and we embed it seamlessly in our AI logic. This is possible because the UI and AI logic are running together on the client.

If the AI.JSX logic lived entirely serverside, then we'd need to serialize everything that gets sent between the UI and AI layers.

Run entirely on the server​

caution

Running entirely on the server is still a work in progress.

In this approach, you generate your HTML on the server and stream it to the client. If your app is full-stack JS, you'd typically do this with the help of a framework like NextJS.

Pros:

Cons:

  • In JS, requires use of a framework like NextJS to get the full benefits. (NextJS is a great choice for new apps, but existing codebases may not be able to adopt it immediately.)

Or, if you're generating HTML from a non-JS app (like Django), you'd have a separate NodeJS service that runs your AI.JSX logic:

Headless AI​

With this pattern, your AI does not directly feed any UI. Instead of acting in response to user actions, it may run offline as part of a batch process. Your app may be triggered by a cron service, task queue, or some other kind of background job.

Instead of UI, the output of your app may be writing to a data store, the file system, etc.