JSON Migration Change Notes

Schemas

(See packages/db/src/schema/)

Every user prompt now has a Zod schema associated with it. The schema provides the contraints that the LLM output must adhere to.

Slidewriter Layout Data

(See apps/generator/tools/generate_slidewriter_enums.py, packages/db/src/schema/slidewriter-layout-data.ts)

I created a tool as part of the exporter to generate a list of valid layout names and placeholders for both the slidedoc and slideshow templates. This is used in the slidewriter schema to enforce what layout names and placeholder names the LLM can use.

Slidewriter Schema

The slidewriter schema is simply an arrry of slides each with a slide number, section number, layout name, and array of placeholders. The placeholder content should be in Markdown format, the exporter will parse the Markdown and adapt it for the PPTX output.

• slides: array
  • slideNumber: number
  • sectionNumber: number
  • layoutName: enum/string (constrained to layouts found in above layout data)
  • placeholders: array
    • name: enum/string (constraints to placeholder names found in above layout data)
    • content: string (Markdown content)

Completion Types

(See apps/app/src/types/completions.ts)

Typescript types for LLM output, should match their corresponding schema. Includes a mapping of Step ID to completion.

LLM Completion API

(See apps/app/src/server/ai/)

The internal functions used to generate LLM completions have been completely refactored and expanded apon.

• completion.ts: Main LLM call functions, all LLM calls eventually pass through here.
• completion-?.ts: Functions for specific types of LLM calls such as Empathy Walk calls or Storymap.
• request.ts: Functions for handling LLM call requests from front-end, includes handler for streaming calls.
• llm-calls.ts: Functions for logging LLM calls to database.
• prompt.ts: Functions for prompt generation, existed prior to JSON migration and largely unchanged except to support JSON objects.
• prompt-retry.ts: Functions for building prompts for retrying after failed validation.
• providers.ts: Functions for configuring and initalizing LLM providers.
• step.ts: Functions for updating steps in database as a result of a LLM call.
• types.ts: Types associated with LLM calls, includes properties expected by the LLM functions.

Front-end Streaming

All front-end LLM calls are streamed, regardless if the request to the LLM provider is streaming. Streams are done via server sent events (SSE). There are five event types...

• object: Current completion data.
• finish: The LLM call has completed successfully, the last object event contains the complete output. The stream will close after this event.
• error: An error occured. The stream will close after this event.
• pass: Start a new completion/validation pass (see below). This event will fire even on the first pass when no validation error has yet occured.
• call: Start a new completion during a chain completion (see below). This event will only fire during chain completion.

Completion Properties

These are some of the properties that can be sent to the getCompletion and getChainCompletion functions...

• processUid: Optional string that is used to track the current completion request for logging.
• messages: Array of messages to send to the LLM, should include system and user prompt.
• schema: Zod schema to force the LLM output to adhere to.
• stream: If true then the LLM output will be streamed by. Even if false the results are still streamed to the front end, they will just stream in as fully completed instead of partials.
• onFinish: Optional callback for when completion finished successfully.
• transform: (chain completion, see below)
• onNext: (chain completion, see below)

Validation Passes

(See apps/app/src/server/ai/completion.ts:L178)

A validation pass is the process of asking the LLM to redo a completion when a previous completion fails to validate (via Zod). It is allowed three attempts before giving up and throwing an error.

Chain Completions

(See apps/app/src/server/ai/completion.ts:L308)

Allows running multiple LLM calls back to back where the previous output informs subsequent calls. Pass a callback function, onNext to prompt the LLM on what to do next, or return null when finished. The transform callback converts the results of each completion to the single final output.
This still needs more work, it needs more flexibility to manipulate the prompts after each call.

Hooks

(See apps/app/src/hooks/use-ai-completion.ts, etc)

AI completion hooks merged in to use-ai-completion.ts which handles both streaming and non-streaming completions.

• use-ai-completion.ts: Entry point for receiving LLM completions in the front end. Processes completion event stream.
• use-empathy-walk.ts: Handles all Empathy Walk related completions, saves step data after a completion finishes. Wraps useAiCompletion.
• use-storymapper.ts: Handles Storymapper completion. Wraps useAiCompletion.
• use-slidewriter.ts: Handles Slidewriter completion. Largely unchanged, does not wrap useAiCompletion, instead calls Slidewriter route via projectService.runSlideWriter.

User Input

User inputs were changes to simple types that are pointers to elements inside the completion object. Available input types are...

• string: Used by RECEPTIVENESS to denote either 'curious' or 'resistant'.
• number: Used by BIG_IDEA to record the index of the selected idea.
• string[]: Used by CHALLENGES_AND_OPPORTUNITIES and CALL_TO_ACTION to record the selected elements. Each string in array is [column header name].[row index] i.e.... challenges.0, opportunities.1, doers.2.
• number[]: Used by AUDIENCE_JOURNEY to record selected row indexs.

AUDIENCE_SUMMARY is unchanged.

Completion Conversion Functions

(See apps/app/src/util/completion-to-*.ts)

Rather then updating all the front-end components to handle the new completion JSON objects I instead opted to create converters to convert them to the values expected by the front-end components.

• completion-to-markdown.ts: Converts completion objects to Markdown, could be useful for displaying completions to users.
• completion-to-sections.ts: Converts storymap and slidewriter completions to Section[] which is needed to display the StoryMap.
• completion-to-table.ts: Converts completion objects to Markdown table objects for use with table front-end components. This might be worth keeping as is since we use generic table components to display different types of completions and this helps convert specific completion types in to generic tabular data.
• input-to-completion.ts: Takes user input data and returns the parts of the completion that was selected.

Front-end Components

(See apps/app/src/components/project-edit/main/)
table-row-selection-step.tsx, table-cell-selection-step.tsx, etc...

Updated step components to use new hooks. Use completion to table functions to convert completion objects to table data that is used for front-end rendering.

Database

So far no changes have been made to the database schema but the data types being stored in the user_project_steps table is no longer compatible with the previous.

• completionJson: The JSON completion straight from the LLM adhering to the types as defined in apps/app/src/types/completions.ts.
• completion: No longer used.
• input: Plain text, number, or array of numbers/text. See 'user input' above.

CLI

(See apps/app/src/cli/)

I only built up enough CLI tools to help me test things during the JSON migration project. I think these could be very useful for dev work in the future and taking some time to further build apon them could be great!

• cli:project:run: Creates a new project and runs through all the steps based on user input data stored in a JSON file (assets/cli-user-data/).
• cli:project:view: Dumps all completion data for an existing project.
• cli:project:view-ew: Dumps all Empathy Walk step data for an existing project.
• debug:prompt: Generate and dump a given prompt for an existing project.
• debug:converter: Converter a PPTX file (hard coded file path).
• debug:slidewriter-schema: Test Slidewriter schema validation.
• debug:storymap-sections: Test Storymap to section conversion. (apps/app/src/util/completion-to-sections.ts::slidedocStorymapToSections)

Exporter

Support for Markdown has been removed. All data sent to the exporter is now JSON, this includes both the slidewriter data as well as the empathy walk.
The slide data schema has been simplified, instead of having different placeholder data types for text, picture, table, and chart, placeholder content are now represented in plain text and parsed by the exporter.

Placeholders

text: Markdown text. Supports simple Markdown formatting... bold, italtic, lists, line breaks.
table: CSV table.
picture: Plain text file name, path pointing to source slide number / name, URL. Can be parsed from Markdown image link.
chart: Plain text path pointing to source slide number / name. Can be parsed from Markdown link.

Admin

Markdown completions have been removed from the admin as they are no longer saved in Markdown format.