Building a Data Classification Agent

Learn how to create an AI-powered document classification system with Prisme.ai

This tutorial guides you through building an intelligent document classification system using Prisme.ai. You’ll create a solution that automatically categorizes uploaded documents into types such as invoices, CVs, quotes, contracts, and more using the power of generative AI.

What You’ll Build

A complete document management system with:

• A user-friendly document upload interface
• Automatic AI-based document classification
• A searchable and filterable document repository
• Document management capabilities (view, categorize, delete)

This solution combines a classification agent built in Agent Creator with the Collection app for document storage, creating a powerful yet simple document management system.

Prerequisites

Before starting this tutorial, make sure you have:

• An active Prisme.ai account
• The Agent Factory App installed in your workspace
• An Agent Factory API key generated in Agent Creator (open any agent → Settings → API Keys) and pasted into the Agent Factory App’s configuration field apiKey after installing it. This is required when the form page is public — without it, Agents.sendMessage returns 401 UNAUTHORIZED for unauthenticated form submissions.
• The Collection app installed in your workspace
• A classification agent created in Agent Creator with system instructions like: Classify the document within these categories: invoice, CV, quote, contract, others. Provide your categorization based on the content. Answer with the category only.
• Basic familiarity with Prisme.ai Builder

Step 1: Creating Your Document Management Workspace

Let’s start by setting up a dedicated workspace for our document management system:

1. Access Builder

   Log in to your Prisme.ai account and navigate to Builder.

2. Create a New Workspace

   Click + Create new workspace, choose From Scratch, name your workspace “Document Management System”, add a short description, and click Create Workspace.

The screenshot above shows the wizard with the example name “Tutorial — Webhook Builder”. For this tutorial, fill in “Document Management System” instead.

Step 2: Building the Document Upload Interface

Now, let’s create the upload interface. In v27 Builder, the frontend of a workspace is a single editable page called index — a React + Vite + Tailwind + Radix SPA, not a block canvas.

If you came from the legacy Pages model, the block YAML (slug: Form, slug: DataTable) is gone. Pages are React SPA source trees; the platform compiles and previews them inside Builder. The page talks to automations through HTTP webhooks and WebSocket events instead of onSubmit / updateOn block events.

1. Open Pages

   In your workspace sidebar, select Pages. The editable index entry is your workspace’s React app.

2. Switch to Code mode

   Use the page toolbar to switch to Code. If the workspace has no source yet, initialize the React/Vite/Tailwind/Radix starter template — Builder will scaffold src/App.tsx, src/main.tsx, src/styles/globals.css, index.html, package.json, and vite.config.ts.

3. Edit src/App.tsx

   Open src/App.tsx and replace its contents with the component below. It renders an upload form (file input + description textarea), lists uploaded documents in a table, and stays in sync with the backend through documents.updated events.

   The component receives this prop interface from the platform:

   interface AppProps {
     sdk: SDK
     user: unknown
     workspace: { id: string; slug: string; name: string }
     backends?: Record<string, { slug: string }>
   }
   

   A complete, working example:

   import { useEffect, useState } from 'react'
   
   interface AppProps {
     sdk: any
     user: unknown
     workspace: { id: string; slug: string; name: string }
   }
   
   interface DocumentRow {
     _id: string
     description: string
     category?: string
     attachment?: string
   }
   
   export default function App({ sdk, workspace }: AppProps) {
     const [description, setDescription] = useState('')
     const [file, setFile] = useState<File | null>(null)
     const [documents, setDocuments] = useState<DocumentRow[]>([])
     const [submitting, setSubmitting] = useState(false)
     const [error, setError] = useState<string | null>(null)
   
     const baseUrl = `${sdk.host}/workspaces/slug:${workspace.slug}/webhooks`
     const headers = {
       'Content-Type': 'application/json',
       ...(sdk.token ? { Authorization: `Bearer ${sdk.token}` } : {}),
       ...(sdk._csrfToken ? { 'x-prismeai-csrf-token': sdk._csrfToken } : {}),
     }
   
     async function loadDocuments() {
       const res = await fetch(`${baseUrl}/list-documents`, { method: 'POST', headers, body: '{}' })
       const data = await res.json()
       setDocuments(data.documents ?? [])
     }
   
     async function handleSubmit(e: React.FormEvent) {
       e.preventDefault()
       setSubmitting(true)
       setError(null)
       try {
         // Demo: send the file name as `attachment`. For real uploads, POST the
         // file to `${sdk.host}/v2/files` first and pass back the returned URL,
         // or read the file as base64 and send it inline.
         const res = await fetch(`${baseUrl}/submit-document`, {
           method: 'POST',
           headers,
           body: JSON.stringify({
             description,
             attachment: file?.name ?? null,
           }),
         })
         const data = await res.json()
         if (!data.success) throw new Error('Submission failed')
         setDescription('')
         setFile(null)
       } catch (err: any) {
         setError(err.message ?? 'Unknown error')
       } finally {
         setSubmitting(false)
       }
     }
   
     async function handleDelete(id: string) {
       await fetch(`${baseUrl}/delete-document`, {
         method: 'POST',
         headers,
         body: JSON.stringify({ id }),
       })
     }
   
     useEffect(() => {
       loadDocuments()
       let cancelled = false
       let cleanup: (() => void) | undefined
   
       ;(async () => {
         const events = await sdk.streamEvents(workspace.id, { 'source.sessionId': true })
         if (cancelled) return
         const handler = () => loadDocuments()
         events.on('documents.updated', handler)
         cleanup = () => events.off?.('documents.updated', handler)
       })()
   
       return () => {
         cancelled = true
         cleanup?.()
       }
     }, [workspace.id])
   
     return (
       <div className="mx-auto max-w-3xl p-6 space-y-8">
         <header>
           <h1 className="text-2xl font-semibold">Document Management System</h1>
           <p className="text-sm text-gray-500">Upload a document and let the agent classify it.</p>
         </header>
   
         <form onSubmit={handleSubmit} className="space-y-4 rounded-xl border p-6">
           <div>
             <label className="block text-sm font-medium mb-1">Document</label>
             <input
               type="file"
               accept=".pdf"
               onChange={(e) => setFile(e.target.files?.[0] ?? null)}
               className="block w-full text-sm"
             />
           </div>
           <div>
             <label className="block text-sm font-medium mb-1">Description</label>
             <textarea
               value={description}
               onChange={(e) => setDescription(e.target.value)}
               placeholder="Describe the document here."
               className="block w-full rounded-md border px-3 py-2 text-sm"
               rows={4}
               required
             />
           </div>
           {error && <p className="text-sm text-red-600">{error}</p>}
           <button
             type="submit"
             disabled={submitting}
             className="rounded-md bg-black px-4 py-2 text-sm font-medium text-white disabled:opacity-50"
           >
             {submitting ? 'Submitting...' : 'Submit'}
           </button>
         </form>
   
         <section>
           <h2 className="text-lg font-semibold mb-3">Uploaded Documents</h2>
           <table className="w-full text-sm border">
             <thead className="bg-gray-50">
               <tr>
                 <th className="text-left p-2">ID</th>
                 <th className="text-left p-2">Description</th>
                 <th className="text-left p-2">Category</th>
                 <th className="text-right p-2">Actions</th>
               </tr>
             </thead>
             <tbody>
               {documents.map((doc) => (
                 <tr key={doc._id} className="border-t">
                   <td className="p-2 font-mono text-xs">{doc._id}</td>
                   <td className="p-2">{doc.description}</td>
                   <td className="p-2">{doc.category ?? '—'}</td>
                   <td className="p-2 text-right">
                     <button
                       onClick={() => handleDelete(doc._id)}
                       className="text-red-600 hover:underline"
                     >
                       Delete
                     </button>
                   </td>
                 </tr>
               ))}
               {documents.length === 0 && (
                 <tr>
                   <td colSpan={4} className="p-4 text-center text-gray-500">
                     No documents yet.
                   </td>
                 </tr>
               )}
             </tbody>
           </table>
         </section>
       </div>
     )
   }
   

   The example uses plain <form>, <input>, <textarea>, <table> styled with Tailwind. Radix UI primitives are available in the template if you want richer controls (dialogs, dropdowns, toasts), but they aren’t required for this tutorial.

   Dependencies are minimal: react (provided by the template) and the injected sdk prop. No extra packages are needed.

4. Preview the app

   Switch to Preview. Builder compiles the source and renders your SPA inside the device viewport (Desktop / Tablet / Mobile). You should see the upload form and an empty documents table.

5. Save the page

   Click Save. Builder synchronizes the source files to the workspace.

Switch to Code to see the file tree (src/App.tsx, src/main.tsx, src/styles/globals.css, index.html, package.json, vite.config.ts) and edit the React source.

Step 3: Creating the Document Classification Automation

Next, let’s set up the automation that will classify documents when they’re uploaded:

Store your agent ID under workspace Configuration as classifierAgentId so it’s easy to update without editing the automation.

1. Navigate to Automations

   In your workspace, go to the Automations section.

2. Create the Submit Document webhook

   Create a new automation named “Submit Document” with the following configuration. It exposes an HTTP endpoint, classifies the document, stores it, and emits a documents.updated event so any open SPA refreshes.

   slug: submit-document
   name: Submit Document
   when:
     endpoint: true
   do:
     - Agents.sendMessage:
         agent_id: '{{config.classifierAgentId}}'
         message:
           parts:
             - type: text
               text: '{{body.description}}'
         output: agentResponse
     - set:
         name: category
         value: '{{agentResponse.task.output.messages[0].parts[0].text}}'
     - Collection.insert:
         data:
           description: '{{body.description}}'
           attachment: '{{body.attachment}}'
           category: '{{category}}'
         output: inserted
     - emit:
         event: documents.updated
         payload:
           id: '{{inserted._id}}'
   output:
     success: true
     category: '{{category}}'
     id: '{{inserted._id}}'
   

   Agents.sendMessage returns an A2A task object. The agent’s text reply is at agentResponse.task.output.messages[0].parts[0].text — the set step above pulls just that string into category so the rest of the automation can use it directly. Open the activity log after a first run to see the full task object (status, usage, context ID).

   Make sure the agent uses a model your organization has enabled in Governe (e.g. gpt-4o). A disallowed model returns an error inside the same parts[0] shape with metadata.error: true.

   The classification logic itself (the categories, the tone of the answer) lives on the agent in Agent Creator, not in this automation. To tune the categorization, edit the agent’s system instructions there.

3. Create the List Documents webhook

   Create another automation named “List Documents”. The SPA calls it on mount and after each documents.updated event to refresh the table.

   slug: list-documents
   name: List Documents
   when:
     endpoint: true
   do:
     - Collection.find:
         query: {}
         sort:
           createdAt: -1
         output: data
   output:
     documents: '{{data}}'
   
4. Create the Delete Document webhook

   Create a final automation named “Delete Document”. It accepts { id } in the request body, removes the row, and emits documents.updated so connected SPAs reload.

   slug: delete-document
   name: Delete Document
   when:
     endpoint: true
   do:
     - conditions:
         '!{{body.id}}':
           - break: {}
         default: []
     - Collection.deleteOne:
         query:
           _id: '{{body.id}}'
     - emit:
         event: documents.updated
         payload:
           id: '{{body.id}}'
           deleted: true
   output:
     success: true
   
5. Deploy Your Automations

   Save all automations and make sure they’re properly configured to work together.

Step 4: Testing and Using Your Document Classification System

Now it’s time to test your document management system:

1. Access your document upload page

   Either preview the page directly inside Builder (Pages > index > Preview), or open the deployed URL at <workspace-slug>.pages.prisme.ai. The SPA owns its own routing, so there is no /lang/upload-docs path anymore.

2. Upload a test document

   • Pick a sample document (PDF) in the file input
   • Add a description that gives context about the document’s content
   • Submit the form
3. Verify classification

   The SPA POSTs to submit-document, which returns the inferred category. The documents.updated event then triggers a reload of list-documents, and the new row appears in the table with its AI-determined classification.

4. Test document management

   Click Delete on a row to call delete-document. The same documents.updated event removes it from every open SPA.

The Preview pane on the right renders the compiled SPA. Once src/App.tsx is in place, you’ll see the upload form and documents table here.

Step 5: Version Control and Deployment

To finalize your document classification system:

1. Push Your Workspace

   Use Push in Builder to save the current state of your workspace to its repository. See Versioning for details on how pushes and versions work.

2. Review Versions

   Open the Versions tab to browse the workspace history and roll back if needed.

3. Configure Access Controls

   Set up appropriate Role-Based Access Control to determine who can use your document management system. See RBAC.

Monitoring and Improving Your System

After deployment, regularly check the system’s performance:

1. Monitor Activity Logs

   Review the activity logs to track document uploads and classifications.

2. Review Classification Accuracy

   Periodically check if documents are being classified correctly and refine your system as needed.

3. Optimize for Performance

   Make adjustments to improve speed and accuracy as your document volume grows.

Extending Your Document Classification System

Your base system is powerful, but consider these enhancements:

• Advanced Classification: Refine your agent’s system instructions in Agent Creator, or build a more specialized agent per document family
• Content Extraction: Extract key information from documents based on their category
• Automated Workflows: Trigger specific actions based on document type (e.g., route invoices to accounting)
• Search Functionality: Add search capabilities to find documents by content or metadata
• Integration: Connect with other systems like CRM or ERP platforms

Next Steps

Build a Webhook Handler

Learn how to create webhook integrations for your document management system

Explore AI Contact Routing

Discover how to route documents to the right department automatically

Create a RAG Agent

Build an agent that can answer questions about your document repository

Integrate with Websites

Connect your document system to web content for enhanced capabilities