Learn how you can use Harmonized Unstructured data with Data 360 and Agentforce.

• Harmonized Unstructured Data With Data 360
• Table of Contents
  • What does it do?
  • How does it work?
  • Demo
    • Welcome screen
    • Message details
    • Harmonized Unstructured Data Modal
  • API Specification
    • Architecture diagram
  • Technologies used
• Configuration
  • Requirements
  • Setup
    • Local environment configuration
  • Deployment
    • Heroku Deployment
• License
• Disclaimer

This application demonstrates Salesforce Data Cloud's unstructured data harmonization capabilities integrated with Agentforce. It provides an interactive chat interface where users can query harmonized unstructured data through natural language conversations.

The demo showcases how Data Cloud can:

• Process and vectorize unstructured data chunks
• Store vectorized data as HTML objects with semantic search capabilities
• Enable Agentforce to retrieve relevant information with proper citations
• Provide transparent insights into AI-generated responses with metrics and source references

Data Processing Pipeline:

1. Data Ingestion: Unstructured data is uploaded to Data Cloud
2. Chunking: Data is split into manageable chunks for processing
3. Harmonization: Each chunk is converted into an Harmonized Unstructured Data Lake Object (HUDLO)
4. Storage: Vectorized data is stored as HTML objects in Data Cloud and mapped to Harmonized Unstructured Data Model Object (HUDMO)
5. Retrieval: Agentforce uses RAG (Retrieval-Augmented Generation) to query relevant HUDMO

User Interaction Flow:

1. User opens the chat widget and initiates an Agentforce session
2. User sends natural language queries about the data
3. Frontend generates HMAC-SHA256 signature for API authentication
4. Backend validates the request and forwards it to Agentforce
5. Agentforce retrieves relevant data using its internal logic
6. Response includes the answer and response message metadata
7. User can view detailed metadata
8. You can fetch the underlying HTML used for the Agentforce response generation

The application exposes four RESTful endpoints, all protected by HMAC-SHA256 signature validation:

GET /api/v1/start-session

• Initializes a new Agentforce session
• Query Parameters: sessionId (external session key)
• Headers: X-Timestamp, X-Signature
• Returns: { sessionId, messages } (Agentforce internal session ID and welcome message)

POST /api/v1/send-message

• Sends a message to an active Agentforce session
• Headers: X-Timestamp, X-Signature, Content-Type: application/json
• Body: { sessionId, message, sequenceId }
• Returns: { messages } (Array containing Agentforce response with metadata)

DELETE /api/v1/delete-session

• Terminates an active Agentforce session
• Headers: X-Timestamp, X-Signature, Content-Type: application/json
• Body: { sessionId }
• Returns: { success: true }

POST /api/v1/get-hudmo

• Retrieves harmonized unstructured data model object (HUDMO) content
• Headers: X-Timestamp, X-Signature, Content-Type: application/json
• Body: { hudmoName, dccid } (HUDMO name and Data Cloud Content ID)
• Returns: { data } (Harmonized HTML content with metadata and source URL)

Authentication: All requests require HMAC-SHA256 signature in headers:

• X-Timestamp: Current timestamp in milliseconds
• X-Signature: HMAC-SHA256(API_SECRET, timestamp + method + path)

Client

• React - UI framework
• TypeScript - Type-safe JavaScript
• Vite - Build tool and dev server
• Tailwind CSS - Utility-first CSS framework
• shadcn/ui - Component library
• Web Crypto API - HMAC-SHA256 signature generation

Server

• Node.js - JavaScript runtime
• Express - Web framework
• Salesforce Einstein Agentforce API v1 - AI agent integration
• OAuth 2.0 Client Credentials Flow - Salesforce authentication
• HMAC-SHA256 - Request signature validation

For a more detailed overview of the development & production dependencies, please check server package.json or client package.json.

To run this application locally, you will need the following:

• An active Salesforce account with Data Cloud
• Node.js version 20 or later installed (type node -v in your terminal to check). Follow instructions if you don't have node installed
• npm version 10.0.0 or later installed (type npm -v in your terminal to check). Node.js includes npm
• git installed. Follow the instructions to install git
• A Salesforce account enabled with Agentforce

1. Clone the repository

   git clone https://github.com/mvrzan/salesforce-data-cloud-unstructured-harmonization.git cd salesforce-data-cloud-unstructured-harmonization

2. Configure Server Environment Variables

   Copy the example file and fill in your Salesforce credentials:

   cd server cp .env.example .env

   Edit server/.env with your values:

   SALESFORCE_LOGIN_URL=https://your-instance.my.salesforce.com CLIENT_ID=your_salesforce_client_id CLIENT_SECRET=your_salesforce_client_secret AGENTFORCE_AGENT_ID=your_agentforce_agent_id API_SECRET=your_generated_secret_key

   Generate a secure API secret:

   node -e "console.log(require('crypto').randomBytes(32).toString('hex'))"

3. Configure Client Environment Variables

   cd ../client cp .env.example .env

   Edit client/.env with the same API secret and local API URL:

   VITE_API_URL=http://localhost:3000 VITE_API_SECRET=your_generated_secret_key

   ⚠️ Important: The API_SECRET on the server must match VITE_API_SECRET on the client.

4. Install Dependencies

   Install server dependencies:

   cd server npm install

   Install client dependencies:

   cd ../client npm install

5. Start the Application

   Start the server (from the server directory):

   npm run dev

   In a new terminal, start the client (from the client directory):

   npm run dev

6. Access the Application

   Open your browser and navigate to http://localhost:5173

Once you are happy with your application, you can deploy it to Heroku!

Prerequisites:

• Heroku CLI installed
• Heroku account created

Deployment Steps:

1. Create a Heroku App

   heroku create your-app-name

2. Set Environment Variables

   Configure all required environment variables in Heroku:

   heroku config:set SALESFORCE_LOGIN_URL=https://your-instance.my.salesforce.com heroku config:set CLIENT_ID=your_salesforce_client_id heroku config:set CLIENT_SECRET=your_salesforce_client_secret heroku config:set AGENTFORCE_AGENT_ID=your_agentforce_agent_id heroku config:set API_SECRET=your_generated_secret_key

3. Build and Deploy Client

   Build the client with production environment variables. Set both the API URL (pointing to your Heroku app) and the API secret:

   cd client VITE_API_URL=https://your-app-name.herokuapp.com VITE_API_SECRET=your_generated_secret_key npm run build

   The built files will be in client/dist/ and should be served by your server.

   Note: Make sure the VITE_API_URL points to your Heroku app URL and the VITE_API_SECRET matches the API_SECRET you set in step 2.

4. Deploy to Heroku

   git push heroku main

5. Open Your App

   heroku open

For more detailed deployment instructions, please follow the official Heroku documentation.

Security Note: When deploying publicly, be aware that the API_SECRET will be visible in the client bundle. For production use with external users, consider implementing additional security measures such as user authentication or IP whitelisting.

MIT

This software is to be considered "sample code", a Type B Deliverable, and is delivered "as-is" to the user. Salesforce bears no responsibility to support the use or implementation of this software.