Hooks API

Detailed documentation for all Hooks APIs provided by React SDK.

useAaaSPilotKit

Provides controller instance and states maintained internally by Provider.

Options Parameter

For complete configuration options, please refer to Vanilla JS - Configuration Options.

v1.1.0 added asr configuration support for Baidu/Azure dual ASR service providers.

Interface Definition

function useAaaSPilotKit<AS extends BaseAgentService = BaseAgentService>(): {
    controller: ReturnType<typeof createAaaSPilotKit<AS>> | null;
    isReady: boolean;
    isMuted: boolean;
    isRendering: boolean;
    create: () => void;
    dispose: () => void;
}

Return Values

Property	Type	Description
controller	IAaaSPilotKitController | null	Controller instance, may be null
isReady	boolean	Whether the controller is ready
isMuted	boolean	Whether in muted state
isRendering	boolean	Whether currently rendering
create	() => void	Manually create controller
dispose	() => void	Manually destroy controller

Usage Example

Example 1:

function Dashboard() {
    const {controller, isReady, isMuted, isRendering} = useAaaSPilotKit();

    const handleSendMessage = useCallback(
        () => {
            if (controller && isReady) {
                controller.input('Hello, I would like to learn about product information');
            }
        },
        [controller, isReady]
    );

    const handleToggleMute = useCallback(
        () => {
            if (controller) {
                controller.mute(!isMuted);
            }
        },
        [controller, isMuted]
    );

    return (
        <div>
            <div>Status: {isReady ? 'Ready' : 'Initializing'}</div>
            <div>Audio: {isMuted ? 'Muted' : 'Normal'}</div>
            <div>Rendering: {isRendering ? 'In progress' : 'Idle'}</div>

            <button onClick={handleSendMessage} disabled={!isReady}>
                Send Message
            </button>
            <button onClick={handleToggleMute} disabled={!controller}>
                {isMuted ? 'Unmute' : 'Mute'}
            </button>
        </div>
    );
}

Example 2:

const Dashboard: FC<IPilotKitProps> = props => {
    const [isInactive, setIsInactive] = useState(false);

    const {create, dispose} = useAaaSPilotKit();

    const onInactivity = useCallback(
        () => {
            setIsInactive(true);
            dispose();
        },
        [dispose]
    );

    useAaaSPilotKitEvents({
        onInactivity
    });

    const restart = useCallback(
        () => {
            setIsInactive(false);
            create();
        },
        [create]
    );

    if (isInactive) {
        return (
            <div>
                Timeout

                <button
                    onClick={restart}
                >
                    Restart
                </button>
            </div>
        );
    }

    return (
        <Dialog
            {...props}
        />
    );
};

Notes

• Must be called within AaaSPilotKitProvider scope
• controller may be null, please check for null before calling
• create/dispose are used for manual lifecycle control, use with caution

useAaaSPilotKitEvents

Binds React component callback functions with the underlying event bus.

Interface Definition

function useAaaSPilotKitEvents(props: {
    onReady?: () => void;
    onAsrStart?: () => void;
    onAsrMessage?: (payload: IAsrMessageEventPayload) => void;
    onMicrophoneAvailable?: (result: MicrophoneAvailableResult) => void;
    onRenderStart?: (payload: IRenderStartEventPayload) => void;
    onInactivity?: (payload: IInactivityEventPayload) => void;
    onBusy?: (payload: IBusyEventPayload) => void;
    onInterrupt?: (payload: IInterruptEventPayload) => void;
    onConversationChange?: (payload: IConversationChangeEventPayload) => void;
    onIsRenderingChange?: (isRendering: boolean) => void;
    onReplyStart?: (payload: IReplyStartEventPayload) => void;
    onTtft?: (payload: ITtftEventPayload) => void;
    onError?: (error: IErrorEventPayload) => void;
}): void;

Supported Events

Event	Description	Payload Type
onReady	Controller initialization complete	None
onAsrStart	Start speech recognition	None
onAsrMessage	Speech recognition result	IAsrMessageEventPayload
onMicrophoneAvailable	Microphone device availability check result	MicrophoneAvailableResult
onRenderStart	Start rendering broadcast	IRenderStartEventPayload
onInactivity	User inactive for long time	IInactivityEventPayload
onBusy	System busy (route limit exceeded, etc.)	IBusyEventPayload
onInterrupt	Conversation interrupted	IInterruptEventPayload
onConversationChange	Conversation content changed	IConversationChangeEventPayload
onIsRenderingChange	Rendering state changed	boolean
onReplyStart	Digital Employee started broadcasting	IReplyStartEventPayload
onTtft	Agent first character latency performance metric	ITtftEventPayload
onError	Error occurred	IErrorEventPayload

onMicrophoneAvailable Detailed Explanation

🎙️ Microphone device availability check result event.

Trigger Timing:

• When manually calling controller.checkAudioDevice()
• Before ASR starts automatic detection (requires configuring checkAudioDeviceBeforeStart: true)

Payload Type:

interface MicrophoneAvailableResult {
  available: boolean;           // Whether device is available
  error?: AudioDeviceError;     // Error type (only when available=false)
  userMessage?: string;         // User-friendly error message text
  devices?: MediaDeviceInfo[];  // Detected audio device list
  permissionState?: PermissionState; // Permission state
}

Usage Example:

function DeviceStatusAlert() {
  const [deviceStatus, setDeviceStatus] = useState<string>('');
  const [showAlert, setShowAlert] = useState(false);

  useAaaSPilotKitEvents({
    onMicrophoneAvailable: (result) => {
      if (!result.available) {
        // Device unavailable, show error message
        setDeviceStatus(result.userMessage || 'Device unavailable');
        setShowAlert(true);
      } else {
        // Device available
        setDeviceStatus('Device normal');
        setShowAlert(false);
      }
    }
  });

  if (!showAlert) return null;

  return (
    <div className="alert alert-error">
      <p>{deviceStatus}</p>
      {/* Provide solution based on error type */}
      <button onClick={() => setShowAlert(false)}>Got it</button>
    </div>
  );
}

Detailed Explanation: See Vanilla JS Events - microphone_available

Usage Example

function EventHandler() {
    const [asrText, setAsrText] = useState('');
    const [conversations, setConversations] = useState([]);

    useAaaSPilotKitEvents({
        onReady: () => {
            console.log('Digital Employee ready');
        },

        onAsrMessage: (payload) => {
            setAsrText(payload.text);
            if (payload.completed) {
                console.log('Speech recognition complete:', payload.text);
            }
        },

        onRenderStart: (payload) => {
            console.log('Start broadcasting:', payload.text);
        },

        onConversationChange: (payload) => {
            console.log('Conversation update:', payload);
        },

        onError: (error) => {
            console.error('Error occurred:', error);
            // Handle based on error type
            if (error.severity === 'critical') {
                alert(`Critical error: ${error.message}`);
            }
        }
    });

    return (
        <div>
            <div>Current recognition: {asrText}</div>
            <div>Conversations: {conversations.length}</div>
        </div>
    );
}

Dynamic Event Handling

function DynamicEventHandler() {
    const [logEnabled, setLogEnabled] = useState(true);

    // Event handlers can change dynamically
    useAaaSPilotKitEvents({
        onAsrMessage: logEnabled ?
            (payload) => console.log('ASR:', payload.text) :
            undefined,

        onConversationChange: useCallback((payload) => {
            // Use useCallback to optimize performance
            updateConversationInState(payload);
        }, [])
    });

    return (
        <div>
            <label>
                <input
                    type="checkbox"
                    checked={logEnabled}
                    onChange={(e) => setLogEnabled(e.target.checked)}
                />
                Enable logging
            </label>
        </div>
    );
}

useConversationList

Monitors and maintains local conversation list.

Interface Definition

function useConversationList(): {
    conversationList: AnyConversation[];
}

Return Values

• conversationList: Conversation list array

Usage Example

function ConversationPanel() {
    const {conversationList} = useConversationList();

    return (
        <div className="conversation-panel">
            <div className="header">
                Conversation History: ({conversationList.length})
            </div>
            <div className="conversation-list">
                {conversationList.map((conversation, index) => (
                    <ConversationView
                        key={conversation.id}
                        conversation={conversation}
                    />
                ))}
            </div>
        </div>
    );
}

useConversation

Converts single conversation content to a renderable list.

Interface Definition

function useConversation(conversation: AnyConversation): {
    conversationContents: (NAIWorkerConversationBean.IContentChangePayload & {uiId: string})[];
}

Parameters

• conversation: Single conversation object

Return Values

• conversationContents: Renderable content list, each item contains uiId for React key

Usage Example

// ConversationView is the <ConversationView ... /> in useConversationList example
function ConversationView({conversation}) {
    const {conversationContents} = useConversation(conversation);

    return (
        <div className="conversation">
            <div className="conversation-header">
                {conversation.type === 'client' ? 'User' : 'Digital Employee'}
            </div>
            <div className="conversation-body">
                {conversationContents.map(item => (
                    <div
                        key={item.uiId}
                        className={`content-item ${item.completed ? 'completed' : 'typing'}`}
                    >
                        <span className="mod-label">{item.mod}</span>
                        <div className="content">{item.content}</div>
                        {!item.completed && <span className="typing-indicator">...</span>}
                    </div>
                ))}
            </div>
        </div>
    );
}

Advanced Usage

function EnhancedConversationView({conversation}) {
    const {conversationContents} = useConversation(conversation);

    // Calculate total character count
    const totalChars = conversationContents.reduce(
        (sum, item) => sum + (item.content?.length || 0),
        0
    );

    // Is currently typing
    const isTyping = conversationContents.some(item => !item.completed);

    return (
        <div className="enhanced-conversation">
            <div className="stats">
                Characters: {totalChars} | Status: {isTyping ? 'Typing' : 'Complete'}
            </div>
            {conversationContents.map(item => (
                <div key={item.uiId} className="content-item">
                    {item.content}
                </div>
            ))}
        </div>
    );
}

Combining with Other Hooks

function CompleteConversationSystem() {
    const {controller, isReady} = useAaaSPilotKit();
    const {conversationList} = useConversationList();
    const [selectedIndex, setSelectedIndex] = useState(-1);

    // Auto-select latest conversation
    useEffect(
        () => {
            if (conversationList.length > 0) {
                setSelectedIndex(conversationList.length - 1);
            }
        },
        [conversationList.length]
    );

    const selectedConversation = conversationList[selectedIndex];

    return (
        <div className="conversation-system">
            <div className="sidebar">
                {conversationList.map((conv, index) => (
                    <div
                        key={conv.id || index}
                        className={`conversation-item ${index === selectedIndex ? 'active' : ''}`}
                        onClick={() => setSelectedIndex(index)}
                    >
                        {conv.type === 'client' ? 'User' : 'AI'} - {new Date(conv.timestamp).toLocaleTimeString()}
                    </div>
                ))}
            </div>
            <div className="main">
                {selectedConversation && (
                    <ConversationView conversation={selectedConversation} />
                )}
            </div>
        </div>
    );
}

useConversationEnd

Monitors AIWorkerConversationBean's conversation_end event.

Interface Definition

function useConversationEnd(
    conversation: AIWorkerConversationBean | null,
    onConversationEnd?: (payload: NAIWorkerConversationBean.IConversationEndPayload) => void
): void

Parameters

Parameter	Type	Description
conversation	AIWorkerConversationBean | null	Conversation instance to monitor
onConversationEnd	(payload) => void	Callback function when conversation completes

Payload Type

interface IConversationEndPayload {
    id: string;
    text: string;
    contents: IContent[];
    timestamp: number;
}

Usage Example

function ConversationStatus({ conversation }) {
    const [status, setStatus] = useState('In progress');

    useConversationEnd(
        conversation instanceof AIWorkerConversationBean ? conversation : null,
        (payload) => {
            setStatus('Completed');
            console.log('Conversation ended, total characters:', payload.text.length);
        }
    );

    return <div className="status-badge">{status}</div>;
}