Skip to content

feat: Add multi-modal chat support (file uploads + voice input)#51

Open
Isaccseven wants to merge 2 commits into
langflow-ai:mainfrom
Isaccseven:feat/multimodal-chat
Open

feat: Add multi-modal chat support (file uploads + voice input)#51
Isaccseven wants to merge 2 commits into
langflow-ai:mainfrom
Isaccseven:feat/multimodal-chat

Conversation

@Isaccseven
Copy link
Copy Markdown

Summary

Adds file upload and voice-to-text input capabilities to the embedded chat widget, addressing #27.

File Upload

  • Paperclip button opens native file picker (hidden when file_upload prop is false)
  • Selected files show as previews above the input area (image thumbnails or file icon + name)
  • Files uploaded to Langflow server via POST /api/v1/files/upload/{flow_id} on send
  • File paths included in tweaks object targeting the file_component prop
  • Multiple files supported; sequential upload with error handling
  • Upload failure shows error message and prevents message send

Voice Input

  • Microphone button activates browser's Web Speech API (SpeechRecognition)
  • Button automatically hidden in browsers without support (Firefox)
  • Transcribed text appended to input field for review before sending
  • Pulsing red animation during active recording
  • Optional voice_language prop for BCP 47 language code

New Props

Prop Type Default Description
file_upload boolean false Enable file attachment button
file_component string - Flow component ID that accepts files
voice_input boolean false Enable voice-to-text button
voice_language string Browser default Speech recognition language

Files Changed

  • src/index.tsx — New prop registrations
  • src/chatWidget/index.tsx — Pass props + new CSS classes
  • src/chatWidget/chatWindow/index.tsx — File picker, voice input, upload integration
  • src/controllers/uploadFiles.ts — New file upload controller
  • src/types/chatWidget/index.ts — FileAttachment type
  • README.md — Document new props

Testing

  • Build compiles successfully (npm run build:react + npm run build:bundle)
  • Both features default to false — existing behavior unchanged
  • File upload: set file_upload="true" and file_component="<your-component-id>"
  • Voice input: set voice_input="true" (test in Chrome/Edge)

Closes #27

Add file upload and voice-to-text input capabilities to the chat widget:

- File upload: paperclip button opens native file picker, files uploaded
  to Langflow server via POST /api/v1/files/upload/{flow_id}, file
  references included in tweaks object for the configured component
- Voice input: microphone button uses Web Speech API for speech-to-text,
  transcribed text inserted into input field for review before sending
- Both features opt-in via boolean props (file_upload, voice_input)
- New props: file_component (target component ID), voice_language (BCP 47)
- File previews with thumbnails for images, remove button for each file
- Voice button hidden in browsers without SpeechRecognition support
- Pulsing animation on mic button during active recording

Closes langflow-ai#27
@Isaccseven
Copy link
Copy Markdown
Author

Ready for review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Any plan to support multi-modal chat?

1 participant