The processing pipeline
When you upload a document to Upstream, it goes through a multi-step pipeline that turns raw text into structured, searchable knowledge. Here's what happens at each stage.
1. Upload
Your document is received and added to the processing queue. You'll see it appear on the Upload page with a status indicator.
2. Parse
Upstream extracts the text content from your document. Supported file types include:
- DOCX (Microsoft Word)
- TXT (plain text)
- And other common document formats
The parser handles formatting, tables, and multi-page documents automatically.
3. Transcripts
Upstream works with transcripts you already have. It does not process audio or video, and it does not detect or identify speakers on its own.
Speaker names in your transcript (for example, Sarah: We had trouble importing clients) are kept in the text. They help Upstream attribute what was said during extraction. For WebVTT (.vtt) files, timestamps are removed and the speaker names and spoken text are kept. If your transcript has no speaker names, there is nothing to attribute from, so extracted items may not say who raised them.
4. Entity extraction
Upstream analyzes your text for knowledge items such as pain points, objections and feature requests. Items can carry a confidence score from 0 to 100. This estimate may be adjusted over time; it is not proof that the statement is accurate or supported by its linked source.
5. Relationship mapping
Connections between entities are discovered automatically. For example, Upstream might identify that a Pain Point leads_to a Feature Request, or that Customer Feedback supports a Market Insight.
6. Embedding generation
Your content is indexed for semantic search. This means when you ask questions in chat, Upstream can find relevant information even if you use different words than the original document.
7. Review
Depending on the import path and configured threshold, some extracted items are automatically approved; others wait in the Review Queue. Review-isolated imports keep their suggestions pending regardless of score. Check the displayed status rather than assuming a score means someone reviewed an item.
Document labels and generated tags
For newly uploaded or imported documents, tags supplied with the source (including
GitHub labels and the github-issue marker) are retained alongside current
AI-generated topic tags. Repeat processing replaces previous AI topics rather
than accumulating them. Empty or failed AI tagging leaves only the source tags.
An accepted source update replaces the source label set, including label removals.
This is forward-only: previously lost labels are not recovered. Source categories remain a separate classification, and this does not rewrite retained chunk or search-index metadata.
Enrichment summaries
After extraction, Upstream can add summaries and tags to entities. For document enrichment, the model receives an excerpt of the original text, which may not include the whole document. A source link, confidence score or automatic approval does not guarantee that a generated assertion is supported by the source.
Enrichment summaries are limited to 200 characters. If a returned summary is blank, malformed or too long, Upstream skips that entity's enrichment update instead of cutting off the summary. The extracted entity remains; any previous enrichment summary and tags remain unchanged and are not automatically corrected.
Check processing status
During processing, the Upload screen shows these stages: Read document text, Split into sections, Prepare search, Find knowledge items, Add topic tags, and Finish processing. The percentage reflects the stage reached, not elapsed time or how much work remains. Finishing processing does not mean a person verified the results.
Back to uploads leaves the status view. It does not cancel an upload request or stop server-side processing. Use Knowledge → Sources to find a saved document again.
What to do if processing seems stuck
Processing usually completes within a few minutes. If a document appears stuck:
- Wait a few more minutes. Larger documents and complex PDFs take longer to process.
- Refresh the page. The status may have updated.
- Re-upload the document. If the status hasn't changed after 15 minutes, try uploading the document again.
- Check the file. Make sure the document isn't corrupted, password-protected, or an unsupported format.
Tips for documents that extract well
Not all documents are created equal. You'll get better results with:
- Clear structure. Documents with headings, bullet points, and short paragraphs produce more accurate extractions.
- Specific language. Concrete statements ("Customers say pricing is confusing") extract better than vague ones ("There are some concerns").
- Transcripts with speaker labels. If you're uploading a transcript, export it with speaker names included. Upstream relies on the names in your transcript for attribution and does not add them itself.
- Focused content. A single-topic document often extracts better than a long, multi-topic one. Consider splitting large documents into smaller, focused files.
Related articles
Still need help?
Email us at hello@getupstream.ai — we read every message.