Skip to content
JMJahanzeb Mohsin
All work

Case study Nº 02

Web app

ScopeSignal

Catches scope creep in client Slack channels before anyone says yes to it.

Type
Web app
Stack
Next.js · TypeScript · Supabase · pgvector · OpenAI · Slack API

The live demo is the app's own sandbox: the real interface with preloaded data. No sign-up, and nothing is saved.

ScopeSignal flagging a request for paid media work as out of scope, with the matching contract clause and its confidence
Fig. 1 · ScopeSignal flagging a request for paid media work as out of scope, with the matching contract clause and its confidence

ScopeSignal reads an agency's signed proposal, watches the client's Slack channels, and warns the account manager when a request looks like work nobody agreed to. Each alert quotes the contract clause it relied on.

1

What it does

An agency uploads its signed proposal and any addendums. ScopeSignal then watches the Slack channels where that client posts. When an approved client contact asks for something, it decides whether the request is in scope, out of scope, or unclear, and explains why with the clause it used.

Out-of-scope and unclear requests show up in an alert inbox and go to the account manager as a Slack DM or an email. Feedback on each alert is saved so false positives can be tuned out. There is also a sandbox for trying the classifier without connecting Slack.

2

How it works

Proposals are parsed, split into chunks and embedded into pgvector. Each chunk keeps its effective date, so a newer signed addendum outranks the older clause it replaces.

Slack messages arrive through a signed Events API endpoint, get filtered (watched channel, approved client contact) and land in a job queue. A background worker pulls the recent thread for context and the most relevant contract chunks, then asks the model for a verdict, a confidence score and a citation.

3

Measured, not guessed

The PDF parser was picked after comparing pdf-parse and pdfjs-dist on seven public proposals, RFPs and contracts. Retrieval settings came from a sweep over 20 labelled client questions.

For the model, gpt-5-mini and gpt-5.2 both scored 90% on the test set, and the smaller one cost about 40% as much, so it is the default. The full numbers are in the repo.

4

When it is not sure

Anything under 0.6 confidence goes to a person instead of being called. So do messages with no matching contract text, messages that are not in English, and wording like "maybe" or "not sure".

On the data side, every Supabase table has row-level security, Slack tokens are stored encrypted, and worker logs never contain contract text or Slack messages.

□

Plates

Alert list with the verdict, confidence, status, source channel and the contract evidence for each client request
Fig. 2 · Alert list with the verdict, confidence, status, source channel and the contract evidence for each client request
An indexed proposal and its addendum, split into chunks for retrieval
Fig. 3 · An indexed proposal and its addendum, split into chunks for retrieval
The Slack channels and client contacts ScopeSignal watches
Fig. 4 · The Slack channels and client contacts ScopeSignal watches